ioztat is a storage load analysis tool for OpenZFS

Related tags

Miscellaneousioztat
Overview

ioztat

ioztat is a storage load analysis tool for OpenZFS. It provides iostat-like statistics at an individual dataset/zvol level.

The statistics offered are read and write operations per second, read and write throughput per second, and the average size (in kilobytes) of read and write operations issued in the current reporting interval. Viewing these statistics at the individual dataset level allows system administrators to identify storage "hot spots" in larger multi-tenant systems--particularly those with many VMs or containers operating essentially independent workloads.

This sample output shows activity which has taken place in the most recent second, on a the ssd zpool of a ZFS virtualization host:

[email protected]:~# ioztat -y -c1 ssd
dataset                                         w/s      wMB/s        r/s      rMB/s   wareq-sz   rareq-sz
ssd                                            0.00       0.00       0.00       0.00       0.00       0.00
   images                                      0.00       0.00       0.00       0.00       0.00       0.00
      DC1                                     17.96       0.10       3.99       0.05       5.66      12.29
      DC2                                     21.95       0.14       0.00       0.00       6.59       0.00
      QB                                       2.00       0.01       0.00       0.00       7.17       0.00
      SAP-TC                                   2.99       0.02       0.00       0.00       6.83       0.00
      SAP4-WIN2019                             2.99       0.05       0.00       0.00      17.07       0.00
      nagios                                   0.00       0.00       0.00       0.00       0.00       0.00
      qemu                                     0.00       0.00       0.00       0.00       0.00       0.00
         autostart                             0.00       0.00       0.00       0.00       0.00       0.00
   iso                                         0.00       0.00       0.00       0.00       0.00       0.00
   unsnapped                                   0.00       0.00       0.00       0.00       0.00       0.00
      rp9                                      0.00       0.00       0.00       0.00       0.00       0.00

For the most part, ioztat behaves the same way that the system standard iostat tool does, with similar arguments.

usage: ioztat [-h] [-s SORT] [-i INTERVAL] [-c COUNT] [-y] [-b]
              pool [pool ...]

iostat for ZFS datasets

positional arguments:
  pool         ZFS pool

optional arguments:
  -h, --help   show this help message and exit
  -s SORT      Sort by: name / wps / wMBps / rps / rMBps
  -i INTERVAL  Time between each report
  -c COUNT     Number of reports generated
  -y           Skip the initial "summary" report
  -b           Use binary (power-of-two) prefixes

The only required argument is the name of at least one zpool to monitor. Without any other arguments, ioztat first prints a summary record showing activity per dataset since the most recent system boot, then prints a new record showing the most recent activity once per second. The -i argument can be used to change the report interval, and the -c argument can be used to limit ioztat to a certain number of intervals before exiting.

For those who wish a continually-updated, easy to read summary of pool activity, watch -n1 ioztat poolname -c1 -y will suit nicely.

Comments
  • Rework formatting

    Rework formatting

    This makes ioztat output more closely resemble zpool iostat:

                                  operations      bandwidth         opsize
    dataset                       read   write    read   write    read   write
    --------------------------  ------  ------  ------  ------  ------  ------
    rpool
       ROOT
          default                   23       0   62.6K       0    2.7K     759
       cvs                           0       0    1.6K     892   24.2K  108.4K
       home                          0       0       0       0       0       0
          freaky                     3       2   29.7K    8.3K   10.2K    3.9K
    

    This also adds -e for exact values (-p in zpool iostat, which we're currently using elsewhere) and -H for scripted mode that drops the header and separates fields by tabs.

    The intent is for this to eventually fix #23, probably by adjusting the space for dataset.

    opened by Freaky 12
  • Painting the bikeshed

    Painting the bikeshed

    Should bandwidth be changed to throughput? This is more correct, but deviates from zpool-iostat(8).

    Should we default to binary mode, since that's all zpool-iostat(8) supports?

    Should we default to a single iteration? This is what iostat(1) and zpool-iostat(8) both do. They also accept interval and count as positional arguments.

    Is + as an ellipsis for truncated dataset names reasonable, or should it be ... despite the additional space it uses?

    opened by Freaky 8
  • Overwrite old reports in terminal by default

    Overwrite old reports in terminal by default

    I thought that this script put a little too much into the terminal. This PR makes it so that only the most recent report is visible, which can be disabled with the new -n argument.

    opened by Aluminite 4
  • No warning if zfs version too low (was: doesn't find datasets on Debian Buster, zfs 0.7.12-2)

    No warning if zfs version too low (was: doesn't find datasets on Debian Buster, zfs 0.7.12-2)

    ioztat -Hey as suggested by the man page doesn't return anything.

    ioztat -SIn rpool, also suggested by the man page, gives "dataset not found: 'rpool'" (rpool of course exists).

    Is there a way to further debug this? Am I missing a dependency?

    opened by steffen-AEI 3
  • Make numeric precision variable

    Make numeric precision variable

    Matching ZFS number formatting behaviour, use between 0 and 2 decimal places according to available space, and further reduce the column size to again match zpool-iostat and make more room for dataset names.

    This includes the careful rounding behaviour of the previous implementation, which I believe ZFS lacks.

    opened by Freaky 3
  • Fix -y when not combined with -c

    Fix -y when not combined with -c

    Fix an error when args.count is None. While I'm here also fix a count of zero that was ignored due to zero being falsey.

    Simplify the logic slightly by changing skipsummary from a bool to a generic skip count.

    opened by Freaky 3
  • Output is insensitive to terminal width

    Output is insensitive to terminal width

    The current output width is 106 columns, which by happy coincidence is precisely the width of my usual terminals.

    This coincidence seems unlikely to be universal, and some sensitivity to this would seem to be in order.

    opened by Freaky 3
  • Fix and simplify overwrite mode

    Fix and simplify overwrite mode

    This fixes overwrite mode by replacing the line counting and iterated cursor repositioning with a clear screen and placing the cursor at 1,1 - exactly what GNU watch does.

    I've also done some tidying, hoisting some iteration and filtering logic into dedicated generator functions which helps to further simplify the main loop.

    opened by Freaky 3
  • python3 shebang--does this work in FreeBSD?

    python3 shebang--does this work in FreeBSD?

    @Freaky, I discovered after merging your patches today that ioztat does not work on the Python2.7 that my Ubuntu workstation at the home office provides with "env python". I had to update the shebang to "env python3" to get it to work again.

    This should be fine for Ubuntu, but does it work in FreeBSD-land as well? I don't know if there's a python3 executable in your environment, or whether there is only python and you get what you get.

    opened by jimsalterjrs 3
  • Format name with all intermediates for disambiguation

    Format name with all intermediates for disambiguation

    Fixes #5

    This replaces the "just-the-rightmost-dataset-name-component" logic by "components-of-the-name-unique-after-the-previous-one". Output with default options looks like

    dataset                                         w/s      wMB/s        r/s      rMB/s   wareq-sz   rareq-sz
    zroot                                          0.00       0.00       0.00       0.00       0.00       0.00
       ROOT/default                                0.00       0.00       3.88       0.00       0.00       0.16
       tmp                                         0.00       0.00       0.00       0.00       0.00       0.00
    

    (e.g. here instead of default being indented two, and ROOT being invisible, we get the dataset-name-components that distinguish it from zroot, the previously-printed name)

    opened by adriaandegroot 3
  • Unmounted datasets lead to confusing gaps in output

    Unmounted datasets lead to confusing gaps in output

    Stats for unmounted datasets aren't exported, so you can end up with incomplete hierarchies in ioztat output. For example here rpool and rpool/ROOT aren't mounted, leaving rpool/ROOT/default without the path leading up to it:

    dataset                                         w/s      wMB/s        r/s      rMB/s   wareq-sz   rareq-sz
          default                                  0.00       0.00       0.97       0.00       0.00       0.05
    

    Similarly, rpool/usr/local/poudriere/ports/default misses the mountpoint-less poudriere/ports segments:

       usr                                         0.00       0.00       0.97       0.00       0.00       0.01
          local                                    0.00       0.00       0.00       0.00       0.00       0.00
                   default                         0.00       0.00       0.00       0.00       0.00       0.00
    
    opened by Freaky 3
  • using dataclasses

    using dataclasses

    Hi,

    I started of my own version of this, but found yours basically the last moment … My name of choice was objset, but the poll is finished I guess.

    I used dataclasses - ported here. And updated the fields to match current OpenZFS git.

    opened by commonism 7
  • no activity reported for zfs replication

    no activity reported for zfs replication

    is it expected that ioztat does not seem to report activity for zfs replication ?

    if this is not a bug but by design, maybe it's worth mentioning in the

    opened by devZer0 0
  • Packages

    Packages

    Produce .deb/.rpm/etc packages. sanoid is already doing this. I'm currently using sanoid package in latest ubuntu release.

    Getting these included will take some time but is worthwhile I think.

    enhancement help wanted 
    opened by morganchristiansson 2
Releases(v2.0.1)
  • v2.0.1(Mar 4, 2022)

  • v2.0.0(Mar 3, 2022)

    Added

    • An iostat(8) man page
    • CHANGELOG.md, this Changelog
    • -e and -H flags for use in scripting [#26]
    • -I flag for totals since last update instead of per-second
    • -o flag to overwrite prior reports [#9] [#20]
    • -P and -p flags to override dataset name display [#19]
    • -S flag for including child dataset statistics in parents [#32]
    • -T d|u flag for adding a timestamp to each report [#25]
    • -V, --version flags [#13] [#14]
    • -x flag for extended statistics, including unlink counts under -xx [#33]
    • interval and count positional arguments [#31]

    Changed

    • Shebang line is now /usr/bin/env python3 [#18]
    • Formatting has changed from resembling GNU iostat(8) to ZFS zpool-iostat(8) [#26]
    • Header is now printed periodically on a tty unless -N is specified [#26]
    • Display is clamped to the terminal width by truncating dataset names if necessary [#26]
    • Sort field names have been changed, with fallbacks for compatibility [#26]
    • Sort options are now case-insensitive [#26]
    • count now defaults to 1 unless an interval is specified [#31]
    • dataset is now an optional argument [#20]
    • Binary (1024-based) formatting is now default, with new -D flag for decimal
    • Average I/O sizes are now hidden beyind -x flag by default to reduce clutter [#33]
    • Exit with an error if a requested dataset does not exist or is not mounted

    Removed

    • -b flag. Binary mode is now the default to match other iostat tools.

    Fixed

    • count and interval must now be positive [#31]
    • WIFSIGNALED() status is now propagated properly to caller
    • Sleep interval now adjusts to compensate for runtime
    • Unhandled BrokenPipeError exception on SIGPIPE
    • Unhandled FileNotFoundError exception on Linux if a dataset is destroyed while enumerating files in /proc
    • Unhandled CalledProcessError exception on FreeBSD if a requested pool does not exist when using sysctl(8) fallback
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Jan 20, 2022)

    Thanks to the efforts of @Freaky and @adriaandegroot, ioztat has new features, improved output readability, and now works on FreeBSD as well as Linux!

    New features include initial selection at the dataset level rather than pool level, optional suppression of zero-activity datasets with -z, and optional suppression of recursion into child datasets with -n.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Jan 12, 2022)

    The first release of ioztat builds on efforts from the Reddit r/zfs community, including:

    u/55rzs (initial creation) u/d1722825 (substantial refactoring and cleanup) u/mercenary_sysadmin (addition of the -y flag to allow for easy use with the GNU watch command)

    Enjoy!

    Source code(tar.gz)
    Source code(zip)
Owner
Jim Salter
Jim Salter
Pixelarticons - Pixel Art Icons made simple for Flutter, powered by pixelarticons and fontify

Pixelarticons - Pixel Art Icons made simple for Flutter, powered by pixelarticons and fontify

lask 16 Dec 12, 2022
Fofa asset consolidation script

资产收集+C段整理二合一 基于fofa资产搜索引擎进行资产收集,快速检索目标条件下的IP,URL以及标题,适用于资产较多时对模糊资产的快速检索,新增C段整理功能,整理出

白泽Sec安全实验室 36 Dec 01, 2022
An advanced NFT Generator

NFT Generator An advanced NFT Generator Free software: GNU General Public License v3 Documentation: https://nft-generator.readthedocs.io. Features TOD

NFT Generator 5 Apr 21, 2022
AMTIO aka All My Tools in One

AMTIO AMTIO aka All My Tools In One. I plan to put a bunch of my tools in this one repo since im too lazy to make one big tool. Installation git clone

osintcat 3 Jul 29, 2021
K2HASH Python library - NoSQL Key Value Store(KVS) library

k2hash_python Overview k2hash_python is an official python driver for k2hash. Install Firstly you must install the k2hash shared library: curl -o- htt

Yahoo! JAPAN 3 Oct 19, 2022
AndroidEnv is a Python library that exposes an Android device as a Reinforcement Learning (RL) environment.

AndroidEnv is a Python library that exposes an Android device as a Reinforcement Learning (RL) environment.

DeepMind 814 Dec 26, 2022
Displays Christmas-themed ASCII art

Christmas Color Scripts Displays Christmas-themed ASCII art. This was mainly inspired by DistroTube's Shell Color Scripts Screenshots ASCII Shadow Tex

1 Aug 09, 2022
You can change your mac address with this program.

1 - Warning! You can use this program with Kali Linux. Therefore if you don't install the Kali Linux. Firstly you need to install Kali Linux. 2 - Star

Mustafa Bahadır Doğrusöz 1 Jun 10, 2022
A tool that automatically creates fuzzing harnesses based on a library

AutoHarness is a tool that automatically generates fuzzing harnesses for you. This idea stems from a concurrent problem in fuzzing codebases today: large codebases have thousands of functions and pie

261 Jan 04, 2023
Multitrack exporter for OP-Z

Underbridge for OP-Z Multitrack exporter Description Exports patterns and projects individual audio tracks to seperate folders for use in your DAW. Py

Thomas Herrmann 71 Dec 25, 2022
Structured, dependable legos for starknet development.

Structured, dependable legos for starknet development.

Alucard 127 Nov 23, 2022
Scripts for BGC analysis in large MAGs and results of their application to soil metagenomes within Chernevaya Taiga RSF-funded project

Scripts for BGC analysis in large MAGs and results of their application to soil metagenomes within Chernevaya Taiga RSF-funded project

1 Dec 06, 2021
This repository contains various tools useful for offensive operations (reversing, etc) regarding the PE (Portable Executable) format

PE-Tools This repository contains various tools useful for offensive operations (reversing, etc) regarding the PE (Portable Executable) format Install

stark0de 4 Oct 13, 2022
My Solutions to 120 commonly asked data science interview questions.

Data_Science_Interview_Questions Introduction 👋 Here are the answers to 120 Data Science Interview Questions The above answer some is modified based

Milaan Parmar / Милан пармар / _米兰 帕尔马 181 Dec 31, 2022
Mengzhan (John) code for Closed Loop Control system of Sharp Wave Ripples in Hippocampus CA3 region

ClosedLoopControl_Yu Mengzhan (John) code for Closed Loop Control system of Sharp Wave Ripples in Hippocampus CA3 region Creating Python Virtual Envir

Mengzhan (John) Liufu 1 Jan 22, 2022
Just a simple python script to generate graphs of salt state requisites.

saltstatevis Just a simple python script to generate graphs of salt state requisites. Installation Requirements You will need to install graphviz to r

Dwayn Matthies 3 May 04, 2022
🇮🇳 A Indian Flag Animation Project Made With Python

🇮🇳 A Indian Flag Animation Project Made With Python

MuFaz-TG 2 Oct 21, 2022
Simple macOS StatusBar app to remind you to unplug your laptop when sufficiently charged

ChargeMon Simple macOS StatusBar app to monitor battery charge status and remind you to unplug your Mac when the battery is sufficiently charged Overv

Rhet Turnbull 5 Jan 25, 2022
My tools box script for sigma

sigma_python_toolbox My tools box script for sigma purpose My goal is not to replace sigma but to put at disposal the scripts that I think to help me

4 Jun 20, 2022
Rufus port to linux, writed on Python3

Rufus-for-Linux Rufus port to linux, writed on Python3 Программа будет иметь тот же интерфейс что и оригинал, и тот же функционал. Программа создается

10 May 12, 2022