Create matplotlib visualizations from the command-line

Overview

MatplotCLI

Create matplotlib visualizations from the command-line

MatplotCLI is a simple utility to quickly create plots from the command-line, leveraging Matplotlib.

plt "scatter(x,y,5,alpha=0.05); axis('scaled')" < sample.json

plt "hist(x,30)" < sample.json

MatplotCLI accepts both JSON lines and arrays of JSON objects as input. Look at the recipes section to learn how to handle other formats like CSV.

MatplotCLI executes python code (passed as argument) where some handy imports are already done (e.g. from matplotlib.pyplot import *) and where the input JSON data is already parsed and available in variables, making plotting easy. Please refer to matplotlib.pyplot's reference and tutorial for comprehensive documentation about the plotting commands.

Data from the input JSON is made available in the following way. Given the input myfile.json:

{"a": 1, "b": 2}
{"a": 10, "b": 20}
{"a": 30, "c$d": 40}

The following variables are made available:

data = {
    "a": [1, 10, 30],
    "b": [2, 20, None],
    "c_d": [None, None, 40]
}

a = [1, 10, 30]
b = [2, 20, None]
c_d = [None, None, 40]

col_names = ("a", "b", "c_d")

So, for a scatter plot a vs b, you could simply do:

plt "scatter(a,b); title('a vs b')" < myfile.json

Notice that the names of JSON properties are converted into valid Python identifiers whenever they are not (e.g. c$d was converted into c_d).

Execution flow

  1. Import matplotlib and other libs;
  2. Read JSON data from standard input;
  3. Execute user code;
  4. Show the plot.

All steps (except step 3) can be skipped through command-line options.

Installation

The easiest way to install MatplotCLI is from pip:

pip install matplotcli

Recipes and Examples

Plotting JSON data

MatplotCLI natively supports JSON lines:

echo '
    {"a":0, "b":1}
    {"a":1, "b":0}
    {"a":3, "b":3}' |
plt "plot(a,b)"

and arrays of JSON objects:

echo '[
    {"a":0, "b":1},
    {"a":1, "b":0},
    {"a":3, "b":3}]' |
plt "plot(a,b)"

Plotting from a csv

SPyQL is a data querying tool that allows running SQL queries with Python expressions on top of different data formats. Here, SPyQL is reading a CSV file, automatically detecting if there's an header row, the dialect and the data type of each column, and converting the output to JSON lines before handing over to MatplotCLI.

cat my.csv | spyql "SELECT * FROM csv TO json" | plt "plot(x,y)"

Plotting from a yaml/xml/toml

yq converts yaml, xml and toml files to json, allowing to easily plot any of these with MatplotCLI.

cat file.yaml | yq -c | plt "plot(x,y)"
cat file.xml | xq -c | plt "plot(x,y)"
cat file.toml | tomlq -c | plt "plot(x,y)"

Plotting from a parquet file

parquet-tools allows dumping a parquet file to JSON format. jq -c makes sure that the output has 1 JSON object per line before handing over to MatplotCLI.

parquet-tools cat --json my.parquet | jq -c | plt "plot(x,y)"

Plotting from a database

Databases CLIs typically have an option to output query results in CSV format (e.g. psql --csv -c query for PostgreSQL, sqlite3 -csv -header file.db query for SQLite).

Here we are visualizing how much space each namespace is taking in a PostgreSQL database. SPyQL converts CSV output from the psql client to JSON lines, and makes sure there are no more than 10 items, aggregating the smaller namespaces in an All others category. Finally, MatplotCLI makes a pie chart based on the space each namespace is taking.

psql -U myuser mydb --csv  -c '
    SELECT
        N.nspname,
        sum(pg_relation_size(C.oid))*1e-6 AS size_mb
    FROM pg_class C
    LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
    GROUP BY 1
    ORDER BY 2 DESC' |
spyql "
    SELECT
        nspname if row_number < 10 else 'All others' as name,
        sum_agg(size_mb) AS size_mb
    FROM csv
    GROUP BY 1
    TO json" |
plt "
nice_labels = ['{0}\n{1:,.0f} MB'.format(n,s) for n,s in zip(name,size_mb)];
pie(size_mb, labels=nice_labels, autopct='%1.f%%', pctdistance=0.8, rotatelabels=True)"

Plotting a function

Disabling reading from stdin and generating the output using numpy.

plt --no-input "
x = np.linspace(-1,1,2000);
y = x*np.sin(1/x);
plot(x,y);
axis('scaled');
grid(True)"

Saving the plot to an image

Saving the output without showing the interactive window.

cat sample.json |
plt --no-show "
hist(x,30);
savefig('myimage.png', bbox_inches='tight')"

Plot of the global temperature

Here's a complete pipeline from getting the data to transforming and plotting it:

  1. Downloading a CSV file with curl;
  2. Skipping the first row with sed;
  3. Grabbing the year column and 12 columns with monthly temperatures to an array and converting to JSON lines format using SPyQL;
  4. Exploding the monthly array with SPyQL (resulting in 12 rows per year) while removing invalid monthly measurements;
  5. Plotting with MatplotCLI .
curl https://data.giss.nasa.gov/gistemp/tabledata_v4/GLB.Ts+dSST.csv |
sed 1d |
spyql "
  SELECT Year, cols[1:13] AS temps
  FROM csv
  TO json" |
spyql "
  SELECT
    json->Year + ((row_number-1)%12)/12 AS year,
    json->temps AS temp
  FROM json
  EXPLODE json->temps
  WHERE json->temps is not Null
  TO json" |
plt "
scatter(year, temp, 2, temp);
xlabel('Year');
ylabel('Temperature anomaly w.r.t. 1951-80 (ºC)');
title('Global surface temperature (land and ocean)')"

You might also like...
These data visualizations were created for my introductory computer science course using Python
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

These data visualizations were created as homework for my CS40 class. I hope you enjoy!
These data visualizations were created as homework for my CS40 class. I hope you enjoy!

Data Visualizations These data visualizations were created as homework for my CS40 class. I hope you enjoy! Nobel Laureates by their Country of Birth

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

A Python package for caclulations and visualizations in geological sciences.

geo_calcs A Python package for caclulations and visualizations in geological sciences. Free software: MIT license Documentation: https://geo-calcs.rea

Make scripted visualizations in blender
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Standardized plots and visualizations in Python
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

Visualizations of some specific solutions of different differential equations.
Visualizations of some specific solutions of different differential equations.

Diff_sims Visualizations of some specific solutions of different differential equations. Heat Equation in 1 Dimension (A very beautiful and elegant ex

Data aggregated from the reports found at the MCPS COVID Dashboard into a set of visualizations.

Montgomery County Public Schools COVID-19 Visualizer Contents About this project Data Support this project About this project Data All data we use can

Comments
  • stats about input data

    stats about input data

    option to print simple statistics about the input data. e.g. for each field

    • number of missing values
    • number of distinct values
    • avg, min, max (if numeric)
    • number of nan, inf (if float)
    • ...
    enhancement good first issue 
    opened by dcmoura 0
Releases(v0.2.0)
Owner
Daniel Moura
Daniel Moura
Define fortify and autoplot functions to allow ggplot2 to handle some popular R packages.

ggfortify This package offers fortify and autoplot functions to allow automatic ggplot2 to visualize statistical result of popular R packages. Check o

Sinhrks 504 Dec 23, 2022
UNMAINTAINED! Renders beautiful SVG maps in Python.

Kartograph is not maintained anymore As you probably already guessed from the commit history in this repo, Kartograph.py is not maintained, which mean

1k Dec 09, 2022
Gesture controlled media player

Media Player Gesture Control Gesture controller for media player with MediaPipe, VLC and OpenCV. Contents About Setup About A tool for using gestures

Atharva Joshi 2 Dec 22, 2021
Statistical data visualization using matplotlib

seaborn: statistical data visualization Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing

Michael Waskom 10.2k Dec 30, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 01, 2023
Declarative statistical visualization library for Python

Altair http://altair-viz.github.io Altair is a declarative statistical visualization library for Python. With Altair, you can spend more time understa

Altair 8k Jan 05, 2023
Visualize large time-series data in plotly

plotly_resampler enables visualizing large sequential data by adding resampling functionality to Plotly figures. In this Plotly-Resampler demo over 11

PreDiCT.IDLab 604 Dec 28, 2022
GitHub Stats Visualizations : Transparent

GitHub Stats Visualizations : Transparent Generate visualizations of GitHub user and repository statistics using GitHub Actions. ⚠️ Disclaimer The pro

YuanYap 7 Apr 05, 2022
An interactive dashboard for visualisation, integration and classification of data using Active Learning.

AstronomicAL An interactive dashboard for visualisation, integration and classification of data using Active Learning. AstronomicAL is a human-in-the-

45 Nov 28, 2022
Arras.io Highest Scores Over Time Bar Chart Race

Arras.io Highest Scores Over Time Bar Chart Race This repo contains a python script (make_racing_bar_chart.py) that can generate a csv file which can

Road 2 Jan 16, 2022
The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based metabolomics.

MINT (Metabolomics Integrator) The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based m

Sören Wacker 0 May 04, 2022
Investment and risk technologies maintained by Fortitudo Technologies.

Fortitudo Technologies Open Source This package allows you to freely explore open-source implementations of some of our fundamental technologies under

Fortitudo Technologies 11 Dec 14, 2022
Sentiment Analysis application created with Python and Dash, hosted at socialsentiment.net

Social Sentiment Dash Application Live-streaming sentiment analysis application created with Python and Dash, hosted at SocialSentiment.net. Dash Tuto

Harrison 456 Dec 25, 2022
🎨 Python3 binding for `@AntV/G2Plot` Plotting Library .

PyG2Plot 🎨 Python3 binding for @AntV/G2Plot which an interactive and responsive charting library. Based on the grammar of graphics, you can easily ma

hustcc 990 Jan 05, 2023
Simple spectra visualization tool for astronomers

SpecViewer A simple visualization tool for astronomers. Dependencies Python = 3.7.4 PyQt5 = 5.15.4 pyqtgraph == 0.10.0 numpy = 1.19.4 How to use py

5 Oct 07, 2021
3D-Lorenz-Attractor-simulation-with-python

3D-Lorenz-Attractor-simulation-with-python Animação 3D da trajetória do Atrator de Lorenz, implementada em Python usando o método de Runge-Kutta de 4ª

Hevenicio Silva 17 Dec 08, 2022
Make your BSC transaction simple.

bsc_trade_history Make your BSC transaction simple. 中文ReadMe Background: inspired by debank ,Practice my hands on this small project Blog:Crypto-BscTr

foolisheddy 7 Jul 06, 2022
A grammar of graphics for Python

plotnine Latest Release License DOI Build Status Coverage Documentation plotnine is an implementation of a grammar of graphics in Python, it is based

Hassan Kibirige 3.3k Jan 01, 2023
Glue is a python project to link visualizations of scientific datasets across many files.

Glue Glue is a python project to link visualizations of scientific datasets across many files. Click on the image for a quick demo: Features Interacti

675 Dec 09, 2022
DrawBot lets you draw images taken from the internet on Skribbl.io, Gartic Phone and Paint

DrawBot You don't speak french? No worries, english translation is over here. C'est quoi ? DrawBot est un logiciel codé par V2F qui va prendre possess

V2F 205 Jan 01, 2023