Create matplotlib visualizations from the command-line

Overview

MatplotCLI

Create matplotlib visualizations from the command-line

MatplotCLI is a simple utility to quickly create plots from the command-line, leveraging Matplotlib.

plt "scatter(x,y,5,alpha=0.05); axis('scaled')" < sample.json

plt "hist(x,30)" < sample.json

MatplotCLI accepts both JSON lines and arrays of JSON objects as input. Look at the recipes section to learn how to handle other formats like CSV.

MatplotCLI executes python code (passed as argument) where some handy imports are already done (e.g. from matplotlib.pyplot import *) and where the input JSON data is already parsed and available in variables, making plotting easy. Please refer to matplotlib.pyplot's reference and tutorial for comprehensive documentation about the plotting commands.

Data from the input JSON is made available in the following way. Given the input myfile.json:

{"a": 1, "b": 2}
{"a": 10, "b": 20}
{"a": 30, "c$d": 40}

The following variables are made available:

data = {
    "a": [1, 10, 30],
    "b": [2, 20, None],
    "c_d": [None, None, 40]
}

a = [1, 10, 30]
b = [2, 20, None]
c_d = [None, None, 40]

col_names = ("a", "b", "c_d")

So, for a scatter plot a vs b, you could simply do:

plt "scatter(a,b); title('a vs b')" < myfile.json

Notice that the names of JSON properties are converted into valid Python identifiers whenever they are not (e.g. c$d was converted into c_d).

Execution flow

  1. Import matplotlib and other libs;
  2. Read JSON data from standard input;
  3. Execute user code;
  4. Show the plot.

All steps (except step 3) can be skipped through command-line options.

Installation

The easiest way to install MatplotCLI is from pip:

pip install matplotcli

Recipes and Examples

Plotting JSON data

MatplotCLI natively supports JSON lines:

echo '
    {"a":0, "b":1}
    {"a":1, "b":0}
    {"a":3, "b":3}' |
plt "plot(a,b)"

and arrays of JSON objects:

echo '[
    {"a":0, "b":1},
    {"a":1, "b":0},
    {"a":3, "b":3}]' |
plt "plot(a,b)"

Plotting from a csv

SPyQL is a data querying tool that allows running SQL queries with Python expressions on top of different data formats. Here, SPyQL is reading a CSV file, automatically detecting if there's an header row, the dialect and the data type of each column, and converting the output to JSON lines before handing over to MatplotCLI.

cat my.csv | spyql "SELECT * FROM csv TO json" | plt "plot(x,y)"

Plotting from a yaml/xml/toml

yq converts yaml, xml and toml files to json, allowing to easily plot any of these with MatplotCLI.

cat file.yaml | yq -c | plt "plot(x,y)"
cat file.xml | xq -c | plt "plot(x,y)"
cat file.toml | tomlq -c | plt "plot(x,y)"

Plotting from a parquet file

parquet-tools allows dumping a parquet file to JSON format. jq -c makes sure that the output has 1 JSON object per line before handing over to MatplotCLI.

parquet-tools cat --json my.parquet | jq -c | plt "plot(x,y)"

Plotting from a database

Databases CLIs typically have an option to output query results in CSV format (e.g. psql --csv -c query for PostgreSQL, sqlite3 -csv -header file.db query for SQLite).

Here we are visualizing how much space each namespace is taking in a PostgreSQL database. SPyQL converts CSV output from the psql client to JSON lines, and makes sure there are no more than 10 items, aggregating the smaller namespaces in an All others category. Finally, MatplotCLI makes a pie chart based on the space each namespace is taking.

psql -U myuser mydb --csv  -c '
    SELECT
        N.nspname,
        sum(pg_relation_size(C.oid))*1e-6 AS size_mb
    FROM pg_class C
    LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
    GROUP BY 1
    ORDER BY 2 DESC' |
spyql "
    SELECT
        nspname if row_number < 10 else 'All others' as name,
        sum_agg(size_mb) AS size_mb
    FROM csv
    GROUP BY 1
    TO json" |
plt "
nice_labels = ['{0}\n{1:,.0f} MB'.format(n,s) for n,s in zip(name,size_mb)];
pie(size_mb, labels=nice_labels, autopct='%1.f%%', pctdistance=0.8, rotatelabels=True)"

Plotting a function

Disabling reading from stdin and generating the output using numpy.

plt --no-input "
x = np.linspace(-1,1,2000);
y = x*np.sin(1/x);
plot(x,y);
axis('scaled');
grid(True)"

Saving the plot to an image

Saving the output without showing the interactive window.

cat sample.json |
plt --no-show "
hist(x,30);
savefig('myimage.png', bbox_inches='tight')"

Plot of the global temperature

Here's a complete pipeline from getting the data to transforming and plotting it:

  1. Downloading a CSV file with curl;
  2. Skipping the first row with sed;
  3. Grabbing the year column and 12 columns with monthly temperatures to an array and converting to JSON lines format using SPyQL;
  4. Exploding the monthly array with SPyQL (resulting in 12 rows per year) while removing invalid monthly measurements;
  5. Plotting with MatplotCLI .
curl https://data.giss.nasa.gov/gistemp/tabledata_v4/GLB.Ts+dSST.csv |
sed 1d |
spyql "
  SELECT Year, cols[1:13] AS temps
  FROM csv
  TO json" |
spyql "
  SELECT
    json->Year + ((row_number-1)%12)/12 AS year,
    json->temps AS temp
  FROM json
  EXPLODE json->temps
  WHERE json->temps is not Null
  TO json" |
plt "
scatter(year, temp, 2, temp);
xlabel('Year');
ylabel('Temperature anomaly w.r.t. 1951-80 (ºC)');
title('Global surface temperature (land and ocean)')"

You might also like...
These data visualizations were created for my introductory computer science course using Python
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

These data visualizations were created as homework for my CS40 class. I hope you enjoy!
These data visualizations were created as homework for my CS40 class. I hope you enjoy!

Data Visualizations These data visualizations were created as homework for my CS40 class. I hope you enjoy! Nobel Laureates by their Country of Birth

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

A Python package for caclulations and visualizations in geological sciences.

geo_calcs A Python package for caclulations and visualizations in geological sciences. Free software: MIT license Documentation: https://geo-calcs.rea

Make scripted visualizations in blender
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Standardized plots and visualizations in Python
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

Visualizations of some specific solutions of different differential equations.
Visualizations of some specific solutions of different differential equations.

Diff_sims Visualizations of some specific solutions of different differential equations. Heat Equation in 1 Dimension (A very beautiful and elegant ex

Data aggregated from the reports found at the MCPS COVID Dashboard into a set of visualizations.

Montgomery County Public Schools COVID-19 Visualizer Contents About this project Data Support this project About this project Data All data we use can

Comments
  • stats about input data

    stats about input data

    option to print simple statistics about the input data. e.g. for each field

    • number of missing values
    • number of distinct values
    • avg, min, max (if numeric)
    • number of nan, inf (if float)
    • ...
    enhancement good first issue 
    opened by dcmoura 0
Releases(v0.2.0)
Owner
Daniel Moura
Daniel Moura
China and India Population and GDP Visualization

China and India Population and GDP Visualization Historical Population Comparison between India and China This graph shows the population data of Indi

Nicolas De Mello 10 Oct 27, 2021
A simple code for plotting figure, colorbar, and cropping with python

Python Plotting Tools This repository provides a python code to generate figures (e.g., curves and barcharts) that can be used in the paper to show th

Guanying Chen 134 Jan 02, 2023
Wikipedia WordCloud App generate Wikipedia word cloud art created using python's streamlit, matplotlib, wikipedia and wordcloud packages

Wikipedia WordCloud App Wikipedia WordCloud App generate Wikipedia word cloud art created using python's streamlit, matplotlib, wikipedia and wordclou

Siva Prakash 5 Jan 02, 2022
Collection of scripts for making high quality beautiful math-related posters.

Poster Collection of scripts for making high quality beautiful math-related posters. The poster can have as large printing size as 3x2 square feet wit

Nattawut Phetmak 3 Jun 09, 2022
Some problems of SSLC ( High School ) before outputs and after outputs

Some problems of SSLC ( High School ) before outputs and after outputs 1] A Python program and its output (output1) while running the program is given

Fayas Noushad 3 Dec 01, 2021
Create charts with Python in a very similar way to creating charts using Chart.js

Create charts with Python in a very similar way to creating charts using Chart.js. The charts created are fully configurable, interactive and modular and are displayed directly in the output of the t

Nicolas H 68 Dec 08, 2022
Pglive - Pglive package adds support for thread-safe live plotting to pyqtgraph

Live pyqtgraph plot Pglive package adds support for thread-safe live plotting to

Martin Domaracký 15 Dec 10, 2022
Graphing communities on Twitch.tv in a visually intuitive way

VisualizingTwitchCommunities This project maps communities of streamers on Twitch.tv based on shared viewership. The data is collected from the Twitch

Kiran Gershenfeld 312 Jan 07, 2023
Backend app for visualizing CANedge log files in Grafana (directly from local disk or S3)

CANedge Grafana Backend - Visualize CAN/LIN Data in Dashboards This project enables easy dashboard visualization of log files from the CANedge CAN/LIN

13 Dec 15, 2022
Time series visualizer is a flexible extension that provides filling world map by country from real data.

Time-series-visualizer Time series visualizer is a flexible extension that provides filling world map by country from csv or json file. You can know d

Long Ng 3 Jul 09, 2021
A tool for creating SVG timelines from simple JSON input.

A tool for creating SVG timelines from simple JSON input.

Jason Reisman 432 Dec 30, 2022
Parallel t-SNE implementation with Python and Torch wrappers.

Multicore t-SNE This is a multicore modification of Barnes-Hut t-SNE by L. Van der Maaten with python and Torch CFFI-based wrappers. This code also wo

Dmitry Ulyanov 1.7k Jan 09, 2023
This is my favourite function - the Rastrigin function.

This is my favourite function - the Rastrigin function. What sparked my curiosity and interest in the function was its complexity in terms of many local optimum points, which makes it particularly in

1 Dec 27, 2021
Streamlit component for Let's-Plot visualization library

streamlit-letsplot This is a work-in-progress, providing a convenience function to plot charts from the Lets-Plot visualization library. Example usage

Randy Zwitch 9 Nov 03, 2022
Productivity Tools for Plotly + Pandas

Cufflinks This library binds the power of plotly with the flexibility of pandas for easy plotting. This library is available on https://github.com/san

Jorge Santos 2.7k Dec 30, 2022
Comparing USD and GBP Exchange Rates

Currency Data Visualization Comparing USD and GBP Exchange Rates This is a bar graph comparing GBP and USD exchange rates. I chose blue for the UK bec

5 Oct 28, 2021
Automatically Visualize any dataset, any size with a single line of code. Created by Ram Seshadri. Collaborators Welcome. Permission Granted upon Request.

AutoViz Automatically Visualize any dataset, any size with a single line of code. AutoViz performs automatic visualization of any dataset with one lin

AutoViz and Auto_ViML 1k Jan 02, 2023
Automatically visualize your pandas dataframe via a single print! 📊 💡

A Python API for Intelligent Visual Discovery Lux is a Python library that facilitate fast and easy data exploration by automating the visualization a

Lux 4.3k Dec 28, 2022
A Python Library for Self Organizing Map (SOM)

SOMPY A Python Library for Self Organizing Map (SOM) As much as possible, the structure of SOM is similar to somtoolbox in Matlab. It has the followin

Vahid Moosavi 497 Dec 29, 2022
Altair extension for saving charts in a variety of formats.

Altair Saver This packge provides extensions to Altair for saving charts to a variety of output types. Supported output formats are: .json/.vl.json: V

Altair 85 Dec 09, 2022