display: a browser-based graphics server

Overview

display: a browser-based graphics server

A very lightweight display server for Torch. Best used as a remote desktop paired with a terminal of your choice.

Use a Torch REPL (e.g., trepl) via SSH to control Torch and tell it to display stuff (images, plots, audio) to the server. The server then forwards the display data to (one or more) web clients.

Installation

Install for Torch via:

luarocks install https://raw.githubusercontent.com/szym/display/master/display-scm-0.rockspec

Install for Python (numpy required) via:

python setup.py install [--user]

NOTE: The Python client is not yet fully developed.

Quick Start

Launch the server:

th -ldisplay.start [port [hostname]]

Note, there is no authentication so don't use "as is" for sensitive data. By default, the server listens on localhost. Pass 0.0.0.0 to allow external connections on any interface:

th -ldisplay.start 8000 0.0.0.0

Then open http://(hostname):(port)/ in your browser to load the remote desktop.

To actually display stuff on the server, use the display package in a Torch script or REPL:

-- Generic stuff you'll need to make images anyway.
torch = require 'torch'
image = require 'image'

-- Load the display package
display = require 'display'

-- Tell the library, if you used a custom port or a remote server (default is 127.0.0.1).
display.configure({hostname='myremoteserver.com', port=1234})

-- Display a torch tensor as an image. The image is automatically normalized to be renderable.
local lena = image.lena()
display.image(lena)

-- Plot some random data.
display.plot(torch.cat(torch.linspace(0, 10, 10), torch.randn(10), 2))

See example.lua or example.py for a bigger example.

Usage

Each command creates a new window (pane) on the desktop that can be independently positioned, resized, maximized. It also returns the id of the window which can be passed as the win option to reuse the window for a subsequent command. This can be used to show current progress of your script:

for i = 1, num_iterations do
   -- do some work
   ...
   -- update results
   local state_win = display.image(current_state, {win=state_win, title='state at iteration ' .. i})
end

Another common option is title. The title bar can be double-clicked to maximize the window. The x button closes the window. The o button "disconnects" the window so that it will not be overwritten when the script reuses the window id. This is useful if you want to make a quick "copy" of the window to compare progress between iterations.

Images

display.image(tensor, options)

Displays the tensor as an image. The tensor is normalized (by a scalar offset and scaling factor) to be displayable. The image can be panned around and zoomed (with the scroll wheel or equivalent). Double-click the image to restore original size or fill the window.

If the tensor has 4 dimensions, it is considered to be a list of images -- sliced by first dimension. Same thing if it has 3 dimensions but the first dimension has size more than 3 (so they cannot be considered the RGB channels). This is equivalent to passing a list (Lua table) of tensors or the explicit images command. This is convenient when visualizing the trained filters of convolutional layer. Each image is normalized independently. When displaying a list of images, the option labels can be used to put a small label on each sub-image:

display.images({a, b, c, d}, {labels={'a', 'b', 'c', 'd'}})

Finally, the option width can be used to specify the initial size of the window in pixels.

Plotting

display.plot(data, options)

Creates a Dygraph plot which is most useful for visualizing time series. The graph can be zoomed in by selecting a range of X values or zoomed-out by double-clicking it.

The data should either be a 2-dimensional tensor where the each row is a data point and each column is a series, or a Lua table of tables. The first column is always taken as the X dimension. The command supports all the Dygraph options. Most importantly labels is taken as a list (Lua table) of series labels. Again the first label is for the X axis. You can name the Y axis with ylabel.

local config = {
  title = "Global accuracy/recall over time",
  labels = {"epoch", "accuracy", "recall"},
  ylabel = "ratio",
}

for t = 1, noEpoch do
  -- update model, compute data
  local accuracy, recall = train()
  -- update plot data
  table.insert(data, {t, accuracy, recall})
  -- display
  config.win = display.plot(data, config)
end

Other

display.audio(tensor_with_audio, options)

Development

Supported commands

  • pane: creates a new Pane of specified type; arguments are:
    • type: the registered type, e.g., image for ImagePane
    • win: identifier of the window to be reused (pick a random one if you want a new window)
    • title: title for the window title bar
    • content: passed to the Pane.setContent method

Built-in Pane types

image creates a zoomable element

  • src: URL for the element
  • width: initial width in pixels
  • labels: array of 3-element arrays [ x, y, text ], where x, y are the coordinates (0, 0) is top-left, (1, 1) is bottom-right; text is the label content

plot creates a Dygraph, all Dygraph options are supported

  • file: see Dygraph data formats for supported formats
  • labels: list of strings, first element is the X label

text places raw text in

element

audio places raw audio content in an element

Technical overview

The server is a trivial message forwarder:

POST /events -> EventSource('/events')

The Lua client sends JSON commands directly to the server. The browser script interprets these commands, e.g.

{ command: 'pane', type: 'image', content: { src: 'data:image/png;base64,....' }, title: 'lena' }

History

Originally forked from gfx.js.

The initial goal was to remain compatible with the torch/python API of gfx.js, but remove the term.js/tty.js/pty.js stuff which is served just fine by ssh.

Compared to gfx.js:

  • no terminal windows (no term.js)
  • dygraphs instead of nvd3 (have built in zoom and are perfect for time-series plots)
  • plots resize when windows are resized
  • images support zoom and pan
  • image lists are rendered as one image to speed up loading
  • windows remember their positions
  • implementation not relying on the filesystem, supports remote clients (sources)
Owner
Szymon Jakubczak
Szymon Jakubczak
Computational Xmas Tree lights!

Computational Xmas Tree This repo contains the code for the computational illumination of a Christmas Tree! It is based on the work by Matt Parker fro

GSD6338 146 Dec 23, 2022
Python Digital Art Generator

Python Digital Art Generator The main goal of this repository is to generate all possible layers permutations given by the user in order to get unique

David Cuentas Mar 3 Mar 12, 2022
Tool to create a Phunk image with a custom background

Create Phunk image Tool to create a Phunk image with a custom background Installation Clone the repo git clone https://github.com/albanow/etherscan_sa

Albano Pena Torres 6 Mar 31, 2022
Convert HDR photos taken by iPhone 12 (or later) to regular HDR images

heif-hdrgainmap-decode Convert HDR photos taken by iPhone 12 (or later) to regular HDR images. Installation First, make sure you have the following pa

Star Brilliant 5 Nov 13, 2022
Make GIFs from time-stacked xarray.DataArrays (time, [optional band], y, x), dead-simple.

GeoGIF Make GIFs from time-stacked xarray.DataArrays (time, [optional band], y, x), dead-simple. from geogif import gif, dgif gif(data_array) dgif(das

Gabe Joseph 47 Dec 22, 2022
This projects aim is to simulate flowers(Gerbera Daisy) phyllotaxis.

phyllotaxis This projects aim is to simulate flowers(Gerbera Daisy) phyllotaxis. Take a look at the arrangement of this flower's seeds, this project's

amirsalar 3 Dec 10, 2021
vsketch is a Python generative art toolkit for plotters

Generative plotter art environment for Python

Antoine Beyeler 380 Dec 29, 2022
QR Generator using GUI with Tinker

BinCat Token System Very simple python script with GUI that generates QR codes. It don't include a QR "decription" tool. It only generate-it and thats

Hipotesi 1 Nov 06, 2021
Conversion of Image, video, text into ASCII format

asciju Python package that converts image to ascii Free software: MIT license

Aju Tamang 11 Aug 22, 2022
QR-code Generator with a basic GUI.

Qr_generator_python Qr code generator with a basic GUI. ❔ About the QR-Code-Generator This project Generates QR codes to sites, e-mails and plain text

Tecixck 2 Oct 11, 2021
Xmas-Tree-GIF-Tool - Convert any given animated gif file into an animation in GIFT CSV format

This repo is made to participate in Matt Parker's XmasTree 2021 event. Convert a

Aven Zitzelberger 2 Dec 30, 2021
Convert photos to paintings with python

Convert-photos-to-paintings Before the changes After the changes Before the changes After the changes This code is written in the Python programming l

Amir Hussein Sharifnezhad 3 May 31, 2022
Change the image one color channel at a time.

Building-a-Contact-Sheet This hands-on Project is in Python 3 Programming Specialization offered by University of Michigan via Coursera. change the im

Eszter Pai 1 Jan 03, 2022
PyGtk Color - A couple of python scripts to select a color (for scripting usage)

Selection Scripts This repository contains two scripts to be used within a scripting project, to aquire a color value. Both scripts requir

Spiros Georgaras 1 Oct 31, 2021
Simple to use image handler for python sqlite3.

SQLite Image Handler Simple to use image handler for python sqlite3. Functions Function Name Parameters Returns init databasePath : str tableName : st

Mustafa Ozan Çetin 7 Sep 16, 2022
Multi-view 3D reconstruction using neural rendering. Unofficial implementation of UNISURF, VolSDF, NeuS and more.

Multi-view 3D reconstruction using neural rendering. Unofficial implementation of UNISURF, VolSDF, NeuS and more.

Jianfei Guo 683 Jan 04, 2023
Docbarcodes extracts 1D and 2D barcodes from scanned PDF documents or images. It can be used to automate extraction and processing of all kind of documents.

Intro Barcodes are being used in many documents or forms to enable machine reading capabilities and reduce manual processing effort. Simple 1D barcode

Arlind Nocaj 3 Jun 18, 2022
Create a random fluent image based on multiple colors.

FluentGenerator Create a random fluent image based on multiple colors. Navigation Example Install Update Usage In Python console FluentGenerator Fluen

1 Feb 02, 2022
Program to export all new icons from the latest Fortnite patch

Assets Exporter This program allows you to generate all new icons of a patch in png! Requierements Python =3.8 (installed on your computer) If you wa

ᴅᴊʟᴏʀ3xᴢᴏ 6 Jun 24, 2022
Dynamic image server for web and print

Quru Image Server - dynamic imaging for web and print QIS is a high performance web server for creating and delivering dynamic images. It is ideal for

Quru 84 Jan 03, 2023