Oil is a new Unix shell. It's our upgrade path from bash to a better language and runtime

Related tags

Command-line Toolsoil
Overview

Oil Source Code

Build Status

Oil is a new Unix shell. It's our upgrade path from bash to a better language and runtime! (Why Create a New Unix Shell? / 2019 FAQ)

It's written in Python, so the code is short and easy to change. But we automatically translate it to C++ with custom tools, to make it fast and small. The deployed executable doesn't depend on Python.

This README is at the root of the git repo.

Contributing

  • Try making the dev build of Oil with the instructions on the Contributing page. This should take 1 to 5 minutes if you have a Linux machine.
  • If it doesn't, let us know. You can post on the #oil-dev channel of oilshell.zulipchat.com, or file an issue on Github.
  • Feel free to grab an issue from Github. Let us know what you're thinking before you get too far.

Quick Start on Linux

After following the instructions on the Contributing page, you'll have a Python program that you can quickly run and change! Try it interactively:

bash$ bin/osh

osh$ name=world
osh$ echo "hello $name"
hello world
  • Try running a shell script you wrote with bin/osh myscript.sh.
  • Try the Oil language with bin/oil.

Let us know if any of these things don't work! The continuous build tests them at every commit.

Dev Build vs. Release Build

Again, note that the developer build is very different from the release tarball. The Contributing page describes this difference in detail.

The release tarballs are linked from the home page. (Developer builds don't work on OS X, so use the release tarballs on OS X.)

Important: We Accept Small Contributions!

Oil is full of many ideas, which may be intimidating at first.

But the bar to contribution is very low. It's basically a medium size Python program with many tests, and many programmers know how to change such programs. It's great for prototyping.

  • For OSH compatibility, I often merge failing spec tests. You don't even have to write code! The tests alone help. I search for related tests with grep xtrace spec/*.test.sh, where xtrace is a shell feature.
  • You only have to make your code work in Python. Plain Python programs are easy to modify. The semi-automated translation to C++ is a separate step, although it often just works.
  • You can influence the design of the Oil language. If you have an itch to scratch, be ambitious. For example, you might want to show us how to implement nonlinear pipelines.

Docs

The Wiki has many developer docs. Feel free to edit them. If you make a major change, let us know on Zulip!

There are also READMEs in some subdirectories, like opy/ and mycpp/.

If you're confused, the best thing to do is to ask on Zulip and someone should produce a pointer and/or improve the docs.

Docs for end users are linked from each release page.

Repository Structure

Try this to show a summary of what's in the repo and their line counts:

$ metrics/source-code.sh overview

(Other functions in this file may be useful as well.)

A Collection of Interpreters

Oil is naturally structured as a set of mutually recursive parsers and evaluators. These interpreters are specified at a high-level: with regular languages, Zephyr ASDL, and a statically-typed subset of Python.

bin/              # Main entry points like bin/osh (source in bin/oil.py)
frontend/         # Lexing/Parsing code common to Oil and OSH
osh/              # OSH parsers and evaluators (cmd, word, sh_expr)
oil_lang/         # Oil parser and evaluator
core/             # Other code shared between Oil and OSH
pylib/            # Borrowed from the Python standard library.
tools/            # User-facing tools, e.g. the osh2oil translator

DSLs / Code Generators

Here are the tools that transform that high-level code to efficient code:

asdl/             # ASDL implementation, derived from CPython
pgen2/            # Parser Generator, borrowed from CPython
mycpp/            # Experimental translator from typed Python to C++.
                  # Depends on MyPy.
opy/              # Python compiler in Python (mycpp/ will replace it)
  lib/            # Common code
  compiler2/      # Bytecode compiler
  byterun/        # Metacircular bytecode VM in Python
  gold/           # tests
  byterun/        # Unused bytecode interpreter

Native Code

We have native code to support both the dev build (running under CPython) and the oil-native build (pure C++):

Python-2.7.13/    # CPython is the initial basis for the Oil VM
native/           # Python extension modules, e.g. libc.c
cpp/              # C++ code which complements the mycpp translation

Several Kinds of Tests

Unit tests are named foo_test.py and live next to foo.py.

test/             # Test automation
  gold/           # Gold Test cases
  gold.sh         
  sh_spec.py      # shell spec test framework
  spec.sh         # Types of test runner: spec, unit, gold, wild
  unit.sh         
  wild.sh
testdata/
spec/             # Spec test cases
  bin/            # tools used in many spec tests
  testdata/       # scripts for specific test cases
  errors/         # TODO: migrate these bad shell scripts
types/            # Scripts for running MyPy and PyAnnotate, etc.

Dev Tools and Scripts

We use a lot of automation to improve the dev process. It's largely written in shell, of course!

benchmarks/       # Benchmarks should be run on multiple machines.
metrics/          # Metrics don't change between machines (e.g. code size)
client/           # Demonstration of OSH as a headless server.
build/            # Build automation
  oil-defs/       # Files that define our slice of CPython.
  dev.sh          # For development builds, running CPython
devtools/         # For Oil developers (not end users)
  release.sh      # The (large) release process.
demo/             # Demonstrations of bash/shell features.  Could be
                  # moved to tests/ if automated.
  old/            # A junk drawer.
web/              # HTML/JS/CSS for tests and tools
soil/             # Multi-cloud continuous build (e.g. sourcehut, Github)
services/         # Other cloud services

Temp Dirs

Directories that begin with _ are not stored in git. The dev tools above create and use these dirs.

_bin/             # Native executables are put here
_build/           # Temporary build files
_devbuild/        # Developer build files not deleted upon 'make clean'
  gen/            # Generated Python and C code
_deps/            # build dependencies like re2c
_tmp/             # Test suites and other temp files
  spec/
  wild/
    raw/
    www/
  osh-parser/
  osh-runtime/
  vm-baseline/
  oheap/
  startup/
  ...
_release/         # Source release tarballs are put here
  VERSION/        # Published at oilshell.org/release/$VERSION/
    benchmarks/
    doc/
    metrics/
    test/
      spec.wwz
      wild.wwz
      ...
    web/          # Static files, copy of $REPO_ROOT/web
      table/

Build System for End Users

This is very different than the developer build of Oil.

Makefile
configure
install

Doc Sources

doc/              # A mix of docs
doctools/         # Tools that use lazylex/ to transform Markdown/HTML
lazylex/          # An HTML lexer which doctools/ builds upon.
README.md         # This page, which is For Oil developers

LICENSE.txt       # For end users
INSTALL.txt

More info

Python Files Not Translated to C++

mycpp/
  mylib.py  # statically typed equivalents of Python's data structures
pylib/      # copied from Python stdlib
core/
  py{error,os,util}.py  # too complicated to translate
*/*_def.py  # abstract definitions
*/*_gen.py  # code generators
Redial is a simple shell application that manages your SSH sessions on Unix terminal.

redial redial is a simple shell application that manages your SSH sessions on Unix terminal. What's New 0.7 (19.12.2019) Basic support for adding ssh

Bahadır Yağan 186 Oct 28, 2022
Unofficial Open Corporates CLI: OpenCorporates is a website that shares data on corporations under the copyleft Open Database License. This is an unofficial open corporates python command line tool.

Unofficial Open Corporates CLI OpenCorporates is a website that shares data on corporations under the copyleft Open Database License. This is an unoff

Richard Mwewa 30 Sep 08, 2022
jenkins-tui is a terminal based user interface for Jenkins.

jenkins-tui 📦 jenkins-tui is a terminal based user interface for Jenkins. 🚧 ⚠️ This app is a prototype and in very early stages of development. Ther

Craig Gumbley 22 Oct 24, 2022
Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages.

OpenCV on Wheels Pre-built CPU-only OpenCV packages for Python. Check the manual build section if you wish to compile the bindings from source to enab

OpenCV 3.2k Jan 04, 2023
Use case: quick JSON processing/restructuring with jq without terminal

alfred-jq Alfred workflow to conveniently process JQ on clipboard based on a jq query Also available at: packal/jq Use case: quick JSON processing/res

T on Meta Mode 5 Sep 30, 2022
Get Air Quality Index for your city/country 😷

Air Quality Index CLI Get Air Quality index for your City. Installation $ pip install air-quality-cli Contents Air Quality Index CLI Installation Cont

Yankee 40 Oct 21, 2022
A Python module and command-line utility for converting .ANS format ANSI art to HTML

ansipants A Python module and command-line utility for converting .ANS format ANSI art to HTML. Installation pip install ansipants Command-line usage

4 Oct 16, 2022
CLI client for FerrisChat

A CLI Client for @FerrisChat using FerrisWheel

FerrisChat 2 Apr 01, 2022
A super simple terminal command shortener 🐟

pcmd A super simple terminal command shortener 🐟 Source code : https://github.com/j0fiN/pcmd Documentation : https://j0fin.github.io/pcmd About Durin

9 Mar 02, 2022
A python-based terminal application that displays current cryptocurrency prices

CryptoAssetPrices A python-based terminal application that displays current cryptocurrency prices. Covered Cryptocurrencies Bitcoin (BTC) Ethereum (ET

3 Apr 21, 2022
A startpage configured aesthetically with terminal-esque link formatting

Terminal-y Startpage Setup Clone the repository, then make an unformatted.txt file following the specifications in example.txt. Run format.py Open ind

belkarx 13 May 01, 2022
A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli. Stream your favorite shows straight from the command line.

A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli. Installation pip install -r requirements.txt It use

Jonardon Hazarika 17 Dec 11, 2022
A curated list of awesome things related to Textual

Awesome Textual | A curated list of awesome things related to Textual. Textual is a TUI (Text User Interface) framework for Python inspired by modern

Marcelo Trylesinski 5 May 08, 2022
Random scripts and other bits for interacting with the SpaceX Starlink user terminal hardware

starlink-grpc-tools This repository has a handful of tools for interacting with the gRPC service implemented on the Starlink user terminal (AKA "the d

270 Dec 29, 2022
Zecwallet-Python is a simple wrapper around the Zecwallet Command Line LightClient written in Python

A wrapper around Zecwallet Command Line LightClient, written in Python Table of Contents About Installation Usage Examples About Zecw

Priveasy 2 Sep 06, 2022
texel - Command line interface for reading spreadsheets inside terminal

texel - Command line interface for reading spreadsheets inside terminal. Sometimes, you have to deal with spreadsheets. Those are sad times. Fortunate

128 Dec 19, 2022
Colab-xterm allows you to open a terminal in a cell

colab-xterm Colab-xterm allows you to open a terminal in a cell. Usage Install package and load the extension !pip install git+https://github.com/popc

InfuseAI 194 Dec 29, 2022
Doro is a CLI based pomodoro app and countdown timer application built using python.

Doro - CLI based pomodoro app Doro is a CLI based pomodoro app and countdown timer application built using python. Install $ pip install doro Usage Po

Suresh Kumar 14 May 23, 2022
CPOST is a CLI tool to assist with the proper sizing of Clara Deploy pipelines

CPOST (Clara Pipeline Operator Sizing Tool) Tool to measure resource usage of Clara Platform pipeline operators Cpost is a tool that will help you run

NVIDIA Corporation 5 Sep 27, 2021
A very simple python script to encode and decode PowerShell one-liners.

PowerShell Encoder A very simple python script to encode and decode PowerShell one-liners. I used Raikia's PowerShell encoder ALOT, but one day it wen

John Tear 5 Jul 29, 2022