Macro recording and metaprogramming in Python

Overview

macro-kit

macro-kit is a package for efficient macro recording and metaprogramming in Python using abstract syntax tree (AST).

The design of AST in this package is strongly inspired by Julia metaprogramming. Similar methods are also implemented in builtin ast module but macro-kit is more focused on the macro generation and customization.

Installation

  • use pip
pip install macro-kit
  • from source
pip install git+https://github.com/hanjinliu/macro-kit

Examples

  1. Define a macro-recordable function
from macrokit import Macro, Expr, Symbol
macro = Macro()

@macro.record
def str_add(a, b):
    return str(a) + str(b)

val0 = str_add(1, 2)
val1 = str_add(val0, "xyz")
macro
[Out]
var0x24fdc2d1530 = str_add(1, 2)
var0x24fdc211df0 = str_add(var0x24fdc2d1530, 'xyz')
# substitute identifiers of variables
# var0x24fdc2d1530 -> x
macro.format([(val0, "x")]) 
[Out]
x = str_add(1, 2)
var0x24fdc211df0 = str_add(x, 'xyz')
# substitute to _dict["key"], or _dict.__getitem__("key")
expr = Expr(head="getitem", args=[Symbol("_dict"), "key"])
macro.format([(val0, expr)])
[Out]
_dict['key'] = str_add(1, 2)
var0x24fdc211df0 = str_add(_dict['key'], 'xyz')
  1. Record class
macro = Macro()

@macro.record
class C:
    def __init__(self, val: int):
        self.value = val
    
    @property
    def value(self):
        return self._value
    
    @value.setter
    def value(self, new_value: int):
        if not isinstance(new_value, int):
            raise TypeError("new_value must be an integer.")
        self._value = new_value
    
    def show(self):
        print(self._value)

c = C(1)
c.value = 5
c.value = -10
c.show()
[Out]
-10
macro.format([(c, "ins")])
[Out]
ins = C(1)
ins.value = -10     # setattr (and setitem) will not be recorded in duplicate
var0x7ffed09d2cd8 = ins.show()
macro.eval({"C": C})
[Out]
-10
  1. Record module
import numpy as np
macro = Macro()
np = macro.record(np) # macro-recordable numpy

arr = np.random.random(30)
mean = np.mean(arr)

macro
[Out]
var0x2a0a2864090 = numpy.random.random(30)
var0x2a0a40daef0 = numpy.mean(var0x2a0a2864090)
from dask import array as da
dask_macro = macro.format([(np, "da")])
dask_macro
[Out]
var0x2a0a2864090 = da.random.random(30)
var0x2a0a40daef0 = da.mean(var0x2a0a2864090)
output = {}
dask_macro.eval({"da": da}, output)
output
[Out]
{:da: 
   
    ,
 :var0x2a0a2864090: dask.array
    
     ,
 :var0x2a0a40daef0: dask.array
     
      }

     
    
   
You might also like...
Find dependent python scripts of a python script in a project directory.

Find dependent python scripts of a python script in a project directory.

SysInfo is an app developed in python which gives Basic System Info , and some detailed graphs of system performance .
SysInfo is an app developed in python which gives Basic System Info , and some detailed graphs of system performance .

SysInfo SysInfo is an app developed in python which gives Basic System Info , and some detailed graphs of system performance . Installation Download t

An OData v4 query parser and transpiler for Python

odata-query is a library that parses OData v4 filter strings, and can convert them to other forms such as Django Queries, SQLAlchemy Queries, or just plain SQL.

Python program to do with percentages and chances, random generation.

Chances and Percentages Python program to do with percentages and chances, random generation. What is this? This small program will generate a list wi

A Python library for reading, writing and visualizing the OMEGA Format
A Python library for reading, writing and visualizing the OMEGA Format

A Python library for reading, writing and visualizing the OMEGA Format, targeted towards storing reference and perception data in the automotive context on an object list basis with a focus on an urban use case.

A simple and easy to use Spam Bot made in Python!

This is a simple spam bot made in python. You can use to to spam anyone with anything on any platform.

RapidFuzz is a fast string matching library for Python and C++

RapidFuzz is a fast string matching library for Python and C++, which is using the string similarity calculations from FuzzyWuzzy

pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

A Python package implementing various colour checker detection algorithms and related utilities.
A Python package implementing various colour checker detection algorithms and related utilities.

A Python package implementing various colour checker detection algorithms and related utilities.

Releases(v0.3.8)
  • v0.3.8(Mar 16, 2022)

    Bug fixes

    • Length 1 tuple (1,) and length 0 set set() was not recorded correctly.
    • Import and __all__ did not match.
    • The __getitem__ was not correctly defined when a slice is used as indices.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.7(Jan 22, 2022)

  • v0.3.6(Jan 13, 2022)

    Changes

    • Mock object for easier Expr construction. Instead of writing like Expr("getattr", [Expr("getattr", ...), b]), you can use m = Mock("m") and create Expr object by m.a(0, "s").
    • Full support of Python 3.7, and add tox test.

    Bug Fixes

    • The __eq__ method will return False instead of raise Exception to make macro recording safer.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.4(Dec 12, 2021)

  • v0.3.3(Dec 10, 2021)

    New Features

    • New ast type supports: break, continue and raise.
    • Modules will be registered to Symbol whenever it is converted into a Symbol so that you'll not have to pass module object when you call eval. For instance, instead of macro.eval({"np", np}) you only have to call macro.eval().

    Changes

    • The "type" argument in Symbol is removed because it is never used.

    Bug Fixes

    • Hash of Symbol was not completely safe.
    • mModule did not return constant values correctly (such as np.pi).
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Dec 8, 2021)

    New Features

    • register_type and Symbol.register_type can be used as a decorator.
    • Symbol.var("x") can create unique variable x.
    • Many missing AST types are supported, such as % (mod operation), += (in-place operations) and unary operations.
    • Validators are implemented in Expr to assert correct grammar for certain header (although they are not perfect). For instance, Expr("getattr", [Symbol("x"), "name"]) used to return :(x.'name') but now it returns :(x.name).
    • Many special methods, such as __int__ and __hash__, are correctly converted to int(X) in macro.
    • Macro can used as a field of a class. Unique Macro object will be created for different instances.

    Changes

    • Attribute _last_setval is moved to Macro.
    • Method type definition is changed to delayed definition.
    • Type mapping of symbol now uses dictionary as much as possible.
    • The type keyword argument in Symbol constructor is deleted because it was not used.

    Bug Fixes

    • Boolean operations (and and or) could not be parsed.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Dec 5, 2021)

    New Features

    • Support function definition, if, for, and many other expressions.
    • Macro.callbacks attribute for expression-added callbacks.
    • flags keyword argument to specify what kind of expressions will be recorded.

    Changes

    • Macro inherits Expr but head is fixed to Head.block to simplify module.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Oct 24, 2021)

    New Features

    • Support string parsing using ast.
    • optimize method of Macro can remove unused returned variables. It makes macro looks better.

    Changes

    • Head is different from that in v0.1.0, thus may be inconsistent.
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Oct 23, 2021)

Owner
Department of Biological Sciences, School of Science, The University of Tokyo. My major is biophysics, especially microtubule and motor proteins.
DUQ is a python package for working with physical Dimensions, Units, and Quantities.

DUQ is a python package for working with physical Dimensions, Units, and Quantities.

2 Nov 02, 2022
A Tool that provides automatic kerning for ligature based OpenType fonts in Microsoft Volt

Kerning A Tool that provides automatic kerning for ligature based OpenType fonts in Microsoft Volt There are three stages of the algorithm. The first

Sayed Zeeshan Asghar 6 Aug 01, 2022
Simplex using Jordan exchanges taught in 236A

Simplex for 236A Python script to solve LP using simplex by Jordan exchanges taught in 236A. You will need python installed along with the 'numpy' and

Kunal Kishore 1 Nov 30, 2021
Grank is a feature-rich script that automatically grinds Dank Memer for you

Grank Inspired by this repository. This is a WIP and there will be more functions added in the future. What is Grank? Grank is a feature-rich script t

42 Jul 20, 2022
This is a python table of data implementation with styles, colors

Table This is a python table of data implementation with styles, colors Example Table adapts to the lack of data Lambda color features Full power of l

Урядов Алексей 5 Nov 09, 2021
🍰 ConnectMP - An easy and efficient way to share data between Processes in Python.

ConnectMP - Taking Multi-Process Data Sharing to the moon 🚀 Contribute · Community · Documentation 🎫 Introduction : 🍤 ConnectMP is the easiest and

Aiden Ellis 1 Dec 24, 2021
A toolkit for writing and executing automation scripts for Final Fantasy XIV

XIV Scripter This is a tool for scripting out series of actions in FFXIV. It allows for custom actions to be defined in config.yaml as well as custom

Jacob Beel 1 Dec 09, 2021
A python tool give n number of inputs and parallelly you will get a output by separetely

http-status-finder Hello Everyone!! This is kavisurya, In this tool you can give n number of inputs and parallelly you will get a output by separetely

KAVISURYA V 3 Dec 05, 2021
✨ Un pierre feuille ciseaux totalement fait en Python par moi, et en français.

Pierre Feuille Ciseaux ❗ Un pierre feuille ciseaux totalement fait en Python par moi. 🔮 Avec l'utilisation du module "random", j'ai pu faire un choix

MrGabin 3 Jun 06, 2021
Helpful functions for use alongside the rich Python library.

🔧 Rich Tools A python package with helpful functions for use alongside with the rich python library. 󠀠󠀠 The current features are: Convert a Pandas

Avi Perl 14 Oct 14, 2022
Python code to remove empty folders from Windows/Android.

Empty Folder Cleaner is a program that deletes empty folders from your computer or device and removes clutter to improve performance. It supports only windows and android for now.

Dark Coder Cat | Vansh 4 Sep 27, 2022
extract gene TSS/TES site form gencode/ensembl/gencode database GTF file and export bed format file.

GetTsite python Package extract gene TSS/TES site form gencode/ensembl/gencode database GTF file and export bed format file. Install $ pip install Get

laojunjun 7 Nov 21, 2022
Check the basic quality of any dataset

Data Quality Checker in Python Check the basic quality of any dataset. Sneak Peek Read full tutorial at Medium. Explore the app Requirements python 3.

MalaDeep 8 Feb 23, 2022
osqueryIR is an artifact collection tool for Linux systems.

osqueryIR osqueryIR is an artifact collection tool for Linux systems. It provides the following capabilities: Execute osquery SQL queries Collect file

AbdulRhman Alfaifi 7 Nov 02, 2022
A simple example for calling C++ functions in Python by `ctypes`.

ctypes-example A simple example for calling C++ functions in Python by ctypes. Features call C++ function int bar(int* value, char* msg) with argumene

Yusu Pan 3 Nov 23, 2022
💉 코로나 잔여백신 예약 매크로 커스텀 빌드 (속도 향상 버전)

Korea-Covid-19-Vaccine-Reservation 코로나 잔여 백신 예약 매크로를 기반으로 한 커스텀 빌드입니다. 더 빠른 백신 예약을 목표로 하며, 속도를 우선하기 때문에 사용자는 이에 대처가 가능해야 합니다. 지정한 좌표 내 대기중인 병원에서 잔여 백신

Queue.ri 21 Aug 15, 2022
JavaScript to Python Translator & JavaScript interpreter written in 100% pure Python🚀

Pure Python JavaScript Translator/Interpreter Everything is done in 100% pure Python so it's extremely easy to install and use. Supports Python 2 & 3.

Piotr Dabkowski 2.1k Dec 30, 2022
A fast Python implementation of Ac Auto Mechine

A fast Python implementation of Ac Auto Mechine

Jin Zitian 1 Dec 07, 2021
Python based utilities for interacting with digital multimeters that are built on the FS9721-LP3 chipset.

Python based utilities for interacting with digital multimeters that are built on the FS9721-LP3 chipset.

Fergus 1 Feb 02, 2022
Application for easy configuration of swap file and swappiness priority in slackware and others linux distributions.

Swap File Program created with the objective of assisting in the configuration of swap file in Distributions such as Slackware. Required packages: pyt

Mauricio Ferrari 3 Aug 06, 2022