Python ASN.1 library with a focus on performance and a pythonic API

Overview

asn1crypto

A fast, pure Python library for parsing and serializing ASN.1 structures.

GitHub Actions CI Travis CI AppVeyor CircleCI PyPI

Features

In addition to an ASN.1 BER/DER decoder and DER serializer, the project includes a bunch of ASN.1 structures for use with various common cryptography standards:

Standard Module Source
X.509 asn1crypto.x509 RFC 5280
CRL asn1crypto.crl RFC 5280
CSR asn1crypto.csr RFC 2986, RFC 2985
OCSP asn1crypto.ocsp RFC 6960
PKCS#12 asn1crypto.pkcs12 RFC 7292
PKCS#8 asn1crypto.keys RFC 5208
PKCS#1 v2.1 (RSA keys) asn1crypto.keys RFC 3447
DSA keys asn1crypto.keys RFC 3279
Elliptic curve keys asn1crypto.keys SECG SEC1 V2
PKCS#3 v1.4 asn1crypto.algos PKCS#3 v1.4
PKCS#5 v2.1 asn1crypto.algos PKCS#5 v2.1
CMS (and PKCS#7) asn1crypto.cms RFC 5652, RFC 2315
TSP asn1crypto.tsp RFC 3161
PDF signatures asn1crypto.pdf PDF 1.7

Why Another Python ASN.1 Library?

Python has long had the pyasn1 and pyasn1_modules available for parsing and serializing ASN.1 structures. While the project does include a comprehensive set of tools for parsing and serializing, the performance of the library can be very poor, especially when dealing with bit fields and parsing large structures such as CRLs.

After spending extensive time using pyasn1, the following issues were identified:

  1. Poor performance
  2. Verbose, non-pythonic API
  3. Out-dated and incomplete definitions in pyasn1-modules
  4. No simple way to map data to native Python data structures
  5. No mechanism for overridden universal ASN.1 types

The pyasn1 API is largely method driven, and uses extensive configuration objects and lowerCamelCase names. There were no consistent options for converting types of native Python data structures. Since the project supports out-dated versions of Python, many newer language features are unavailable for use.

Time was spent trying to profile issues with the performance, however the architecture made it hard to pin down the primary source of the poor performance. Attempts were made to improve performance by utilizing unreleased patches and delaying parsing using the Any type. Even with such changes, the performance was still unacceptably slow.

Finally, a number of structures in the cryptographic space use universal data types such as BitString and OctetString, but interpret the data as other types. For instance, signatures are really byte strings, but are encoded as BitString. Elliptic curve keys use both BitString and OctetString to represent integers. Parsing these structures as the base universal types and then re-interpreting them wastes computation.

asn1crypto uses the following techniques to improve performance, especially when extracting one or two fields from large, complex structures:

  • Delayed parsing of byte string values
  • Persistence of original ASN.1 encoded data until a value is changed
  • Lazy loading of child fields
  • Utilization of high-level Python stdlib modules

While there is no extensive performance test suite, the CRLTests.test_parse_crl test case was used to parse a 21MB CRL file on a late 2013 rMBP. asn1crypto parsed the certificate serial numbers in just under 8 seconds. With pyasn1, using definitions from pyasn1-modules, the same parsing took over 4,100 seconds.

For smaller structures the performance difference can range from a few times faster to an order of magnitude or more.

Related Crypto Libraries

asn1crypto is part of the modularcrypto family of Python packages:

Current Release

1.4.0 - changelog

Dependencies

Python 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8 or pypy. No third-party packages required.

Installation

pip install asn1crypto

License

asn1crypto is licensed under the terms of the MIT license. See the LICENSE file for the exact license text.

Documentation

The documentation for asn1crypto is composed of tutorials on basic usage and links to the source for the various pre-defined type classes.

Tutorials

Reference

Continuous Integration

Various combinations of platforms and versions of Python are tested via:

Testing

Tests are written using unittest and require no third-party packages.

Depending on what type of source is available for the package, the following commands can be used to run the test suite.

Git Repository

When working within a Git working copy, or an archive of the Git repository, the full test suite is run via:

python run.py tests

To run only some tests, pass a regular expression as a parameter to tests.

python run.py tests ocsp

PyPi Source Distribution

When working within an extracted source distribution (aka .tar.gz) from PyPi, the full test suite is run via:

python setup.py test

Package

When the package has been installed via pip (or another method), the package asn1crypto_tests may be installed and invoked to run the full test suite:

pip install asn1crypto_tests
python -m asn1crypto_tests

Development

To install the package used for linting, execute:

pip install --user -r requires/lint

The following command will run the linter:

python run.py lint

Support for code coverage can be installed via:

pip install --user -r requires/coverage

Coverage is measured by running:

python run.py coverage

To change the version number of the package, run:

python run.py version {pep440_version}

To install the necessary packages for releasing a new version on PyPI, run:

pip install --user -r requires/release

Releases are created by:

  • Making a git tag in PEP 440 format

  • Running the command:

    python run.py release

Existing releases can be found at https://pypi.org/project/asn1crypto/.

CI Tasks

A task named deps exists to download and stage all necessary testing dependencies. On posix platforms, curl is used for downloads and on Windows PowerShell with Net.WebClient is used. This configuration sidesteps issues related to getting pip to work properly and messing with site-packages for the version of Python being used.

The ci task runs lint (if flake8 is available for the version of Python) and coverage (or tests if coverage is not available for the version of Python). If the current directory is a clean git working copy, the coverage data is submitted to codecov.io.

python run.py deps
python run.py ci
Owner
Will Bond
Software engineer, occasional designer. Working for @sublimehq. Proud husband and father.
Will Bond
Privfiles - Encrypted file storage using Fernet with zero Javascript

Privfiles - Encrypted file storage using Fernet with zero Javascript Source code for the onion service: l3n6v6dm63frml22tlmzacnasvp7co7wylu4hhcs34ukxe

Ward 5 Jul 30, 2022
bitcoin-ticker is a E-ink ticker that shows usefull information about bitcoin

bitcoin-ticker bitcoin-ticker is a E-ink ticker that shows usefull information about bitcoin. Due to the limited refresh lifetime, new information is

32 Nov 09, 2022
The (Python-based) mining software required for the Nintendo Switch mining project.

ntgbtminer - Nintendo Switch edition This is a version of ntgbtminer that works with the Nintendo Switch bitcoin miner. ntgbtminer ntgbtminer is a no

4 Jun 03, 2021
PeGuard - Windows PE crypter and packing utility

PEGUARD PEGUARD is a file crypter and packing utility. This project was original

11 Nov 28, 2022
Simple encryption-at-rest with key rotation support for Python.

keyring Simple encryption-at-rest with key rotation support for Python. N.B.: keyring is not for encrypting passwords--for that, you should use someth

Dann Luciano 1 Dec 23, 2021
📊Python implementation of the Colin Talks Crypto Bitcoin Bull Run Index (CBBI).

Colin Talks Crypto Bitcoin Bull Run Index (CBBI) This is a Python implementation of the Colin Talks Crypto Bitcoin Bull Run Index (CBBI). It makes use

Kamil Monicz 86 Jan 02, 2023
A discord bot to crop an NFT image living on the Solana blockchain.

NFT Discord Cropper This discord bot crops an NFT in your set measures by getting it through the .cache file which has been used to make a candy machi

Rude Golems 7 Mar 21, 2022
Crypto Stats and Tweets Data Pipeline using Airflow

Crypto Stats and Tweets Data Pipeline using Airflow Introduction Project Overview This project was brought upon through Udacity's nanodegree program.

Matthew Greene 1 Nov 24, 2021
Tutela: an Ethereum and Tornado Cash Anonymity Tool

Tutela: an Ethereum and Tornado Cash Anonymity Tool The repo contains open-source code for Tutela, an anonymity tool for Ethereum and Tornado Cash use

TutelaLabs 96 Dec 05, 2022
Kyrie Eleison - The best and unique way to encrypt some data or a file safely

Encrypt your important data and files easily and safely with Kyrie Eleison.

Billy 39 Oct 27, 2022
A simple python program to sign text using either the RSA or ISRSAC algorithm with GUI built using tkinter library.

Digital Signatures using ISRSAC Algorithm A simple python program to sign text using either the RSA or ISRSAC algorithm with GUI built using tkinter l

Vasu Mandhanya 3 Nov 15, 2022
A community effort to bring back Duino-Coin

Duino-Coin-Revived A community effort to bring back Duino-Coin! Along with reviving the cryptocurrency, we will add many improvements to it, including

1 Dec 22, 2021
Zach Brewer 1 Feb 18, 2022
Block Chain for RiceSupply Chain and Agriculture Traceability

Block Chain for RiceSupply Chain and Agriculture Traceability Project Under Development Folder: Building a BlockChain Basic blockchain structure using

Chandru S Raghavan 3 Jan 19, 2022
An encryption format offering better security, performance and ease of use than PGP.

An encryption format offering better security, performance and ease of use than PGP. File a bug if you found anything where we are worse than our competition, and we will fix it.

27 Dec 25, 2022
Retrieve ECDSA signature R,S,Z values from blockchain rawtx or txid.

rsz Retrieve ECDSA signature R,S,Z values from blockchain rawtx or txid. Info The script parse the data of rawtx to fetch all the inputs in the transa

iceland 29 Nov 18, 2022
This is a simple application to generate HD wallet addresses for cryptocurrency coins.

HD-Wallet-Address This is a mini service to generate addresses in the master HD-Wallet. It will use py_crypto_hd_wallet package as a base. Prerequisit

Amin Abbasi 1 Dec 16, 2021
Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters.

modeval Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters. Basic U

2 Jan 01, 2022
Enchpyter, is able to encrypt and decrypt words as you determine, of course, according to the alphabet.

Enchpyter Enchpyter is a program do encrypt and decrypt any word you want (just letters). You enter how many letters jumps and write the word, so, the

João Assalim 2 Oct 10, 2022
blockchain address database

Blockchain Address Ownership Database The database is in data/addresses.db This is a SQLite database of addresses from several blockchains. It's obtai

37 Nov 26, 2022