Just a small test with lists in cython

Overview

Test for lists in cython

Algorithm

  1. create a list of 10^4 lists each with 10^4 floats values (namely: 0.1) - 2 nested for
  2. iterate each list and compute the cumulative product - 2 netsted for

Implementations

  • Julia using Julia objects
  • Rust using Rust objects
  • Pure-python
  • Cython pure-python mode with no annotations
  • Cython pure-python mode with Python annotations
  • Cython pure-python mode with Cython annotations
  • Standard Cython

Setup

  • poetry install --no-root
  • poetry run python setup.py build_ext --inplace
  • poetry run python -c "import julia; julia.install()"

Results

  • poetry run python main.py

Tested on:

  • Intel Core i5-8250U - 64 bits - Quad Core L2 cache: 6 MiB
  • Linux 5.4.105-1-MANJARO x86_64
  • GCC 10.2.0
  • Python 3.9.2
  • Cython 0.29.17
  • Julia 1.6
  • Cargo 1.51
  No annotations Annotations from typing Annotations from cython
Pure Python 11.08
C++ Cython (pure-python mode) 02.25 02.81 02.96
C Cython (pure-python mode) 02.23 02.97 02.91
C Cython (.pyx)
02.77
Julia (pyjulia) 03.50
Julia (pyjulia) parallel 02.11
Rust (Pyo3) parallel before 08.72
Rust (Pyo3) parallel after 29.17

Cython is fast, but none of these methods are able to release the GIL. Moreover, in pure-python mode, Cython effectiveness decreases while using typing annotations. Last but not least, it's hard to understand which solution is better with Cython. The average time is 2.7

Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library. Moreover, it's tricky because after having run some code its perfomance decreases.

Numba is still tricky with lists. I tried to use them, but it fails. In my experience, numba lists in nopython mode slows down the code.

Julia is fast (only 30% slower than the average Cython). With multithreading it's even faster than Cython.

Considering echosystem, multithreading and ease of use, Julia is a clear winner here.

Owner
Federico Simonetta
Musician and computer engineer.
Federico Simonetta
Fi - A simple Python 3.9+ command-line application for managing Fidelity portfolios

fi fi is a simple Python 3.9+ command-line application for managing Fidelity por

Darik Harter 2 Feb 26, 2022
Faker is a Python package that generates fake data for you.

Faker is a Python package that generates fake data for you. Whether you need to bootstrap your database, create good-looking XML documents, fill-in yo

Daniele Faraglia 15.2k Jan 01, 2023
FaceBot is a script to automatically create a facebook account using the selenium and chromedriver modules.

FaceBot is a script to automatically create a facebook account using the selenium and chromedriver modules. That way, we don't need to input full name, email and password and date of birth. All will

Fadjrir Herlambang 2 Jun 17, 2022
Tutorial for integrating Oxylabs' Residential Proxies with Selenium

Oxylabs’ Residential Proxies integration with Selenium Requirements For the integration to work, you'll need to install Selenium on your system. You c

Oxylabs.io 8 Dec 08, 2022
A modern API testing tool for web applications built with Open API and GraphQL specifications.

Schemathesis Schemathesis is a modern API testing tool for web applications built with Open API and GraphQL specifications. It reads the application s

Schemathesis.io 1.6k Dec 30, 2022
Cornell record & replay mock server

Cornell: record & replay mock server Cornell makes it dead simple, via its record and replay features to perform end-to-end testing in a fast and isol

HiredScoreLabs 134 Sep 15, 2022
create custom test databases that are populated with fake data

About Generate fake but valid data filled databases for test purposes using most popular patterns(AFAIK). Current support is sqlite, mysql, postgresql

Emir Ozer 2.2k Jan 04, 2023
Python drivers for YeeNet firmware

yeenet-router-driver-python Python drivers for YeeNet firmware This repo is under heavy development. Many or all of these scripts are not likely to wo

Jason Paximadas 1 Dec 26, 2021
FakeDataGen is a Full Valid Fake Data Generator.

FakeDataGen is a Full Valid Fake Data Generator. This tool helps you to create fake accounts (in Spanish format) with fully valid data. Within this in

Joel GM 64 Dec 12, 2022
Code for "SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism"

SUGAR Code for "SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism" Overview train.py: the cor

41 Nov 08, 2022
HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom

hey is a tiny program that sends some load to a web application. hey was originally called boom and was influenced from Tarek Ziade's tool at tarekzia

Jaana Dogan 14.9k Jan 07, 2023
pytest plugin for manipulating test data directories and files

pytest-datadir pytest plugin for manipulating test data directories and files. Usage pytest-datadir will look up for a directory with the name of your

Gabriel Reis 191 Dec 21, 2022
Pytest-rich - Pytest + rich integration (proof of concept)

pytest-rich Leverage rich for richer test session output. This plugin is not pub

Bruno Oliveira 170 Dec 02, 2022
A pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine

pytest-elasticsearch What is this? This is a pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine. It

Clearcode 65 Nov 10, 2022
XSSearch - A comprehensive reflected XSS tool built on selenium framework in python

XSSearch A Comprehensive Reflected XSS Scanner XSSearch is a comprehensive refle

Sathyaprakash Sahoo 49 Oct 18, 2022
A library for generating fake data and populating database tables.

Knockoff Factory A library for generating mock data and creating database fixtures that can be used for unit testing. Table of content Installation Ch

Nike Inc. 30 Sep 23, 2022
This repository has automation content to test Arista devices.

Network tests automation Network tests automation About this repository Requirements Requirements on your laptop Requirements on the switches Quick te

Netdevops Community 17 Nov 04, 2022
The Good Old Days. | Testing Out A New Module-

The-Good-Old-Days. The Good Old Days. | Testing Out A New Module- Installation Asciimatics supports Python versions 2 & 3. For the precise list of tes

Syntax. 2 Jun 08, 2022
Integration layer between Requests and Selenium for automation of web actions.

Requestium is a Python library that merges the power of Requests, Selenium, and Parsel into a single integrated tool for automatizing web actions. The

Tryolabs 1.7k Dec 27, 2022
Implement unittest, removing all global variable and returning values

Implement unittest, removing all global variable and returning values

Placide 1 Nov 01, 2021