The most widely used Python to C compiler

Overview

Welcome to Cython!

Cython is a language that makes writing C extensions for Python as easy as Python itself. Cython is based on Pyrex, but supports more cutting edge functionality and optimizations.

The Cython language is very close to the Python language, but Cython additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code.

This makes Cython the ideal language for wrapping external C libraries, and for fast C modules that speed up the execution of Python code.

You can support the Cython project via Github Sponsors or Tidelift.

Installation:

If you already have a C compiler, just run following command:

pip install Cython

otherwise, see the installation page.

License:

The original Pyrex program was licensed "free of restrictions" (see below). Cython itself is licensed under the permissive Apache License.

See LICENSE.txt.

Contributing:

Want to contribute to the Cython project? Here is some help to get you started.

We are currently building the next great Cython edition: Cython 3.0. You can help us make the life of Python 3.x users easier.

Get the full source history:

Note that Cython used to ship the full version control repository in its source distribution, but no longer does so due to space constraints. To get the full source history from a downloaded source archive, make sure you have git installed, then step into the base directory of the Cython source distribution and type:

make repo

The following is from Pyrex:

This is a development version of Pyrex, a language for writing Python extension modules.

For more info, take a look at:

  • Doc/About.html for a description of the language
  • INSTALL.txt for installation instructions
  • USAGE.txt for usage instructions
  • Demos for usage examples

Comments, suggestions, bug reports, etc. are most welcome!

Copyright stuff: Pyrex is free of restrictions. You may use, redistribute, modify and distribute modified versions.

The latest version of Pyrex can be found here.

Greg Ewing, Computer Science Dept
University of Canterbury
Christchurch, New Zealand
A citizen of NewZealandCorp, a wholly-owned subsidiary of USA Inc.
Comments
  • Large output size of compiled code

    Large output size of compiled code

    Hi All,

    I'm trying to reduce the packaged size of the scientific computing ecosystem to make it friendlier for distributed deployments using tools like docker, kubernetes, etc.. An issue for this topic generally is here: https://github.com/ContinuumIO/anaconda-issues/issues/8242

    One important issue that came up from @mingwandroid relates to Cython, see https://github.com/ContinuumIO/anaconda-issues/issues/8242#issuecomment-359524128

    It seems the shared libraries are chock full of instruction code

    PyInit_cython_special is huge; more than 1MB.

    why all of these movq instructions are not being coalesced into a memset.

    In a separate conversation he said the following:

    Now ideally you'd like your compiler to be smart enough to coalesce all those clears over the whole object into 1 memset, and then even better, put them all together so that a single memset could do it. the package size on disk is swamped by the instructions to do these clears! instead of a call to a function to memset it's a huge amount of instructions, like tens of thousands. and that takes space on the extension module text segment.

    To be honest, I don't have much expertise here, but I thought I'd raise an issue here in case this sort of problem can be resolved within Cython, or, if not, if anyone here has any additional helpful thoughts.

    Thank you all for your time.

    enhancement Code Generation Build System 
    opened by mrocklin 52
  • Verbatim C code using docstring syntax.

    Verbatim C code using docstring syntax.

    Allow including snippets of C code in a cdef extern from * block using docstring syntax:

    cdef extern from *:
        """
        static long square(long x)
        {
            return x * x;
        }
        """
        long square(long)
    

    This is an alternative to adding an external .h file containing the C code. This would be useful especially for simple things, like a one-line macro.

    I still have to write documentation, but I'll wait for feedback first.

    Edit: This is currently a syntax error and thus cannot conflict with existing code.

    enhancement 
    opened by jdemeyer 50
  • Make GitHub Actions work on Windows

    Make GitHub Actions work on Windows

    I added tests for all versions of python to windows and solved some problems to let them work, but some don't: 2.7, 3.4 - since no suitable compiler version is installed 3.9 - inline, details: #3450, #4379 3.5 - one cpp-only test fails, output

    I also uncommented the coverage tests and added some code to the .sh to make them work. They work for a decent amount of time, which is the opposite of what was said in the comments.

    If anyone wants to try to fix the problems and run 2.7 (and maybe 3.4), then you will need to use this, probably relying on both versions of this answer.

    Testing 
    opened by 0dminnimda 39
  • cdef dataclasses

    cdef dataclasses

    Used cython.dataclasses.dataclass and cython.dataclasses.field to mark dataclasses and their fields.

    Tries to match the interface provided by a regular dataclass as much as possible. This means taking the types from the dataclasses module if available (so they match exactly) or a fallback Python version that just implements the core parts (obtained with PyRun_SimpleString in the C source).

    Use of placeholders in generated __init__ code means the code in the C file isn't hugely readable. Probably not a huge issue, but don't really see a way round that.

    As part of this I've also also implemented a Cython version of typing.ClassVar. Although really designed for use with dataclasses it behaves sensibly when used in types in a normal cdef class. Potentially this might be worth documenting more thoroughly?

    Status

    • [x] Both annotated variables and cdef attributes included in dataclass - done (but assignment syntax for cdef attributes is a bit clunky because it needs to be on a separate line)
    • [x] visibility of the attributes decided to be visible by default for annotations, invisible for cdef attributes
    • [x] non-public attributes omitted from __dataclass_fields__
    • [x] moving "directives" into cython.dataclasses and cython.typing submodules
      • [ ] I'd quite like these cython.dataclasses submodules and their attributes to be available at runtime and just forwarded to their standard library modules if available. This may be fiddly
    • [x] "frozen" option of dataclasses works. (Obviously hard to enforce at C level)

    Old commentary on design decisions (can now mostly be ignored)

    When finished closes https://github.com/cython/cython/issues/2903 - however, some design decisions pending before it's finished:

    What attributes should be included in the dataclass? Definitely annotated variables. Maybe regular cdef variables?

    What should the visibility of the attributes be? There's a few options:

    1. Default to invisible, like for a standard cdef class. This is obviously consistent. The issue with this is that there's a few functions in the dataclasses module like asdict which assume that every attribute declared in __dataclass_fields__ is readable. If they aren't then the classes won't be compatible with those interfaces.

      If so, should non-public attributes be omitted from __dataclass_fields__? This seems inconsistent since they do appear in the destructor, the repr, and affect the comparisons.

    2. Default to visible. This is inconsistent with the standard cdef class behaviour, but would make them as compatible as possible with standard dataclasses. It would also make sense for most use-cases I think. One problem is that the syntax doesn't really exist of override that (only public and readonly are defined).

    3. Annotated variables default to visible, cdef variables to invisible? It kind of makes sense and gives a way to control visibility, but it'd be a little inconsistent with everything. (I'm leaning towards this as the answer)

    The likely implementation deviates from the normal cdef class behaviour where

    cdef class C:
       a: `int` = 0
    

    makes a class-variable instead of an instance variable. I think this is unavoidable and makes sense in this case, but comments on this welcome too?

    It dumps a number of names in the cython scope (dataclass, field, InitVar, ClassVar). Would some sort of subscoping be better? Especially given that it isn't 100% obvious that any of these but dataclass related to dataclasses?

    feature Cython Language Feature 
    opened by da-woods 39
  • Pythonise the documentation according to #4187: Basic Tutorial (cython_tutorial.rst)

    Pythonise the documentation according to #4187: Basic Tutorial (cython_tutorial.rst)

    This is the first step in adding Pure Python code versions to all Cython documentation!

    Note, since the current tests for .py files do not add cython to namespace, some tests are expected to fail due to the convention not to use import cython in most pure python files.

    To be able to try this, you will need to install doc-requirements.txt (it has been updated).

    Lots of process details and standardization are found here: #4187

    Small preview: Small preview

    Documentation 
    opened by 0dminnimda 36
  • walrus operator/named expressions

    walrus operator/named expressions

    Fixes https://github.com/cython/cython/issues/2636

    With the exception of the the parser changes I don't think this is too bad so I'm marking it as ready.

    It implements the assignment expression largely as a composite of existing nodes, which I think is a reasonable approach.

    feature Python Semantics 
    opened by da-woods 36
  • no attribute __reduce_cython__

    no attribute __reduce_cython__

    https://github.com/cython/cython/issues/1894#issuecomment-339966952.

    I'm getting an AttributeError due to a missing __reduce_cython__ attribute in an embedded environment when I build lxml with Cython 0.26 or 0.27. 0.25(.2) works fine. The issue seems to be triggered by reinitializing the environment but unfortunately I was not able to find a minimal sample that replicates it yet.

    I did a git-bisect and found https://github.com/cython/cython/commit/f8b3405e926d2ba9bc2ee24d79848235875ee12e to be the first broken commit.

    Will try to find a simple test case, but I'm not sure how soon I'll have a result.

    EDIT: This was resolved in Cython 0.28. See https://github.com/cython/cython/issues/1953#issuecomment-398128940.

    opened by cschramm 36
  • Late includes

    Late includes

    This pull request adds support for a late #include. My proposed syntax is to add a leading pipe to the filename:

    cdef extern from "|spam.h":
        ...
    

    For this cdef extern block, Cython will generate the #include statement after its own variable declarations. So it can be used if the C code in spam.h needs to refer to Cython variables.

    cysignals has a use-case which is currently "solved" by some ugly hack involving .pxi files (for which I need #483). If this pull request is accepted, cysignals would no longer need to rely on .pxi files: https://github.com/sagemath/cysignals/pull/49

    Of course, there are plenty of bikeshedding opportunities for the syntax, but I care about the feature, not the syntax. I chose the leading pipe because it is unlikely to occur in actual filenames. (Also: think of the pipe as "piping" the Cython variables into the header file).

    opened by jdemeyer 36
  • [ENH] C functions should propagate exceptions by default

    [ENH] C functions should propagate exceptions by default

    Is your feature request related to a problem? Please describe. C functions that do not return Python objects cannot currently propagate exceptions by default but require an explicit except clause. In Python code, declared return types default to safe exception propagation instead. Both should behave the same.

    cdef int func():
        raise TypeError  # is not propagated
    
    @cython.cfunc
    def pyfunc() -> cython.int:
        raise TypeError  # is propagated
    
    cdef int no_exc_func():
        return 0  # no exception raised
    
    cdef extern from *:
        int no_exc_cfunc()  # no exception expected
    

    Describe the solution you'd like C functions should also propagate their exceptions by default.

    cdef int func():
        raise TypeError  # CHANGE: should get propagated as with `except? -1`, the default exception value.
    
    @cython.exceptval(check=False)
    cdef int func():
        raise TypeError  # is not propagated
    
    cdef int no_exc_func():
        return 0  # CHANGE: no exception raised, but tested by caller
    
    cdef int explicit_no_exc_func() noexcept:    # <- CHANGE
        return 0  # no exception raised, and not tested by caller
    
    cdef extern from *:
        int no_exc_cfunc()  # unclear - should exceptions also be expected for `extern` functions?
    

    Questions:

    • non-extern cdef functions declared in .pxd files should probably be affected, too, since they should match the implementation in the corresponding .pyx / .py file.
    • C functions defined in cdef extern blocks are unlikely to benefit from this change. Can we live with keeping them different?

    Also see https://github.com/cython/cython/issues/3580 https://github.com/cython/cython/issues/933

    enhancement Python Semantics P: blocker 
    opened by scoder 35
  • Add a Pythran backend for Numpy operation

    Add a Pythran backend for Numpy operation

    I've been working for quite some time on the usage of Pythran as a backend for the Numpy operations that Cython can generate. The associated PR on github can be found here: . This work has been sponsored by the OpenDreamKit project (https://github.com/OpenDreamKit/OpenDreamKit/).

    First of all, the Pythran project (https://github.com/serge-sans-paille/pythran) is a (subset of) Python to C++ compiler, that aims at optimizing "scientific" Python code. It also provides a full C++ implementation of a major set of the Numpy API. Some of the advantage of this implementation is that it supports expression templates and SIMD instructions (partially thanks to Boost.SIMD [1]).

    One of the limitation of the current Numpy support of Cython is that it relies on the original Numpy Python module for a lot of computations. The overall idea is to replace these calls by the Numpy implementation provided within the Pythran project.

    I'll discuss in this mail the various choices that have been made, why and some implementation details. Then we'll also show some benchmark to see the potential improvements, which is the point of all this in the end :)

    Pythran limitations

    The Pythran Numpy implementation has some limitations:

    • array "views" are not supported. That means that arrays must be stored in contiguous memory. Fortran and C-style format are supported.
    • the endianness of the integers must be the same that the one of the targeted architecture (note that Cython has the same limitation)

    That's why we did two things:

    • the usage of the Pythran backend needs to be explicitly asked by the user by providing the --np-pythran flag to the Cython compiler, or by using the "np_pythran" flag to the cythonize call (for distutils)
    • in function arguments, Numpy buffers are replaced by fused types to be able to fall back in case of unsupported buffers. More on this below.

    Implementation choices and details within Cython

    a) PythranExpr

    We defined a new type in PyrexTypes.py, which defines a Pythran buffer or expression. A Pythran expression is associated to a Pythran expression template, whose C++ type can be something like "decltype(a+b)". We thus compose every expression/function call like this, which allows us to use Pythran's expression template mechanism.

    We also choose to let the C++ compiler deduced the final type of every expression, and emit errors if something goes wrong. This choice allows not to have to rewrite in Python all the (potentially implicit) conversion rules that can apply in a C/C++ program, which could be error prone. The disadvantage is that it may generate not that trivial error messages for the end-user.

    b) Fused types for function arguments

    As Pythran has some limitations about the Numpy buffers it can support, we chose to replace Numpy buffer arguments by a fused type that can be either a Pythran buffer or the original Numpy buffer. The decision is made to use one type or another according to the limitations described above.

    This allows a fallback to the original Cython implementation in case of an unsupported buffer type.

    Tests

    A flag has been added to the runtests.py script. If provided with a path to a Pythran installation, it will run the C++ tests in "Pythran" mode. This allows to reuse the whole test suite of Cython.

    Benchmark

    The whole idea of this is to get better performances.

    Here is a simple benchmark of what this mode can achieve, using this cython code:

    def sqrt_sum(numpy.ndarray[numpy.float_t, ndim=1] a, numpy.ndarray[numpy.float_t, ndim=1] b):
        return numpy.sqrt(numpy.sqrt(a*a+b*b))
    

    On my computer (Core i7-6700HQ), this gives there results, with an array of 100000000 32-bit floats as input:

    • for the classical Cython version: 960ms
    • for the Cython version using the Pythran backend: 761ms
    • for the Cython version using the Pythran backend using SIMD instructions: 243ms

    which makes a speedup of ~3.9x using SIMD instructions.

    Documentation

    I put an example of how to use this with distutils in the documentation. It could be put elsewhere if needed, or formatted differently.

    opened by aguinet 34
  • Add create_extension() hook

    Add create_extension() hook

    As alternative to #412, add support for a hook function create_extension() allowing complete customization of creating the Extension object after Cython has processed the list of sources and # distutils declarations.

    opened by jdemeyer 34
  • [BUG] Using custom generic type in type alias is not supported

    [BUG] Using custom generic type in type alias is not supported

    Describe the bug

    Cython emits TypeError when TypeAlias contains custom Generic type, while mypy passes it.

    Code to reproduce the behaviour:

    from typing import Generic, TypeVar, TypeAlias
    
    T_co = TypeVar("T_co", covariant=True)
    
    
    class ExprProxy(Generic[T_co]):
        def __init__(self, value) -> None:
            self.value: T_co = value
    
    
    class ConstExpr:
        pass
    
    
    Evaluable: TypeAlias = ConstExpr | int | ExprProxy[ConstExpr]
    # TypeError: 'type' object is not subscriptable
    

    Expected behaviour

    Compile successfully without TypeError

    Environment

    OS: Windows Python version : 3.10.9 Cython version : 0.29.32 and 3.0.0a11 c97b77a

    Additional context

    Full source: https://github.com/armoha/eudplib/blob/f933c6b8da5d1b9bde9327c26719ff066380f405/eudplib/core/allocator/constexpr.pyx#L274

    opened by armoha 1
  • Update Cython Debugger

    Update Cython Debugger

    Wanted to see if we could modernize this a bit - looks like a lot of the documentation still wants users to use Python2.7 alongside this.

    Not sure everything here is 100% correct but I think still represents progress over what is there today. You can get a working Cython debugger image and mini instructions from this Docker repo:

    https://hub.docker.com/repository/docker/willayd/cython-debug

    opened by WillAyd 0
  • [BUG] Misleading error message when trying to cimport in pure Python

    [BUG] Misleading error message when trying to cimport in pure Python

    Describe the bug

    If you use the cimport keyword in pure Python it doesn't work (correctly). However, the error message suggests that you should use the cimport keyword

    Code to reproduce the behaviour:

    from cython.operator cimport dereference
    

    Output is:

    from cython.operator cimport dereference
                         ^
    ------------------------------------------------------------
    
    somefilename.py:1:21: Expected 'import' or 'cimport'
    

    Expected behaviour

    A useful error message. This isn't quite as simple as it looks because the cython module wants to be imported, while most other thing want from cython.cimports.... Although given that this is at the parsing stage, it'd probably be sufficient to just write expected "import"

    Environment

    Current master

    Additional context

    No response

    Error Reporting 
    opened by da-woods 0
  • [docs] Parallelization tutorial

    [docs] Parallelization tutorial

    I've written a parallelization tutorial. It doesn't cover that much new ground from the main parallelization documents, but hopefully it present a separate example of each use case, and is a little less of "here are the parameters, good luck" than the main parallelization reference

    Kind of closes https://github.com/cython/cython/issues/5125

    Documentation 
    opened by da-woods 3
  • [BUG] Syntax error in C array declaration and initialization

    [BUG] Syntax error in C array declaration and initialization

    Describe the bug

    I found it in #5177.

    def main():
        cdef int arr[4] = [1, 2, 3, 4]
    

    This code doesn't compile:

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
    def main():
        cdef int arr[4] = [1, 2, 3, 4]
                        ^
    ------------------------------------------------------------
    
    test.pyx:2:20: Syntax error in C variable declaration
    

    But these two do:

    def main():
        cdef int arr1[4]
        arr1 = [1, 2, 3, 4]
    
        cdef int[4] arr2 = [1, 2, 3, 4]
    

    I think it is already known, because it has been mentioned in the documentation:

    • https://github.com/cython/cython/blob/b2cad768499cf5763251fe757491e9fc24a1f583/docs/src/userguide/language_basics.rst#L124-L129

    but I couldn't find the related issue.

    Anyway, if it can't be fixed easily, it will be good to document it more clearly.

    Code to reproduce the behaviour:

    No response

    Expected behaviour

    No response

    Environment

    Cython version: both 0.29.32 and 3.0.0a11

    Additional context

    No response

    opened by GalaxySnail 0
Releases(3.0.0a11)
A Lite Package focuses on making overwrite and mending functions easier and more flexible.

Overwrite Make Overwrite More flexible In Python A Lite Package focuses on making overwrite and mending functions easier and more flexible. Certain Me

2 Jun 15, 2022
A frontend to ease the use of pulseaudio's routing capabilities, mimicking voicemeeter's workflow

Pulsemeeter A frontend to ease the use of pulseaudio's routing capabilities, mimicking voicemeeter's workflow Features Create virtual inputs and outpu

Gabriel Carneiro 164 Jan 04, 2023
๐ŸŒˆPython cheatsheet for all standard libraries(Continuously Updated)

Python Standard Libraries Cheatsheet Depend on Python v3.9.8 All code snippets have been tested to ensure they work properly. Fork me on GitHub. ไธญๆ–‡ En

nick 12 Dec 27, 2022
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
A totally unrealistic cell growth/reproduction simulation.

A totally unrealistic cell growth/reproduction simulation.

Andrien Wiandyano 1 Oct 24, 2021
A clock purely made with python(turtle)...

Clock A clock purely made with python(turtle)... Requirements Pythone3 IDE or any other IDE Installation Clone this repository Running Open this proje

Abhyush 1 Jan 11, 2022
Web app to find your chance of winning at Texas Hold 'Em

poker_mc Web app to find your chance of winning at Texas Hold 'Em A working version of this project is deployed at poker-mc.ue.r.appspot.com. It's run

Aadith Vittala 7 Sep 15, 2021
Demo of patching a python context manager

patch-demo-20211203 demo of patching a python context manager poetry install poetry run python -m my_great_app to run the code poetry run pytest to te

Brad Smith 1 Feb 09, 2022
Interfaces between napari and pymeshlab library to allow import, export and construction of surfaces.

napari-pymeshlab Interfaces between napari and the pymeshlab library to allow import, export and construction of surfaces. This is a WIP and feature r

Zach Marin 4 Oct 12, 2022
NASH 2021 project... this may or may not end up working ๐Ÿคทโ€โ™‚๏ธ

wavespace synthesiser this is my NASH 2021 project, which may or may not end up working ๐Ÿคทโ€โ™‚๏ธ what is going on? imagine you have a big folder of audio

Ben Hayes 12 May 17, 2022
Collection of Python scripts to perform Eikonal Tomography

Collection of Python scripts to perform Eikonal Tomography

Emanuel Kรคstle 10 Nov 04, 2022
Url-check-migration-python - A python script using Apica API's to migrate URL checks between environments

url-check-migration-python A python script using Apica API's to migrate URL chec

Angelo Aquino 1 Feb 16, 2022
1. ๋„ค์ด๋ฒ„ ์นดํŽ˜ ๋Œ“๊ธ€์„ ๋นจ๋ฆฌ ๋‹ค๋Š” ๊ธฐ๋Šฅ

naver_autoprogram ๊ธฐ๋Šฅ ์„ค๋ช… ๋„ค์ด๋ฒ„ ์นดํŽ˜ ๋Œ“๊ธ€์„ ๋นจ๋ฆฌ ๋‹ค๋Š” ๊ธฐ๋Šฅ ๋„ค์ด๋ฒ„ ์นดํŽ˜ ์ž๋™ ์ถœ์„ ์ฒดํฌ ๊ธฐ๋Šฅ ๋™์ž‘ ๋ฐฉ์‹ ์นดํŽ˜ ๋Œ“๊ธ€ ๊ธฐ๋Šฅ ๊ธฐ๋ณธ ๋™์ž‘์€ ์ฃผ๊ธฐ์ ์ธ ์Šค์ผ€์ฅด ๋™์ž‘์œผ๋กœ ํ•ด๋‹น ์นดํŽ˜ ID ์™€ ํŠน์ • API ์ฃผ์†Œ๋กœ ๋Œ€์ƒ์ด ์ƒˆ๊ธ€์„ ์ž‘์„ฑํ–ˆ๋Š”์ง€ ์ฒดํฌ. ํ•ด๋‹น ๋Œ€์ƒ์ด ์ƒˆ๊ธ€ ๋“ฑ

1 Dec 22, 2021
Sardana integration into the Jupyter ecosystem.

sardana-jupyter Sardana integration into the Jupyter ecosystem.

Marc Espรญn 1 Dec 23, 2021
A pypi package details search python module

A pypi package details search python module

Fayas Noushad 5 Nov 30, 2021
An addin for Autodesk Fusion 360 that lets you view your design in a Looking Glass Portrait 3D display

An addin for Autodesk Fusion 360 that lets you view your design in a Looking Glass Portrait 3D display

Brian Peiris 12 Nov 02, 2022
Recreate the joys of Office Assistant from the comfort of the Python interpreter

Recreate the joys of Office Assistant from the comfort of the Python interpreter.

Louis Sven Goulet 3 May 21, 2022
This python module allows to extract data from the RAW-file-format produces by devices from Thermo Fisher Scientific.

fisher_py This Python module allows access to Thermo Orbitrap raw mass spectrometer files. Using this library makes it possible to automate the analys

8 Oct 14, 2022
A project for Perotti's MGIS350 for incorporating Flask

MGIS350_5 This is our project for Perotti's MGIS350 for incorporating Flask... RIT Dev Biz Apps Web Project A web-based Inventory system for company o

1 Nov 07, 2021
๐ŸŽ‰ ๐ŸŽ‰ PyComp - Java Code compiler written in python.

๐ŸŽ‰ ๐ŸŽ‰ PyComp Java Code compiler written in python. This is yet another compiler meant for babcock students project which was created using pure python

Alumona Benaiah 5 Nov 30, 2022