Repository navigation
Investigate using Py_LIMITED_API to reduce release size #1219
Description
Currently our release size is around 1GB due to the large number of wheels we produce. This has started being a problem since we have hit the default 10 GB limit on PyPI, which is currently holding up release 11.0.0, see pypi/support#3324.
Cython 3 has added support for Python's limited API - which means a single wheel can cover multiple Python versions. This would greatly reduce the size of our releases and possibly speed up CI build times too.
Activity
WyattBlue commentedon Nov 14, 2023
Cython does not support Py_LIMITED_API. See still open issue
jlaine commentedon Nov 14, 2023
The changelog says otherwise:
https://cython.readthedocs.io/en/latest/src/changes.html#initial-support-for-limited-api
da-woods commentedon Dec 9, 2023
To clarify - it's very preliminary in Cython 3. Some very trivial test examples will compile and run in with
Py_LIMITED_APIbut most won't.Cython 3.1 should be better - it'll still be fairly experimental but there's a decent chance things will work (although no promises). The main limitation is memoryviews which aren't possible to support on Python <3.11. No info on likely release dates of Cython 3.1 but I'd hold off on spending much time on this until then.
There may also be some limitations in build tooling that you've have to work around (e.g. setuptools doesn't give compiled files the right version tags)
jlaine commentedon Dec 27, 2023
@da-woods thanks for investigating regarding the current state of Cython.
The tags issue is trivial to work around, i have for instance done so for the "aioquic" package.
WyattBlue commentedon Jul 25, 2024
See: https://github.com/cython/cython/pull/6297/files