Producing Mercurial wheels from Github Actions + modernize build process

[Pierre-Yves David]

On 9/27/24 09:14, PIERRE AUGIER wrote:
> Hello,
>
> I tried to see if Mercurial wheels could be produced on Github Actions.
>
> https://github.com/paugier/mercurial-devel/blob/refs/heads/wheels-with-github-actions/.github/workflows/wheels.yml
>
> Nothing works because simple standard commands do not work for Mercurial.


I am not surprised. As you noticed our setup.py is really ancient and 
has really not caught up with evolution of python packaging of the last 
15 years.



> https://github.com/paugier/mercurial-devel/actions/runs/11064702442
>
> Even producing the sdist fails:
>
> Run python -m build --sdist
> * Creating isolated environment: venv+pip...
> * Installing packages in isolated environment:
>    - setuptools
>    - wheel
> * Getting build dependencies for sdist...
> /!\
> /!\ Could not determine the Mercurial version
> /!\ You need to build a local version first
> /!\ Run `make local` and try again
> /!\
> Run `make local` first to get a working local version
>
> And same error for wheels:
>
> python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps
>    Processing /project
>      Installing build dependencies: started
>      Installing build dependencies: finished with status 'done'
>      Getting requirements to build wheel: started
>      Getting requirements to build wheel: finished with status 'error'
>      error: subprocess-exited-with-error
>      
>      × Getting requirements to build wheel did not run successfully.
>      │ exit code: 1
>      ╰─> [6 lines of output]
>          /!\
>          /!\ Could not determine the Mercurial version
>          /!\ You need to build a local version first
>          /!\ Run `make local` and try again
>          /!\
>          Run `make local` first to get a working local version
>          [end of output]
>
>
> I had a look at the Makefile and setup.py. It seems that they contain a lot of legacy code and that a simplification and modernization would be welcome. For example, setup.py should never be called directly (and it is even better if we can avoid it).
>
> I guess I don't understand why setup.py has to be so complex and I suspect that it would much simpler with another build system, for example Meson (used by Numpy, Scipy and many other projectshttps://mesonbuild.com/Users.html).
>
> I have a bit of experience with using Meson so I can help if one wants to give it a try for Mercurial build.


I don't think we need to look at a new build system yet. Mercurial is 
basically still using 2025's distutils. Modern python packaging is 
likely to be able to do what we need.

However this means the setup.py need to be understand and migrated to 
something more modern. If you are willing to give it a try, I would be 
happy to do a couple of call to help you understand what's in there.

(note: I am not against using something else, but I suspect we don't 
need to)


> Regarding Rust, if I understand correctly, it is now a build option leading to two different Mercurial wheels. Meson-python can also have build options, but this is not good practice for the PyPA. Did you think about having the Rust extensions (and binaries) in another PyPI package (like mercurial-rust)?

We currently have 3 "flavor": "py", "c" and "c+rust". Pure does not 
compile anything, "c" use some compiled CPython extension, "c+rust" also 
have some Rust extensions.

I remember using build option to install different version of dulwich 
from PyPA, so  I guess we could still do. However, if we start to build 
whell, this open the way to shipping rust extension more broadly.


While looking around I discovered `|cibuildwheel` that is compatible 
with Gitlab so it probably works fine with Heptapod and we could be 
building wheel from the main repository directly. … but we need to fix 
the setup.py first…|

Thanks for looking into this !

-- 
Pierre-Yves David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20240927/414abf86/attachment.html>