Producing Mercurial wheels from Github Actions + modernize build process

Thu Oct 10 21:39:55 UTC 2024

Hi all,

I checked two things:

1. Github Actions (for Linux, Windows AMD, macos x86_64 and macos arm) are really free for all public repositories, with no explicit limits.

I wondered how so many open-source projects (some very big with long CI) can rely on GA for testing and other stuffs. This is the simple explanation. 

"GitHub Actions usage is free for standard GitHub-hosted runners in public repositories. For private repositories, each GitHub account receives a certain amount of free minutes and storage for use with GitHub-hosted runners, depending on the account's plan. Any usage beyond the included amounts is controlled by spending limits."

See https://github.com/features/actions and https://docs.github.com/en/billing/managing-billing-for-your-products/managing-billing-for-github-actions/about-billing-for-github-actions

Of course Mercurial can use a public repo (like https://github.com/paugier/mercurial-ga-ci-cache-repo) to use Github Actions for free.

2. Gitlab CI is officially supported by cibuildwheel but I can't find a project really using this.

Among the numerous examples listed in https://cibuildwheel.pypa.io/en/stable/working-examples/ (more than 100), there are only 2 mentions of Gitlab CI (examples) and all projects are hosted on Github.

I tried this https://foss.heptapod.net/mercurial/mercurial-devel/-/merge_requests/989 to build Linux wheels on Heptapod CI but it failed. It seems that cibuildwheel has to use Docker in docker on Gitlab CI. It might actually be simple but I don't know how it could work on Heptapod CI. George, what do you think? Does anyone understand how to fix that? I used to use Kaniko on Heptapod CI but not Docker in docker...

So given these facts, I wonder if it would not be reasonable to reconsider the Github Actions solution, at least while there are no other solutions. It can really be setup in few days with a minimal amount of work. It can be ready for the next release so that users would benefit from wheels.

Raphaël, I also think that we do not need to run all the test suite from all wheels to upload wheels on PyPI. It would technically be possible since with https://github.com/paugier/mercurial-ga-ci-cache-repo, we really have the full Mercurial mercurial-devel repo available and Github Actions are free for public repositories (now even on mac arm if I understood correctly). But I don't see any reason why to do that. It could be sufficient to ensure that few standard things work. Once we are sure that these wheels can be used to install a Mercurial that has been compiled from the right version (a release version that has been extensively tested), that all extensions can be imported and used and that few basic commands work as expected, there is no reason that these wheels would be more broken than wheels compiled locally by people (because cibuildwheel is used by hundreds of projects). Of course, Mercurial developers can also check manually few wheels created on the CI just before the release (the wheels can already be downloaded from the runs page, for example https://github.com/paugier/mercurial-ga-ci-cache-repo/actions/runs/11248552560). I'm not saying that there won't be any problems but (i) for most users it will be a benefit (on most standard platforms, it's just going to work, in particular because it will be checked), (ii) some wheels issues won't be detected by running the full test suite again on the CI used to build the wheels and (iii) wheels issues like ImportError have a simple "fix" for the users: just recompile locally with `pip install mercurial --no-binary mercurial`.

I also would like to react to this:

>     - Wheels can be computed outside of a given release timing, so we
> don't have to "catch the next release window" per se.

Projects uploading wheels upload first the wheels and then the sdist on PyPI so that there is no bad surprise for users with a pip install leading to compilation, which can fail or produce a wheel with a different flavor than the wheels from PyPI (without the Rust extensions for example). All projects that I studied using cibuildwheel upload wheels and sdist from the same CI (in practice running on Github Actions). However, for the next release, I agree that it does not apply!

I'm going to improve https://github.com/paugier/mercurial-ga-ci-cache-repo in different directions:

- full test suite from the repo on Windows and macos.
- compiling the Linux wheels on GA with Rust support. It would be nice to be able to have the default module policy to "C with a fallback to Python".
- simple tests from the wheels: ability to install, import extensions and run few commands.
- possibility to manually trigger the workflows.

Even without automatic link between foss.heptapod.net and Github, I think this work can be a good and cheap temporary solution for testing on platforms not currently tested and build+upload wheels.

Of course, I would also be very happy when a solution based on Heptapod CI would be ready. Let's see what can be done with https://foss.heptapod.net/mercurial/mercurial-devel/-/jobs/2595718

Pierre

----- Mail original -----
> De: "Raphaël Gomès" <raphael.gomes at octobus.net>
> À: mercurial-devel at lists.mercurial-scm.org
> Cc: "PIERRE AUGIER" <pierre.augier at univ-grenoble-alpes.fr>
> Envoyé: Jeudi 10 Octobre 2024 15:53:18
> Objet: Re: Producing Mercurial wheels from Github Actions + modernize build process

> Hi all,
> 
> Pierre-Yves, Pierre and jumped into a VC yesterday to discuss this
> progress and how to best fit the needs of the project while keeping this
> new momentum going.
> 
> Here are the main takeaways:
>     - The Github free tier wouldn't stretch more than a couple runs of
> CI of MacOS/Windows, so it's out of the question even outside of other
> issues we may have with relying on Github for anything Mercurial
>     - We shouldn't be distributing wheels to users without testing
> them, so building a Windows wheel also implies getting a working CI at
> least once every release
>     - Wheels would be equally easy to create in a Heptapod runner
> provided we have the right hardware for each platform
>     - We are currently making progress in fixing the tests for Windows
> and MacOS and setting up runners at least for Windows, thanks in large
> part to Matt Harbison and the Heptapod community
>     - Octobus has two Apple Silicon machines that should be setup as
> runners within the next few weeks
>     - We should create our wheels with everything inside (Python, C and
> Rust on tested platforms, so Linux for now) and set the modulepolicy to
> be "allow" (C with a fallback to Python) by default. This way we have a
> single wheel, and users that want to opt-in to Rust will be able to do
> it without affecting the rest of the users. Maybe we'll turn on Rust
> whenever available at some point, but that implies a little more work in
> core to make it more clever about when to use Rust.
>     - Wheels can be computed outside of a given release timing, so we
> don't have to "catch the next release window" per se.
> 
> Here is the plan we've agreed to:
>     - Pierre Augier will add a manual job in our heptapod CI to create
> a wheel for Linux only. This is an easy step and will already make us
> move forward a little bit
>     - He also agreed to try (but not promise) to take a look at
> improving our setup.py so it's less... horrible.
>     - We will keep making progress with the Windows and MacOS CI as
> well as setting up their runners
>     - We will probably set up a post-landing CI for Windows and MacOS
> because our workers will be too overloaded/slow to be in the normal CI.
> This is not ideal, but until we have more/faster hardware, this is
> already much better than the current situation.
>     - Once that is all set up, I - being the release manager - will
> have to figure out how to best handle the publications of the wheels for
> each release
> 
> I hope I haven't forgotten anything, but in any case, I'm glad that this
> is moving along. :)
> 
> Thank you again to everyone involved,
> Raphaël
> 
> On 10/7/24 11:09 AM, Pierre-Yves David wrote:
>>
>> I just realized that this discussion is taking place on the user
>> mailing list. We should move it on the developer mailing list.
>>
>> On 9/27/24 09:14, PIERRE AUGIER wrote:
>>> Hello,
>>>
>>> I tried to see if Mercurial wheels could be produced on Github Actions.
>>>
>>> https://github.com/paugier/mercurial-devel/blob/refs/heads/wheels-with-github-actions/.github/workflows/wheels.yml
>>>
>>> Nothing works because simple standard commands do not work for Mercurial.
>>>
>>> https://github.com/paugier/mercurial-devel/actions/runs/11064702442
>>>
>>> Even producing the sdist fails:
>>>
>>> Run python -m build --sdist
>>> * Creating isolated environment: venv+pip...
>>> * Installing packages in isolated environment:
>>>    - setuptools
>>>    - wheel
>>> * Getting build dependencies for sdist...
>>> /!\
>>> /!\ Could not determine the Mercurial version
>>> /!\ You need to build a local version first
>>> /!\ Run `make local` and try again
>>> /!\
>>> Run `make local` first to get a working local version
>>>
>>> And same error for wheels:
>>>
>>> python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps
>>>    Processing /project
>>>      Installing build dependencies: started
>>>      Installing build dependencies: finished with status 'done'
>>>      Getting requirements to build wheel: started
>>>      Getting requirements to build wheel: finished with status 'error'
>>>      error: subprocess-exited-with-error
>>>      
>>>      × Getting requirements to build wheel did not run successfully.
>>>      │ exit code: 1
>>>      ╰─> [6 lines of output]
>>>          /!\
>>>          /!\ Could not determine the Mercurial version
>>>          /!\ You need to build a local version first
>>>          /!\ Run `make local` and try again
>>>          /!\
>>>          Run `make local` first to get a working local version
>>>          [end of output]
>>>
>>>
>>> I had a look at the Makefile and setup.py. It seems that they contain a lot of
>>> legacy code and that a simplification and modernization would be welcome. For
>>> example, setup.py should never be called directly (and it is even better if we
>>> can avoid it).
>>>
>>> I guess I don't understand why setup.py has to be so complex and I suspect that
>>> it would much simpler with another build system, for example Meson (used by
>>> Numpy, Scipy and many other projectshttps://mesonbuild.com/Users.html).
>>>
>>> I have a bit of experience with using Meson so I can help if one wants to give
>>> it a try for Mercurial build.
>>>
>>> Regarding Rust, if I understand correctly, it is now a build option leading to
>>> two different Mercurial wheels. Meson-python can also have build options, but
>>> this is not good practice for the PyPA. Did you think about having the Rust
>>> extensions (and binaries) in another PyPI package (like mercurial-rust)?
>>>
>>> Pierre
>>>
>>> --
>>> Pierre Augier - CR CNRShttp://www.legi.grenoble-inp.fr
>>> LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels
>>> BP53, 38041 Grenoble Cedex, Francetel:+33.4.56.52.86.16
>>> _______________________________________________
>>> Mercurial mailing list
>>> Mercurial at lists.mercurial-scm.org
>>> https://lists.mercurial-scm.org/mailman/listinfo/mercurial
>> --
>> Pierre-Yves David
>>
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel at lists.mercurial-scm.org
> > https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel