Producing Mercurial wheels from Github Actions + modernize build process

[Georges Racinet]

Hi Pierre !

On 10/4/24 2:17 PM, PIERRE AUGIER wrote:
> - @gracinet do you think heptapod/heptapod#355 could be implemented quite soon so that we can get a good mirror ofhttps://foss.heptapod.net/mercurial/mercurial-devel on Github?
I may have a better idea, but let me answer the direct question first:

    On one hand, I do not have time for much these days. On the other
    hand, nice wheels would certainly save me (if with Rust) and my
    coworkers quite a bunch of time, so I can consider it (probably not
    for the upcoming 17.5). Notably in maintenance of CI images, such
    wheels would make much easier to change version from the one
    provided in distro packages. I would probably go the virtualenv way
    everywhere and be happy with it.

    I wouldn't make a UI though, as it is too time consuming and not
    necessary to accommodate Mercurial. I suppose that the maintainers
    would not have any problem setting it up with the API :-)

A simpler solution, perhaps more robust and less time-consuming / risky 
for me :

    Why not create a Git project on GitHub dedicated to building
    Mercurial only (not a mirror). Let's call it `hg-bin-build` for the
    time being.

    ts CI/CD actions could then just grab the relevant tarball from
    foss.heptapod.net (or pull from mercurial-scm.org) and perform the
    binary builds. I don't mean moving the actual build recipes in
    there, just to execute them. It would thus have only the GitHub
    action definitions a `VERSION` file.

    There are many advantages: no need to rely on hg-git etc to do that
    (and I do not have to dodge heptapod#1764). One could even imagine a
    CI job on foss.heptapod.net that triggers it in tag pipelines by
    committing and pushing (with Git).

    There can be variations on the same idea. Here's a more self-hosted
    variant: have the `hg-bin-build` Git repo on foss.h.n (somewhat
    easier to use the proper tokens to push on it from a
    `mercurial-devel` CI job) and activate Git push mirroring to GitHub.

    If noone has objections with such a plan, I would prefer it.
    Switching back to using Mercurial-to-Git push mirroring
    (heptapod#355) would still be possible in the future if people find
    it more satisfying.

    In terms of PR, one could say that Mercurial is depending on Git,
    and that is to be avoided. My take is that this use case is well
    bounded and actually should be considered on the positive side as a
    demo that using Mercurial in a Git-centric world is perfectly possible.

> It's not possible to have different flavors for wheels on PyPI.
>
> The right way (from the point of view of the PyPA) to distribute with PyPI "Mercurial with C extensions" and "Mercurial with Rust extensions" is to have a package mercurial (with C extensions) and another package hg-rust, which could be an optional dependency of mercurial so that things like
>
> pipx install mercurial[rust]
>
> works. One would need in mercurial's pyproject.toml
>
> [project.optional-dependencies]
> rust = ["hg-rust"]
This would be fine by me, and indeed a huge progress. A practical 
example: in the Heptapod Development Kit, managing of dependencies is a 
real pain (the whole setup is been done by a collection of Makefiles, 
not my choice) and it happens that packages depending on Mercurial get 
installed hence also Mercurial (without Rust) before the target that is 
supposed to install Mercurial (with Rust), and there is no way to 
replace that with the proper version other than uninstalling the wrong 
one. Of course I tend not to notice immediately, only when some tests 
start failing because the repos have the nodemap. Worse, there is 
sometimes a cached wheel (without Rust). I would be really happy to just 
depend on `hg-rust` or whatever the end name happens to be.

Thank you for your efforts on this subject, it seems we are not so far 
from a solution.

Side note: well yes I would be happy if we could have Windows and MacOS 
runners on foss.heptapod.net. Our past experience with the former is 
that they are awfully time-consuming. Last time I checked, Heptapod 
Runner worked on Darwin, but I have no reasonable place to host it.

Best,

-- 
Georges Racinet
https://orbeet.io/a-propos/cloudcrane,https://heptapod.net
GPG: 09719FFE0B476DC26923F8EEB8EB20361976F291

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20241004/7bdde891/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xB8EB20361976F291.asc
Type: application/pgp-keys
Size: 677 bytes
Desc: OpenPGP public key
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20241004/7bdde891/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20241004/7bdde891/attachment-0001.asc>