Using Rust in Mercurial

[Juan Francisco Cantero Hurtado]

On Tue, Dec 05, 2017 at 11:00:41AM -0800, Gregory Szorc wrote:
> Mercurial Packagers,
> 
> We would like to start using Rust in Mercurial. A brain dump is at
> https://www.mercurial-scm.org/wiki/OxidationPlan. The anticipated sequence
> of events is likely something like:
> 
> 1) Replace exewrapper.c/hg.exe with a Rust binary on Windows
> 2) Rewrite existing C Python extensions in Rust
> 3) Make `hg` a Rust binary (instead of a Python script) on all platforms.
> This requires the Rust binary to link against libpython and embed a Python
> interpreter.
> 
> There is no timeline for these milestones. But I'd like to do #1 in the 4.5
> release if possible and #2 and #3 as soon as possible after that.
> 
> The Mercurial Project itself is interested in shipping a
> self-contained/standalone Mercurial distribution. This would include its
> own bundled version of Python, including the standard library.
> 
> When we start requiring Rust to use Mercurial, I anticipate that downstream
> packagers will want to package an "unbundled" Mercurial - one that links
> against the system libpython2.7 and uses the system Python's standard
> library. This is totally fine and shouldn't be difficult to support.
> 
> My question to packagers is: what is your current level of support for a)
> Rust b) embedding CPython? e.g. what is the minimum version of Rust we can
> target? Do you support a shared libpython2.7.so or only a static
> libpython2.7.a?

Hi, I'm the OpenBSD maintainer.

a) We have rust support on amd64. The support on i386 has some problems
with big programs like Firefox. We don't have support for rustup.

OpenBSD has a 6 months cycle for the -stable release and a kind of rolling
release -current. I try to keep the package updated on -stable when
there are security bugs but I don't update to bugfix releases. -current
is always updated.

So, the rust features used by mercurial should have at least 12 months.
6 months + the time needed by our maintainer to update rust + the
mercurial cycle.

Security updates are a pain with mercurial because you only provide a
patched new version, even without pointing to the commits with the
patches. Patching python code is way easier than rust. Maybe the project
could have someone in charge of create the backports for the security
bugs while the bugs are under embargo instead of the actual situation.
The migration to rust will not help here.

Also, changing to rust, without a C or python fallback, we are going to
lose the package on a lot of platforms. The same for NetBSD, FreeBSD,
Debian or Gentoo. Here is our list of platforms with packages:
https://fastly.cdn.openbsd.org/pub/OpenBSD/snapshots/packages/

b) We have libpython2.7.so.0.0 (the major and minor numbers could change
in the future). We don't have the static lib, but I could add the lib if
you need it.

> 
> I'll likely capture replies on the aforementioned OxidationPlan wiki page.
> If you want to just add relevant info there (anywhere is fine - I'll see
> the wiki updates and refactor if needed), feel free to do that.

Eventualy, you are going to lose every current extension because the
migration will break more and more APIs. So, why not create a DSL
written in rust for small extensions and a rust library for the big
ones?. Bundling python looks like a pain which will give you a lot of
problems.


> I want to emphasize that nothing is yet set in stone. But adding Rust to
> Mercurial is something we would very much like to do and we would like to
> have as many packagers on board as possible.

I'm not against the Rust migration but just be aware that this kind of
change are not always easy from a maintainer perspective. We are not
windows, we don't have isolated programs with their own libraries.
Everything is coordinated in a big tree of packages.

As user, my biggest problem with mercurial is the speed with big repos
(100000-200000 commits), so I'm happy with this change (also with your
zstd work).


-- 
Juan Francisco Cantero Hurtado http://juanfra.info