Things we ought to do to improve our packaging

Steve Borho steve at borho.org
Thu Aug 8 06:13:49 UTC 2013


On Thu, Aug 8, 2013 at 12:43 AM, Brendan Cully <brendan at kublai.com> wrote:

> On Wednesday, 07 August 2013 at 23:37, Steve Borho wrote:
> > On Tue, Aug 6, 2013 at 5:53 PM, Matt Mackall <mpm at selenic.com> wrote:
> >
> > > On Tue, 2013-08-06 at 15:04 -0500, Steve Borho wrote:
> > > >
> > > >
> > > >
> > > > On Tue, Aug 6, 2013 at 2:47 PM, Matt Mackall <mpm at selenic.com>
> wrote:
> > > >         On Tue, 2013-08-06 at 14:15 -0500, Steve Borho wrote:
> > > >         >
> > > >         >
> > > >         >
> > > >         > On Tue, Aug 6, 2013 at 2:01 PM, Augie Fackler
> > > >         <raf at durin42.com> wrote:
> > > >         >         On Tue, Aug 06, 2013 at 01:53:34PM -0500, Matt
> > > >         Mackall wrote:
> > > >         >         > [this discussion needs to be cross-posted]
> > > >         >         >
> > > >         >         > We've got a few long-standing problems with our
> > > >         current
> > > >         >         packaging:
> > > >         >         >
> > > >         >         > - lag official release
> > > >         >         > - not automated
> > > >         >         > - not reproduceable by third parties
> > > >         >         > - not signed
> > > >         >         > - lack nightly builds
> > > >         >         > - notable missing platforms (RHEL)
> > > >         >         > - not uniform in configuration
> > > >         >         >
> > > >         >         > Our attitude to date has been "well, we're not
> > > >         really in
> > > >         >         charge of that,
> > > >         >         > that's done unofficially by volunteers". I think
> > > >         we've
> > > >         >         reached a point
> > > >         >         > where this needs to start changing.
> > > >         >         >
> > > >         >         > In an ideal world, a release would work like
> this:
> > > >         >         >
> > > >         >         > - I tag the release
> > > >         >         > - I push it
> > > >         >         > - Automated builders kick off on a bunch of VMs
> or
> > > >         hosts
> > > >         >         > - Builders run stock makefile targets or scripts
> > > >         in tarball
> > > >         >         > - Packages get signed
> > > >         >         > - Packages get uploaded to one or more locations,
> > > >         including
> > > >         >         > mercurial.selenic.com
> > > >         >         > - Download links get updated
> > > >         >         >
> > > >         >         > Importantly, this same process would also be used
> > > >         to produce
> > > >         >         automated
> > > >         >         > nightly builds for testing and bug reporting and
> > > >         discovering
> > > >         >         packaging
> > > >         >         > problems before release dates.
> > > >         >         >
> > > >         >         > The job of packagers would thus shift from
> > > >         "(manually?)
> > > >         >         building and
> > > >         >         > uploading binaries" to "making sure the committed
> > > >         build
> > > >         >         rules are
> > > >         >         > correct and up-to-date".
> > > >         >         >
> > > >         >         > I think this also needs to be inclusive of builds
> > > >         for
> > > >         >         packaging systems
> > > >         >         > (Debian, Ubuntu, RHEL, Centos, Solaris, Pypi,
> > > >         etc.). These
> > > >         >         actually
> > > >         >         > suffer from more or less the same problems even
> > > >         though the
> > > >         >         distribution
> > > >         >         > model is different. So we should be building
> > > >         nightlies and
> > > >         >         release
> > > >         >         > builds for these systems and be more directly
> > > >         responsible
> > > >         >         for quality
> > > >         >         > and timeliness here as well.
> > > >         >
> > > >         >
> > > >         >         This honestly sounds like a great improvement,
> given
> > > >         how stale
> > > >         >         the
> > > >         >         packages for Ubuntu tend to be, and how some
> > > >         packagers took it
> > > >         >         upon
> > > >         >         themselves in the past to enable all the
> extensions.
> > > >         >
> > > >         >
> > > >         >
> > > >         >
> > > >         > On the Windows side, everything is automated except for two
> > > >         steps:
> > > >         >
> > > >         > 1 - signing packages requires a passphrase to be entered
> > > >         > 2 - uploading is currently manually done through BB's
> clunky
> > > >         web
> > > >         > interface
> > > >         >
> > > >         >
> > > >         > If we had an SCP or FTP URL to upload to, and a sane policy
> > > >         for
> > > >         > deleting old nightlies after a short while, it could be
> > > >         automated.
> > > >
> > > >
> > > >         Well, you already have SCP access, no?
> > > >
> > > >
> > > > Yep, it's just a question of whether you want nightlies in the same
> > > > location as releases, or in a different folder that was size-managed.
> > > >
> > > >         In addition to having a fully automated process that builds
> > > >         releases and
> > > >         nightlies, I think its also important to get to the build
> > > >         script into
> > > >         the tree so that people who need to build their own copies
> for
> > > >         various
> > > >         reasons (need bugfix, need custom deployment, St. Louis
> > > >         trampled by
> > > >         kaiju) can do so.
> > > >
> > > >
> > > > So.. building windows frozen packages is fairly convoluted
> (especially
> > > > for thg but the simple hg MSI is no cakewalk) and has a boatload of
> > > > prereqs to be installed.
> > >
> > > Yep, I'm well aware. And this is actually MORE reason to insist on full
> > > automation. Let me back up and explain some of my motivation here:
> > >
> > > While I think you do a great job with packaging, it's lately become
> > > clear that package maintainers are a significant single point of
> > > failure. And we've also not coincidentally discovered that we don't
> > > actually know how some of our packages are actually built (like OS X)
> > > because the instructions don't work any more. So when the packager
> > > disappears, we've got several problems: find someone with the hardware,
> > > install the OS(es), install the deps, reverse-engineer the build
> > > process, hit unknown number of gotchas..
> > >
> > > The OS X situation is not as complex as Windows, but it's still bad
> > > enough that the current approach is clearly not good enough.
> > >
> > > And that's not the only problem in this area. For instance, we also
> have
> > > real uses for continuous builds that are not being met by monthly
> > > releases. I really want the BTS robot to point people at builds and say
> > > "try this and report back, please", but can't.
> > >
> > > (And it's also a perennial confusion that we announce releases before
> > > there are any packages built.)
> > >
> > > Fortunately, we know how to fix these sorts of things:
> > >
> > > - manifest the instructions and dependencies as build scripts
> > > - version control the build scripts in the canonical repo
> > > - use the build scripts, continuously and non-interactively
> > > - get people to use those builds and discover problems before releases
> > > - fix the problems found via check-in
> > >
> > >
> > > With my 'hackable' Windows builds, I've gone as far as scripting the
> > > download and install of the right versions of Python and gcc and all
> the
> > > other deps if not present to make sure its reproduceable, so I know
> this
> > > sort of thing can be done, even on Windows. I've obviously not yet gone
> > > as far as putting this knowledge in the repo, but I ought to.
> > >
> > >
> > > All that said, let's focus on phase 1: publishing nightly builds,
> > > somehow, somewhere.
> >
> >
> > Angel has offered to build nightlies before (using an automated framework
> > like Jenkins), he just needed a place to upload them. We just need to
> > circle around and get that closed.  The auto-builds might be unsigned at
> > first.
> >
> > As far as getting this into the tree and scripted.. I agree this would be
> > astonishingly useful for the long run (it mostly frees me up as well) but
> > my hg hacking time is extremely limited these days and I don't know when
> > I'll get a chance to work on it.
> >
> > One suggestion I have, if we try to automate this process in the
> hg/contrib
> > folder, is that we make an attempt to drop the requirement on
> Microsoft's C
> > compilers and try to use recent MinGW GCCs to compile the Explorer
> > extension and the Mercurial (and other hg extension) Python extensions.
>  We
> > had problems with GCC some 5-6 years ago but perhaps those are all
> resolved
> > now (that said, the Explorer extension hasn't changed in 19 months, we
> > could probably check in pre-compiled versions in its repo).
>
> We do have an existing buildbot. I'm not sure what advantage there
> would be in a separate Jenkins to do the builds? The buildbot even has
> a windows slave.
>

The only advantage is that Angel already has the prereqs in place to build
packages.

-- 
Steve Borho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial-packaging/attachments/20130808/98f7b72c/attachment-0002.html>


More information about the Mercurial-packaging mailing list