Automated Staged Repos with Continuous Integration

Scott Palmer swpalmer at gmail.com
Sat Dec 10 20:57:33 UTC 2011


Hello Mercurial Users,

I'm trying to develop a workflow in my company to minimize the distribution of broken code.  We use Hudson/Jenkins continuous integration servers and are in the process of migrating from Subversion to Mercurial.  We have some large projects where our development team occasionally has a stretch of time where our builds aren't passing tests and I would like to do something about it.  My plan is to set things up so that only when the continuous integration server validates a build compiles and passes all the unit and integrations tests,does it push the changes on to the "central" server, where the developers pull the changes from.

The idea is that any developer can push to the "integration" repository, but they would always pull from the "validated" repository, so the sharing of broken bits is minimized.  It seems like it should be fairly easy to set up, but I'm wondering about how to set up the "integration" repository.   The issue is that multiple developers will be pushing to it and since the developers will be basing there code on the good code from the "validated" repo I see a situation with multiple heads arising as each developer tries to get their changes to the integration server.  I'm not sure how clear that is.. maybe I can try a diagram:

                      Alice's 
           --Pull-->  Repo    --Push-->                Good Build ?
Validated                                Integration   No -> What do I do?
Repo                                     Repo
  ^        --Pull-->  Bob's   --Push-->                Yes
  |                   Repo                              |
   \                                                   /
    -------------<---------<---------<-------Push<----


Let's say the validated repo has two changesets in it.


Validated 
Repo
1-2

Alice and Bob both pull from the validated repo and begin work on different features or bugs.

Alice finishes first and pushes her changes to the integration server (changesets 3a and 4a)

Integration gets:
1-2-3a-4a

If the build works, those changes are pushed to the validated repo.

Validated Repo gets:
1-2-3a-4a

Meanwhile Bob has added three changes set of his own and want to push:
1-2-3b-4b-5b

When Bob tries to push changes to the integration server he will get a notice about creating multiple heads:

1-2-3a-4a
   \             X  not allowed!
    3b-4b-5b

Bob knows then that he doesn't have the latest code.  If the code has made it to the validated repo he can pull and merge and move on.

1-2-3a-4a----6b-
   \        /  Bob merged to create 6b
    3b-4b-5b

The problem comes when another developer (say Bob) wants to push changes to the integration repo, but the changes that are already there haven't made it to the validated repo yet.(This could happen for many reasons, the integration build could fail, or is still in progress, or multiple developers want to push before the integration builds even start, etc.)

We don't want Bob to pull the potentially broken changes from the integration repo and inherit someone else's problems just so he can share his changes.  Even if he did, the integration build could still fail unless he fixed Alice's bugs, but that's not something he should have to do to share his work.

I think there must be a solution to this problem that would allow us to use an automated continuous integration server as a gateway for sharing changes.  It is too much to have an integration repo for every developer and even if I did the integration server would have to deal with merge issues when it tried to push on the changes from good builds to the validated repo.

Am I just dreaming or is it possible to make something like this work?


Thanks,

Scott




More information about the Mercurial mailing list