Build system thoughts

[Bill Barry]

Greg Ward wrote:
> On 08 December 2008, Bill Barry said:
>   
>> I was thinking about CI and was wondering what everyone here thought 
>> about the following working practice designed for small to medium teams 
>> (probably around 8 to 30 people):
>>
>> 1 outgoing repo (call it stable)
>> 1 "centralized" repo (call it central)
>> developers maintain their own copies of the central repo, all have push 
>> access
>>     
>
> Sounds good so far.  (I.e. I have been thinking along the same lines for
> my group, currently 25 developers working on 3-10 projects [depends how
> you count the projects]).
>
> [...]
>
>   
>> there would be a stable repository maintainer role who would:
>> hg pull
>> hg up -C default
>> hg merge -r "featurename"
>> (possibly dealing with conflicts)
>> hg ci -m "merging in featurename message"
>> hg push
>> hg push -r default stable
>>     
>
> This is where you lose me.  Our company has a culture that
> developers do their own merges, and as the guy who would be stuck
> merging everyone else's features/fixes if we changed that, I am strongly
> in favour of it.  ;-)
>   
Developers would still do their own merges, they just wouldn't merge in 
to the default branch (they could merge out from it to their feature 
branch). The maintainer could refuse to merge if there were conflicts he 
doesn't want to deal with and have the developer who is working on the 
feature merge into the feature branch and then the maintainer can merge 
into default:
maintainer:
hg pull
hg up -C default
hg merge -r "featurename"
(sees conflicts that he doesn't want to bother with, says to developer 
to take care of it)
developer:
hg pull
hg up -C "featurename"
hg merge -r default
(deals with conflicts)
hg ci -m "merged feature to bring in updates from default branch"
hg push
(developer says merge is good and CI passes)
maintainer:
hg pull
hg up -C default
hg merge -r "featurename"
(no more conflict)
hg ci -m "incorporated feature into trunk"
hg push

of course the developer can and is encouraged to merge from the default 
branch often
> (Plus, who better to resolve a conflict than the person who wrote the
> code that's conflicting?)
>
> The other thing that concerns me a bit about your proposed workflow is
> the lateness of integration.  Currently, we do most new feature work
> right on the trunk of our CVS repository.  
This would compare to managing a patch queue and only accepting patches 
when complete. Both integrate rather late. The idea is that it allows 
the decision of feature inclusion to be deferred to as late as possible. 
This allows the main branch to remain stable throughout the iteration 
and that the branch is always in shippable form.  One of the problems 
with always just developing in trunk is that over the course of the 
iteration the quality of the build varies and devs wind up scrambling to 
increase the stability at the end of each iteration ultimately lessening 
their productivity.
> Since we're using (ugh) CVS,
> we only create development branches for features that would destabilize
> the tree for many days.
This is what we are doing as well currently (using hg); we are 
considering this flow as an alternative.
>   The advantage of everyone working on the trunk
> is that we find bad commits pretty soon: the guy next to you updates and
> can no longer compile.  Oops.  Better fix it *now*.  (Of course that is
> sometimes a disadvantage too.)  And we only need one nightly build of
> the trunk, not of every active dev branch.
>   
We would still find bad commits pretty soon (the CI environment will say 
if the branch fails to build). We might not see bad integrations right 
away, but I think in the overwhelming majority of cases I don't think 
this is a problem because it should cause conflicts when merging and 
then the maintainer would push it back to the developer.
> I can see the point of CI builds of active dev branches, but I don't
> like 1) having one gatekeeper responsible for merging other people's
> features 
There isn't any reason why we would only have one gatekeeper; there 
could just as easily be a group and it can rotate through the team. I 
was thinking that for each iteration each developer would have a 
designated pairing partner and maintainer from another pairing team. 
This puts two eyes directly on the code when pairing and keeps another 
in the know about what is going on. With 6 people we would have 
something like:
[1,2] maintained by 3 (3 is in charge of merging changes from 1 and 2 
into the trunk)
[3,4] maintained by 5
[5,6] maintained by 1
randomly assigning pairs and maintainers at the kickoff.
> and 2) the fact that that merge happens late, when it's getting
> close to release time.
Merges don't have to happen late, they can happen any time that the 
developer and CI env say it is ok and the maintainer feels like it. When 
it gets close to release time we can even get more strict, pushing off 
any remotely questionable changes until after the release and then 
deciding to merge them into either the released version or trunk. It 
would work just like dealing with patch queues, except that they are 
actual revisions.
>   IMHO features done on a dev branch should be
> merged to the trunk as soon as the responsible developer(s) says they
> are done.  Then we can all eat each other's dogfood well before branch
> day.
>   
Most of the time, this is what would happen; the difference simply is 
that we can choose not to dogfood until we are ready (by merging from 
the default branch into our feature branch).
>   
>> So what are your thoughts? And is there any tools either already in 
>> existence or under current development which could fulfill this CI 
>> process role (everything else appears doable right now)?
>>     
>
> You should definitely take a look at Buildbot.  It's a great CI
> framework.  Takes a bit of work to wrap your head around it, but once
> you do you can do some neat stuff.
>   
I'll look into it, currently we use CruiseControl.NET, but that is 
designed to just do the latest build; it doesn't have the ability to 
queue builds up and it doesn't have any way to manage multiple branches 
of a source tree (other than manually setting them up as new projects).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20081212/0955af3e/attachment-0001.html>