Betr: RE: Trimming dead branches

[Alban Hertroys]

"Becker, Mischa J" <mischa.becker at kroger.com> wrote on 11/02/2015 
22:41:44:

> > -----Original Message-----
> > From: Mercurial [mailto:mercurial-bounces at selenic.com] On Behalf Of
> > Alban Hertroys
> > Sent: Wednesday, February 11, 2015 1:25 AM
> > To: mercurial at selenic.com
> > Subject: Trimming dead branches
> >
> > We're currently having some issues with failing commits due to file
> > locking issues and those failed commits are causing dead branches in
> > our repo. In TortoiseHg's workbench we can see several new branches
> > with commit messages equal to the final successful commit attempt.
> 
> Your commits obviously committed, otherwise you wouldn't see them in
> the workbench.  This sounds like a problem with a commit hook or 
> other process.  Is there anything actually wrong with those 
> 'failing' commits or are their contents exactly the same as the 
> 'successful' commit?

Tortoise's Visual diff reports that there are no changes between the 
successful commit and the failed commit (branch).

That actually surprises me, as so far I was under the assumption that the 
issue involved our usage of the $Revision$ tags. An assumption that would 
have explained the need for a file lock on commit. Now I'm wondering 
what's causing the locking conflict from the Mercurial side of the 
problem...

I also just noticed another detail in Hg Workbench: the commit looks like 
a branch (the line at the left side shows a split), but both branches are 
named 'default'. hg heads shows a list that any hydra would be jealous of, 
that list includes the failed commits.

>  If those commits are wrong, I suggest seeing 
> if you can't prevent the commit from happening in the first place 
> via the commit hook or rolling it back immediately after the 
> failure. (see hg help rollback)

hg rollback looks like it would hurt us (and is documented to be dangerous 
and deprecated), because "It will also restore the dirstate at the time of 
the last transaction, losing any dirstate changes since that time.".

We tend to have all kinds of uncommitted changes in our repo, partially 
because some changes are temporary (only apply to our dev environment) and 
partially because people tend to postpone their commits and then forget 
about them and partially because some people don't commit at all. There's 
only so much pressure you can apply to people before it becomes punishable 
by law... And then there's all those files that aren't under version 
control at all (for good reasons).
 
> There are three main options for removing those 'failed' commits.
> 
> 1. Use evolve to mark the dead branches as obsolete.  The commits 
> aren't physically removed from the repo but are hidden and invisible
> by default.

I've heard good things about evolve, perhaps it's time to finally look 
into it.
 
> 2. Clone the existing repo to a new repo taking care to exclude the 
> dead branches from the clone. When done, replace the existing repo 
> with the clone.

That solution is problematic with this repo for the same reasons that hg 
rollback would cause issues (see above).

> 3. Enable MQ and strip the 'failed' commits from the repo.  Do this 
> on a copy of the repo to make sure you don't accidently break 
> anything.  Once the copy is fixed, replace the original repo. (In 
> TortoiseHg, once MQ is enabled, you can right-click the bad commit 
> and select Modify History -> Strip...)

(Same issues as with option 2)

> Options 2 & 3 require anyone who has a clone of the main repo to 
> either do the same process themselves to strip out the unwanted 
> commits or to re-pull a new clone after the main repo is fixed.

Well, there really is only one single repo. I do keep a clone just in 
case, but that's not in a state where the contents of the repo are 
actually usable for reporting (which is what we do for a living).

On the "plus" side, there's only 4 of us and I'm the only frequent 
committer.

> > For the record, it appears that those file locking issues are caused 
by
> > a combination of:
> > * the 3 of us using the same repo on a remote Windows share,
> > * which is also used by the (web-based) product we develop in;
> > WebFOCUS, which appears to keep locks on certain files (namely
> > procedures) longer than necessary,
> > * in combination with us using the keywords extension, because our
> > users like to be able to see whether the production version of their
> > reports is up-to-date with the latest developments that they cleared
> > for production use.
> >
> > That's a bit of a tricky setup we realise, but we haven't found a way
> > around that actually works. Separate development environments didn't
> > work out, for example.
> 
> I don't recall if you have ever talked about this on the email list.
> If you are willing to try changing your setup, let us know what 
> problems you were having.  Maybe someone will know a fix.

The impractical setup (from the point of view of hg) is entirely due to 
the product we're working with: WebFOCUS is a web-based BI suite that 
keeps procedures (scripts), markup (HTML, CSS. javascript) and data 
warehouse files (table descriptors and data) in the same flat tree 
structure on the file system. The result is that _our_ files (the ones 
under hg's contrl) and the warehouse files are all over the place. 
Admittedly, part of that mess is historical, from a time before a VCS was 
introduced (which also explains a large part of the bad commit habits, I'm 
sure).

Especially the warehouse data-files can be large and take long to recreate 
through scripts (some of those take several hours to complete). The 
easiest way to re-create those files if we'd each have our own development 
environment and (local) hg repository is to re-run the scripts that create 
them, but that way we quadruple the nightly load on our database-servers 
and they wouldn't finish in time.
Alternatively, we could copy those files over from our central repo, but 
that's difficult to automate - especially when local development causes 
changes in the file format for such files in a local repo. It doesn't help 
that we're on a primitive OS (Windows) either; it's scripting capabilities 
are rather sub-par.

Hence we ended up working on a single central repository that's directly 
under the BI tools control. That was far easier to work with than 
attempting to keep several local environments running and in sync. 
Unfortunately, it appears that a recent upgrade of the BI tool now causes 
(longer?) file locks that break our commits.

It's not really a unique situation, so I am interested in how people solve 
this issue. Google wasn't particularly helpful the last time I searched.

Regards,

Alban.



alban.hertroys at apollovredestein.com
T: 




 	Apollo Vredestein B.V. - P.O. Box 27 - 7500 AA  Enschede - The Netherlands - 
Chamber of Commerce number 34223268 - http://www.apollovredestein.com	 

The information contained in this e-mail is intended solely for the use of the 
individual or entity to whom it is addressed and others authorized to receive 
it. You are hereby notified that any disclosure, copying, distribution or 
action in relation to the contents of this information is strictly prohibited. 
If you are not the intended recipient, please delete this message and any 
attachments and advise the sender by return e-mail. The confidentiality of this 
message is not warranted. Apollo Vredestein B.V. rules out any and every 
liability resulting from this or any other electronic transmission.