push deadlocks (was: pull hook?)

[Lasse Kliemann]

Matt Mackall <mpm at selenic.com> writes:

> On Sat, 2015-05-02 at 13:37 +0200, Lasse Kliemann wrote:
>> What can I do to have an action executed upon a pull before outgoing
>> changesets are fixed?
>
> Generally speaking, nothing. Consider Alice pulls from Bob's repo on
> NFS. Alice doesn't run Bob's hooks.. because that's called "an arbitrary
> code execution security hole". And Bob doesn't run Bob's hooks either..
> because there's no mechanism for that.
>
> But in configurations where hooks ARE honored, you can use the
> preoutgoing hook.

Problem with that is that it is only triggered when the repos itself has
outgoing changesets. If, say, the user pulls from B and B does not have
any new changesets for the user, no pull from A is made. So anything new
in A is ignored for the time being.

> If you're looking to make a Mercurial server cluster, you might want to
> to look at this:
>
> https://bitbucket.org/facebook/hgsql
>
> It uses an SQL server to implement cross-server transactional locking
> and replication (though storage is still done in the usual way).

It looks like I will have to go through this some day.

Right now, I tried to get it working my way, because I couldn't see the
reason why it should not work. Now I think one problem is:

User X pushes to A and the hook in A pushes to B.

At the same time, user Y pushes to B and the hook in B pushes to A.

I presume that only one push per repos per time is allowed, so A->B and
Y->B have to coordinate. Assume that A->B waits for Y->B (but X->A is
already in progess). But Y->B will not conclude before B->A. (*) This
one however waits for X->A, which cannot conclude before A->B has. So we
have a deadlock.

A solution should be to remove the logic in the sentence before (*), so
we have to somehow decouple the push from the hook. I guessed that it
will be enough for this to let the push in the hook run in the
background. So my hg-hook-push script now looks like:

case "$2" in
    '') : ;;
    "$HG_URL") exit 0 ;;
    *) : ;;
esac
(hg push -f "${1?}" > /dev/null 2> /dev/null < /dev/null &)
exit 0

This probably helps in some cases, but could not completely eliminate
the timeouts. Is it possible that two repos pushing into each other can
also deadlock? To exclude this from happening, I invented an
intermediate repos on each server, say A' and B'. The concept is now:

X -> A -> A' -> B

Y -> B -> B' -> A

Together with the background modification above, this drastically
reduced the frequency of the timeouts. But they still happen. *sigh*
At least under relatively extreme conditions, which maybe won't occur in
practice so soon. Still.

Maybe I'm working at the completely wrong end and all my efforts just
coincidently make some race condition that I haven't thought of less
likely to occur, but not impossible.

I know that going for hgsql is probably the best solution. But why
doesn't it work the basic way? This really intrigues me. Should it work
in principle, as far as Mercurial is concerned? Then something's wrong
with my Apache/SSH setup maybe.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20150503/39175ec8/attachment.asc>