live-mirroring of repository

[Lasse Kliemann]

There are repos A and B on different servers. When changesets come into
A, they should immediately be pushed to B. And vice versa: if changesets
come into B, they should immediately be pushed into A. This way a user
can use A or B, whatever has the better connectivity or whatever, and
will always see the latest state.

My solution so far was as follows. In A, there is a hook configured:

changegroup = hg-hook-push URL-OF-B

And in B, we have:

changegroup = hg-hook-push URL-OF-A

The script hg-hook-push is this:

hg push -f "${1?}"
code="$?"
case "${code?}" in
  0|1) exit 0 ;;
  *) exit "${code?}" ;;
esac

What happens if a changeset comes into A? Then the hook is run, pushing
to B. This will trigger the hook in B, which pushes to A. This does
*not* go on forever, since at some point there will be nothing to push,
so there are no incoming changesets in the other repo, thus the hook
will not be executed.

This setup worked for some time. Until recently, users started
complaining about the pushes sometimes hanging for minutes. I could now
reproduce it, at least in the following way. When I push to A and B
approximately simultaneously, then the push hangs saying "searching for
changes", indeed for minutes. Looking at what happens on the servers, it
appears that the hook is already running. On at least one of the two,
say A, I see hg, python, and hg-hook-push processes in ps for
minutes. Sometimes I get "HTTP Error 504: Gateway Timeout" after some
time in the client.

In case it matters, repos A is accessed via SSH and repos B via HTTPS.

If I disable the hook on A _or_ on B, it works fine. But both hooks
enabled cause trouble. I have also tried with a "sleep 1" up to "sleep
10" before the push in hg-hook-push, but this did not help.

On repos A, I modified the hg-hook-push to that it runs:

timeout 10s hg push -f "${1?}"

Then this timeout is triggered sometimes and the client push does not
hang. However, this is no solution since I want the push to happen, also
if it may take more than 10 seconds (which might be the case for larger
changesets sometimes).

Any help is appreciated. Also, of course, if you have a better idea how
to do such a live-mirroring, I will be happy to go with that.

Thanks a lot!

-- 
Kind Regards / MfG
Dr. Lasse Kliemann
Westring 269, 24116 Kiel, Germany
E-Mail: lasse at lassekliemann.de
OpenPGP Key: 0x4296A3E08903D018
Phone: +49 162 66 88 468
Web: http://lassekliemann.de
 
Work Address:
Department of Computer Science
Kiel University
Christian-Albrechts-Platz 4
24118 Kiel, Germany
E-Mail: lki at informatik.uni-kiel.de
Phone: +49 431 880 7454
Web: https://www.informatik.uni-kiel.de/~lki
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial/attachments/20150502/95f39567/attachment.pgp>