Sync subdirectory to public repo

[Shawn Hoover]

On Thu, Sep 10, 2015 at 11:36 PM, Gregory Szorc <gregory.szorc at gmail.com>
wrote:

> The "convert" extension can be used to convert one Mercurial repository to
> another. You can configure it to only convert a sub-directory.
>

`hg convert` is excellent and is what I used to create the repo with
subdirectories. I ran it incrementally and never had a problem. Filemaps
worked flawlessly.


> IIRC `hg convert` is one way only.
>

Yes, it is one way. But... what if a reverse shamap was seeded using the
one way shamap with the commit IDs reversed from another shamap? Then
convert could work either way as long as the right shamap and filemap was
used for each direction.

Ok, I tried it. Given repos main and shared and shamaps to-main and
to-shared, I am treating to-main as the master conversion record. As long
as I always rebuild shamap to-shared from to-main before converting from
main to shared, it seems to work.

Changesets in main that result in no change in shared due to filemap
exclusions (all other subdirectories) are always retried in this naive
scheme because they get blown away from to-shared every time the script is
run. I suppose it could be made more intelligent by merging the reverse map
of to-main into to-shared instead of blowing it away.


> Bi-directional sync is inherently hard because there are race conditions
> between pushing to both repos. A proper solution requires locking both
> repos when a push comes in. Things are much easier if you limit yourself to
> writes from one repository and then "transplant" commits from the writable
> repo to the mirror.
>

Don't the incremental update capability of hg convert and the shamaps
tracking scheme mentioned above solve this? New changesets from the server
have to be pulled, merged, and pushed just like with normal development,
except you have to slip another conversion run in there.

Someone from Facebook should chime in here: it is my understanding that
> many of their GitHub projects are essentially sub-directories of their
> internal monorepo. However, that's hg -> git and they may be using hg-git,
> which is somewhat more powerful than "hg convert." I /think/ they employ a
> non-traditional GitHub workflow and rebase GitHub pull requests on their
> internal repo then mirror the linear history back out to GitHub. Again, I'm
> not 100% on this.
>

Facebook and Google syncing to GitHub from internal monorepos is what I had
in mind with this and I'd love to hear from them :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20150911/0f0be815/attachment-0002.html>