Sync subdirectory to public repo

[Shawn Hoover]

I updated the script to merge the shamaps in both directions instead of
keep one master shamap and always blow away the other one and rebuild it.

It seems to stay in sync. I welcome further input as to whether it is an
abomination to even try.

Here is a script: https://gist.github.com/shoover/ad1601d342208a75f703.
Pardon my use of Ruby. Perhaps writing a small converter_source class in
Python would be cleaner.



On Fri, Sep 11, 2015 at 1:29 PM, Shawn Hoover <shawn.hoover at gmail.com>
wrote:

>
> On Thu, Sep 10, 2015 at 11:36 PM, Gregory Szorc <gregory.szorc at gmail.com>
> wrote:
>
>> The "convert" extension can be used to convert one Mercurial repository
>> to another. You can configure it to only convert a sub-directory.
>>
>
> `hg convert` is excellent and is what I used to create the repo with
> subdirectories. I ran it incrementally and never had a problem. Filemaps
> worked flawlessly.
>
>
>> IIRC `hg convert` is one way only.
>>
>
> Yes, it is one way. But... what if a reverse shamap was seeded using the
> one way shamap with the commit IDs reversed from another shamap? Then
> convert could work either way as long as the right shamap and filemap was
> used for each direction.
>
> Ok, I tried it. Given repos main and shared and shamaps to-main and
> to-shared, I am treating to-main as the master conversion record. As long
> as I always rebuild shamap to-shared from to-main before converting from
> main to shared, it seems to work.
>
> Changesets in main that result in no change in shared due to filemap
> exclusions (all other subdirectories) are always retried in this naive
> scheme because they get blown away from to-shared every time the script is
> run. I suppose it could be made more intelligent by merging the reverse map
> of to-main into to-shared instead of blowing it away.
>
>
>> Bi-directional sync is inherently hard because there are race conditions
>> between pushing to both repos. A proper solution requires locking both
>> repos when a push comes in. Things are much easier if you limit yourself to
>> writes from one repository and then "transplant" commits from the writable
>> repo to the mirror.
>>
>
> Don't the incremental update capability of hg convert and the shamaps
> tracking scheme mentioned above solve this? New changesets from the server
> have to be pulled, merged, and pushed just like with normal development,
> except you have to slip another conversion run in there.
>
> Someone from Facebook should chime in here: it is my understanding that
>> many of their GitHub projects are essentially sub-directories of their
>> internal monorepo. However, that's hg -> git and they may be using hg-git,
>> which is somewhat more powerful than "hg convert." I /think/ they employ a
>> non-traditional GitHub workflow and rebase GitHub pull requests on their
>> internal repo then mirror the linear history back out to GitHub. Again, I'm
>> not 100% on this.
>>
>
> Facebook and Google syncing to GitHub from internal monorepos is what I
> had in mind with this and I'd love to hear from them :)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20150911/ecf1a8ab/attachment-0002.html>