[External] remotefilelog: Large Manifest

[Pulkit Goyal]

On Tue, Mar 3, 2020 at 9:25 PM Son Luong Ngoc <son.luong at booking.com> wrote:
>
> Hi Augie,
>
> On Mar 3, 2020, at 15:51, Augie Fackler <raf at durin42.com> wrote:
> I haven't looked into FB's treemanifest extension, but note there are better-supported tree-manifests in core (confusingly named the same thing, not our fault sorry). Since it sounds like you're exploring converting to hg anyway, I'd encourage making a large re
>
> pository into treemanifest format so narrow can work more effectively.
>
> As a matter of fact I did.
> The conversion using hg-git to treemanifest was slower than normal manifest (i think 1.5 times?)
> Narrow worked, just like the test-narrow-clone setup I copied from.
> But the speed is still not ideal as you still have to iterate through O(n) manifests and changesets on the serverside.

For widening a narrow clone, you don't need to walk the changesets
unless you are using ellipses.
>
> The idea of having these cached via memcached is very appealing to a big repo which is why I am leaning more toward remotefilelog + sparse over narrow currently.

Yep, I also faced this issue around a year and half ago and hacked
some caching thing. IIRC things computed in
https://www.mercurial-scm.org/repo/hg-committed/file/tip/mercurial/changegroup.py#l1155
can be cached which will make widening significantly faster.

>
> I am waiting for the minor release version that fix narrow + remotefilelog compatibility so that I can try both together.
>
> Cheers,
> Son Luong.
>
>
> _______________________________________________
> Mercurial mailing list
> Mercurial at mercurial-scm.org
> https://urldefense.com/v3/__https://www.mercurial-scm.org/mailman/listinfo/mercurial__;!!FzMMvhmfRQ!_63qGqMq_YkgVHr1Th03t1LNiE1M8Eq7Mn_6kLfUH-3dImHzwoDcaziw2RXcVEl1$
>
>