[External] remotefilelog: Large Manifest

[Pulkit Goyal]

On Tue, Mar 3, 2020 at 10:17 PM Son Luong Ngoc <son.luong at booking.com> wrote:
>
> Just to be a bit more clear
>
> On Mar 3, 2020, at 16:55, Son Luong Ngoc <son.luong at booking.com> wrote:
>
>
> But the speed is still not ideal as you still have to iterate through O(n) manifests and changesets on the serverside.
>
> What I mean here is that I found cloning and widening with Narrow currently a bit slow.
> hg clone --narrow my-repo-treemanifest --depth 10 --include README.md  382.37s user 48.91s system 95% cpu 7:33.36 total

Ah, I see. Since you have merges, you should try narrow without
ellipses first i.e. not use the --depth flag. Can you try:
`hg clone --narrow my-repo-treemanifest --include README.md --stream`
and see how long does that take? It should be faster if you have quite
good internet connection because it's a kind of rsync.

About merges and ellipses, there is an algorithm in narrow which is
not yet optimized when there are merges. That needs to be
improved to make `--depth` usable with merges.

>
> The user experience with narrow spec-file format is also not very friendly as you are restricted to full path and no glob support.
IIRC, you can definitely do that. The narrow spec-file format supports
mercurial file patterns mentioned in `hg help patterns`.

> That make things like trying to "clone all the files in the root directory, exclude all the dirs" a bit clunky... unless I am using it wrongly.
> (reference MSFT blog https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/)
>
> Cheers,
> Son Luong.
>
>
> _______________________________________________
> Mercurial mailing list
> Mercurial at mercurial-scm.org
> https://urldefense.com/v3/__https://www.mercurial-scm.org/mailman/listinfo/mercurial__;!!FzMMvhmfRQ!_63qGqMq_YkgVHr1Th03t1LNiE1M8Eq7Mn_6kLfUH-3dImHzwoDcaziw2RXcVEl1$
>
>
>