[External] remotefilelog: Large Manifest

[Son Luong Ngoc]

Hey Pulkit,

> On Mar 4, 2020, at 10:15, Pulkit Goyal <7895pulkit at gmail.com> wrote:
> Ah, I see. Since you have merges, you should try narrow without
> ellipses first i.e. not use the --depth flag. Can you try:
> `hg clone --narrow my-repo-treemanifest --include README.md  --stream`
> and see how long does that take? It should be faster if you have quite
> good internet connection because it's a kind of rsync.
Oh damn adding --stream definitely helped a ton, Clone speed down from 10min to 2 minutes

But widening is still take ~20 mins

> time hg tracked --import-rules some-simple-narrow.spec
searching for changes
adding changesets
adding manifests
adding file changes
files [         <=>                                                                                 ] 9078 -34s
files [                                                                <=>                        ] 81549 -616s

And for some reason the estimation count in the UI is negative :D

> About merges and ellipses, there is an algorithm in narrow which is
> not yet optimized when there are merges. That needs to be
> improved to make `--depth` usable with merges.
Thanks a lot for confirming this.

> Yep, I also faced this issue around a year and half ago and hacked
> some caching thing.
Could you please share how this was done?
Perhaps a patch?

> 
>> 
>> The user experience with narrow spec-file format is also not very friendly as you are restricted to full path and no glob support.
> IIRC, you can definitely do that. The narrow spec-file format supports
> mercurial file patterns mentioned in `hg help patterns`
This is from tests/test-narrow-patterns.t

> Illegal patterns are rejected
> 
>   $ hg tracked --addinclude glob:**
>   abort: invalid prefix on narrow pattern: glob:**
>   (narrow patterns must begin with one of the following: path:, rootfilesin:)
>   [255]

Cheers,
Son Luong.