[External] remotefilelog: Large Manifest

[Son Luong Ngoc]

Hi Yves,

> On Mar 1, 2020, at 20:51, Pierre-Yves David <pierre-yves.david at ens-lyon.org> wrote:
> +1 using zstd don't have a huge impact on size, but speed up read/write up to about 50%
> 
> You can see some number here https://urldefense.com/v3/__https://www.mercurial-scm.org/repo/hg/rev/bb271ec2fbfb__;!!FzMMvhmfRQ!6rrRJJuYzgXSEUsccdjBdOayyRReVBhJmffLSSH9LKaeW20V_3FzB0FhFMjgUoRq$ <https://urldefense.com/v3/__https://www.mercurial-scm.org/repo/hg/rev/bb271ec2fbfb__;!!FzMMvhmfRQ!6rrRJJuYzgXSEUsccdjBdOayyRReVBhJmffLSSH9LKaeW20V_3FzB0FhFMjgUoRq$> 

I can try this but not sure whats the best way to do a format conversion?
A re-convert (from using hg-git) is expensive on large repo.
Trying "hg --config format.revlog-compression=zstd clone local1 local2" now, hopefully I can just copy the hg-git files over after.

> Can you share some information to help with diagnostic ?
I will not be able to give the exact number but I will try to give a rough estimations

> - How many changeset do you have ?
150k

> - How many files do you have in your working copy ?
Should not be relevant since we really try to have a sparse-checkout working, most of my tests are with --noupdates
But I would say around 200k

> - How many different file did you had in the repository ever ? `hg manifest --all | wc -l`
450k

> - How many merge do you have.
Using "git rev-list --merges --count HEAD" in the default branch of our git repo: 100k

> You can also trying basic maintenance thing like `hg debugupgraderepo -o re-delta-all --run` and making a local pulling clone `hg clone --pull local local2`
I am playing with Mercurial in a KVM which, although not much limitation on computing, does have limitation in storage, especially coupling with 
A local re-clone is relatively expensive as I already have 2 copies (1 without treemanifest, 1 with treemanifest, each with a copy of .hg/git inside.
It would be nice if we can graduate treemanifest from experimental soon.
I was thinking of infinitepush but the official support seems relatively weak (I will send a separate email regarding this).

> I don't have much time today. I'll try to look into your other question later.
Thanks a lot for your replies!
Gona look into the format and debug commands later today.

Cheers,
Son Luong.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20200302/ec4998f9/attachment-0002.html>