Slow clone due to missing/invalid entries in tags cache

[Kim Randell]

> -----Original Message-----
> From: Pierre-Yves David <pierre-yves.david at ens-lyon.org>
> Sent: 28 April 2021 10:46
> To: Kim Randell <Kim.Randell at vicon.com>; mercurial at mercurial-scm.org
> Subject: Re: Slow clone due to missing/invalid entries in tags cache
> 
> Mercurial 5.7 has become more aggressive about warming this cache, which is
> why you had so many "missing" entry before. However the 1500s cache
> warming time is definitely more brutal than what I would expect.

Thanks - I also posted more about this on the mercurial-devel list after doing a bit more investigation and not seeing much activity on this list. I can repost that here as well to consolidate the thread if that would help.
 
> How many revisions does your repository have ? Can you share the output of the
> following commands:

130113 revisions

> $ hg debugformat

format-variant     repo
fncache:            yes
dotencode:          yes
generaldelta:       yes
share-safe:          no
sparserevlog:        no
sidedata:            no
persistent-nodemap:  no
copies-sdc:          no
plain-cl-delta:     yes
compression:        zlib
compression-level:  default
 
> $ hg debugrevlog --manifest

format : 1
flags  : generaldelta

revisions     :     130113
    merges    :      19266 (14.81%)
    normal    :     110847 (85.19%)
revisions     :     130113
    empty     :        215 ( 0.17%)
                   text  :          0 ( 0.00%)
                   delta :        215 (100.00%)
    snapshot  :        127 ( 0.10%)
      lvl-0   :              127 ( 0.10%)
    deltas    :     129771 (99.74%)
revision size : 2477041417
    snapshot  :  260963543 (10.54%)
      lvl-0   :        260963543 (10.54%)
    deltas    : 2216077874 (89.46%)

chunks        :     130113
    0x00      :      19910 (15.30%)
    empty     :        215 ( 0.17%)
    0x78 (x)  :     109988 (84.53%)
chunks size   : 2477041417
    0x00      :    1895237 ( 0.08%)
    empty     :          0 ( 0.00%)
    0x78 (x)  : 2475146180 (99.92%)

avg chain length  :     1983
max chain length  :    15098
max chain reach   : 33118665
compression ratio :      312

uncompressed data size (min/max/avg) : 427399 / 8342610 / 5953934
full revision size (min/max/avg)     : 169045 / 2450453 / 2054831
inter-snapshot size (min/max/avg)    : 0 / 0 / 0
delta size (min/max/avg)             : 12 / 1615450 / 17076

deltas against prev  : 119247 (91.89%)
    where prev = p1  :  94605     (79.34%)
    where prev = p2  :   3399     ( 2.85%)
    other            :  21243     (17.81%)
deltas against p1    :  10430 ( 8.04%)
deltas against p2    :     58 ( 0.04%)
deltas against other :     36 ( 0.03%)

> $ hg debugrevlog .hgtags

format : 1
flags  : inline, generaldelta

revisions     :   242
    merges    :    56 (23.14%)
    normal    :   186 (76.86%)
revisions     :   242
    empty     :     2 ( 0.83%)
                   text  :     0 ( 0.00%)
                   delta :     2 (100.00%)
    snapshot  :     4 ( 1.65%)
      lvl-0   :           4 ( 1.65%)
    deltas    :   236 (97.52%)
revision size : 35265
    snapshot  :  1296 ( 3.68%)
      lvl-0   :        1296 ( 3.68%)
    deltas    : 33969 (96.32%)

chunks        :   242
    0x00      :   179 (73.97%)
    empty     :     2 ( 0.83%)
    0x75 (u)  :     2 ( 0.83%)
    0x78 (x)  :    59 (24.38%)
chunks size   : 35265
    0x00      : 13157 (37.31%)
    empty     :     0 ( 0.00%)
    0x75 (u)  :   116 ( 0.33%)
    0x78 (x)  : 21992 (62.36%)

avg chain length  :    46
max chain length  :   103
max chain reach   : 28433
compression ratio :    34

uncompressed data size (min/max/avg) : 54 / 10816 / 5019
full revision size (min/max/avg)     : 55 / 783 / 324
inter-snapshot size (min/max/avg)    : 0 / 0 / 0
delta size (min/max/avg)             : 12 / 3127 / 143

deltas against prev  : 188 (79.66%)
    where prev = p1  : 133     (70.74%)
    where prev = p2  :  26     (13.83%)
    other            :  29     (15.43%)
deltas against p1    :  48 (20.34%)
deltas against p2    :   0 ( 0.00%)
deltas against other :   0 ( 0.00%)

Cheers,

Kim