D8410: nodemap: also use persistent nodemap for manifest

[marmoute (Pierre-Yves David)]

marmoute created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The manifest as a different usage pattern than the changelog. First, while the
  lookup in changelog are not garanteed to match, the lookup in the manifest
  nodemap come from changelog and will exist in the manifest. In addition, looking
  up a manifest almost always result in unpacking a manifest an operation that
  rarely come cheap.
  
  Nevertheless, using a persistent nodemap provide a significant gain for some
  operations.
  
  For our measurementw, we use `hg cat --rev REV FILE` on the our reference
  mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
  side for a total store side of 11,988 MB
  
  File with large history (file: b2g/config/gaia.json, revision: 195a1146daa0)
  
    no optimisation:                             0.358s
    using mmap for index:                        0.297s (-0.061s)
    persistent nodemap for changelog only:       0.275s (-0.024s)
    persistent nodemap for manifest too:         0.258s (-0.017s)
  
  File with small history (file: .hgignore, revision: 195a1146daa0)
  
    no optimisation:                             0.377s
    using mmap for index:                        0.296s (-0.061s)
    persistent nodemap for changelog only:       0.274s (-0.022s)
    persistent nodemap for manifest too:         0.257s (-0.017s)
  
  Same file but using a revision (8ba995b74e18) with a smaller manifest (3944829
  bytes vs  10 bytes)
  
    no optimisation:                             0.192s (-0.185s)
    using mmap for index:                        0.131s (-0.061s)
    persistent nodemap for changelog only:       0.106s (-0.025s)
    persistent nodemap for manifest too:         0.087s (-0.019s)

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8410

AFFECTED FILES
  mercurial/manifest.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -32,6 +32,9 @@
 
   $ f --sha256 .hg/store/00changelog-*.nd
   .hg/store/00changelog-????????????????.nd: sha256=2e029d3200bd1a986b32784fc2ef1a3bd60dc331f025718bcf5ff44d93f026fd (glob)
+
+  $ f --sha256 .hg/store/00manifest-*.nd
+  .hg/store/00manifest-????????????????.nd: sha256=97117b1c064ea2f86664a124589e47db0e254e8d34739b5c5cc5bf31c9da2b51 (glob)
   $ hg debugnodemap --dump-new | f --sha256 --size
   size=121088, sha256=2e029d3200bd1a986b32784fc2ef1a3bd60dc331f025718bcf5ff44d93f026fd
   $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -1557,6 +1557,9 @@
             checkambig=not bool(tree),
             mmaplargeindex=True,
             upperboundcomp=MAXCOMPRESSION,
+            persistentnodemap=opener.options.get(
+                b'exp-persistent-nodemap', False
+            ),
         )
 
         self.index = self._revlog.index



To: marmoute, #hg-reviewers
Cc: mercurial-devel