[PATCH] Add script to rewrite manifest to workaround lack of parent deltas

[Greg Ward]

On Fri, Aug 21, 2009 at 5:58 PM, I wrote:
>> Maybe it's cleaner to pop from the end
>>        i = visit.pop()
> [...]
>>> +            if len(parents_with_child) == 0:
>>> +                next.append(c)
>>> +        visit = next + visit
>> if you pop from the end, then you can do:
>>        visit += next
>
> Not only is it cleaner, it massively improves the shrinkage factor on
> my test repo: from 8.5x smaller to 16.9x smaller.  Specifically,
> pop(0) with "visit = next + visit" shrank a 56.1 MB manifest to 5.5
> MB; fiddling with the order shrank it to 3.3 MB instead.  Wow!  I
> suppose I should test it for correctness though.  ;-)

Damn.  Should have known it was too good to be true; this makes the
sort rather unstable.  Example: with the original, visit.pop(0) and
'visit = next + visit' version, repeated runs of shrink-manifest look
like this:

$ ~/src/hg-crew/contrib/shrink-manifest.py
reading 15043 revs ................
sorting ...
writing 15043 revs ................
old file size:     58830219 bytes (  56.1 MiB)
new file size:      6929734 bytes (   6.6 MiB)
shrinkage: 88.2% (8.5x)
$ rm .hg/store/*.old
$ ~/src/hg-crew/contrib/shrink-manifest.py
reading 15043 revs ................
sorting ...
writing 15043 revs ................
old file size:      6929734 bytes (   6.6 MiB)
new file size:      6929734 bytes (   6.6 MiB)
shrinkage: 0.0% (1.0x)

Good: stable and predictable.

But with visit.pop() and "visit += next", it's not so good:

$ ~/src/hg-crew/contrib/shrink-manifest.py
reading 15043 revs ................
sorting ...
writing 15043 revs ................
old file size:     58830219 bytes (  56.1 MiB)
new file size:      3472373 bytes (   3.3 MiB)
shrinkage: 94.1% (16.9x)
$ rm .hg/store/*.old
$ ~/src/hg-crew/contrib/shrink-manifest.py
reading 15043 revs ................
sorting ...
writing 15043 revs ................
old file size:      3472373 bytes (   3.3 MiB)
new file size:     17573799 bytes (  16.8 MiB)
shrinkage: -406.1% (0.2x)
$ rm .hg/store/*.old
~/src/hg-crew/contrib/shrink-manifest.py
reading 15043 revs ................
sorting ...
writing 15043 revs ................
old file size:     17573799 bytes (  16.8 MiB)
new file size:      5870243 bytes (   5.6 MiB)
shrinkage: 66.6% (3.0x)

Yuck: I don't like that behaviour.  It should be OK to run
shrink-manifest repeatedly on an already-shrunk manifest.  Reverting
to the original algorithm.

Greg