Computing the union of two revlogs

[Greg Ward]

On Fri, Feb 12, 2010 at 8:15 PM, Benoit Boissinot
<benoit.boissinot at ens-lyon.org> wrote:
> On Fri, Feb 12, 2010 at 06:21:50PM -0500, Greg Ward wrote:
>>
>> This seems to work.  I'm going to try to write some unit tests for it,
>> but in the meantime I thought I would bounce it off here and see if
>> anything smells funny to you folks.  In particular, it took me quite a
>> while to get the copy of 'target' to create 'union' just right, and
>> I'm wondering if there is a simpler way.  Also, it takes ~1.5 sec to
>> run on our ~106k changeset repo; if you can think of any good
>> speedups, I'm listening.
>
> Do you have a profiling output?

Good idea!  This is vanilla cProfile+pstats output, not hg-style
profiling.  (My '_revlogUnion()' function is standalone code, not part
of a Mercurial extension.)  But it's pretty clear what's going on:

         530917 function calls (530915 primitive calls) in 2.525 CPU seconds

   Ordered by: cumulative time
   List reduced from 42 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    2.525    2.525 <string>:1(<module>)
        1    1.104    1.104    2.522    2.522
python/ims/dev/hgutil.py:8(_revlogUnion)
   105940    0.287    0.000    0.779    0.000
mercurial/revlog.py:269(__getitem__)
   105938    0.193    0.000    0.393    0.000
mercurial/revlog.py:288(__contains__)
      103    0.000    0.000    0.278    0.003 mercurial/revlog.py:264(load)
      103    0.001    0.000    0.278    0.003 mercurial/revlog.py:234(loadindex)
      103    0.270    0.003    0.277    0.003 mercurial/revlog.py:164(loadblock)
   105939    0.213    0.000    0.213    0.000 {_struct.unpack}
        3    0.193    0.064    0.200    0.067 mercurial/revlog.py:137(loadmap)
   105936    0.163    0.000    0.163    0.000
mercurial/revlog.py:303(__getitem__)
[...a few other dribs and drabs...]

Hmmm.  Doesn't look like there's much more I can squeeze out of that.

Greg