[Bug 4443] New: Memory leaks when serving bundles

[mercurial-bugs at selenic.com]

http://bz.selenic.com/show_bug.cgi?id=4443

          Priority: normal
            Bug ID: 4443
                CC: mercurial-devel at selenic.com
          Assignee: bugzilla at selenic.com
           Summary: Memory leaks when serving bundles
          Severity: bug
    Classification: Unclassified
                OS: All
          Reporter: gregory.szorc at gmail.com
          Hardware: All
            Status: UNCONFIRMED
           Version: unspecified
         Component: Mercurial
           Product: Mercurial

Mozilla encountered a Mercurial outage today. As part of investigating the
servers, we discovered that our mod_wsgi hg processes are consuming a lot of
memory RSS. Hundreds of megabytes of RSS actually. The behavior we see is that
RSS increases by a few MB/s when processing getbundle commands in a pattern
that would indicate a leak in Mercurial.

I'm able to reproduce the memory leak locally by running `hg serve` and
watching the RSS of the process grow during a local clone. I've observed the
following with mozilla-central:

1) The biggest gains in RSS usage occur when streaming changesets. We gradually
go from just a few MB of RSS to 180 MB and then we see a large jump to ~320 MB
before manifest streaming starts.

2) RSS grows steadily during filelog transfer. There is some zigzag, so some
things are getting collected.

3) 2.5.4 peaks at 342 MB RSS at the end of clone. @ peaks at 525 MB.

4) After the clone has finished, the server is still sitting around with 350+
MB RSS. (This may be an intentional caching decision.)

#1 appears to have existed for all of time. I've reproduced with 2.0.2. #2 (and
presumably the related #3) appear to be regressions between 2.5.4 and @.

I'll keep the bug updated as I learn more.

-- 
You are receiving this mail because:
You are on the CC list for the bug.