Excessive memory usage on push
MHarbison at attotech.com
MHarbison at attotech.com
Wed Jun 24 16:10:45 UTC 2015
Pierre-Yves David <pierre-yves.david at ens-lyon.org> wrote on 06/23/2015
11:26:50 PM:
> From: Pierre-Yves David <pierre-yves.david at ens-lyon.org>
> To: Matt Harbison <mharbison72 at gmail.com>, evolve-testers at selenic.com,
> Cc: Matt Harbison <mharbison at attotech.com>
> Date: 06/23/2015 11:27 PM
> Subject: Re: Excessive memory usage on push
>
>
>
> On 06/23/2015 06:56 PM, Matt Harbison wrote:
> > I'm not sure what I've run into here, but I thought I would throw it
out
> > here to see if anyone has any ideas.
> >
> > I have 10.8.5, with hg 3.4.1 acting as a server at work. I didn't
enable
> > evolve on it, but did enable the experimental config:
> >
> > $ hg showconfig experimental
> > experimental.evolution=createmarkers,exchange,allowunstable
> >
> > It pulls from selenic.com at night, and I guess I would have expected
it to
> > pull the obsolete markers too. The local hg repo I do the work on does
> > have
> > evolve enabled, but when I push from it to the Mac server, it looks
like it
> > is trying to push all of the evolve markers (or certainly way more than
> > I've created locally). The progressbar indicated it would take 20+
> > minutes, so I let it run in the background. When the whole system
slowed
> > to a crawl, I noticed the out of memory error, and task manager
indicated
> > that hg.exe was using 1.4GB of memory. (This is Windows 7, 32 bit with
2GB
> > of RAM.)
> >
> > $ hg push -r tip home
> > pushing to https://mharbison@....
> > searching for changes
> > remote has heads on branch 'default' that are not known locally:
> > 7fdd1782fc4e
> > remote: adding changesets
> > remote: adding manifests
> > remote: adding file changes
> > remote: added 8 changesets with 7 changes to 3 files (+1 heads)
> > pushing 75376 obsolescence markers in 1719 pushkey payload
(11255670
> > bytes)
> > abort: out of memory
>
> This probably deserve and upstream bug. Can you try turning bundle2 on
> on you client to use a more modern exchange protocol ?
>
> (set experimental.bundle2-exp=True on your client)
I can file one if you still think it is worth it (see below). Not sure if
bz will allow attaching obsstore files that size.
> >
> > The store/obsstore file on the server is 7,978,472 bytes, the local one
is
> > 8,981,240 bytes. The local hg version is 3.4, evolve is 043e5ca9322f.
>
> what does hg version --verbose says for evolve?
$ ./hg version --verbose
Mercurial Distributed SCM (version 3.4.1+653-647722c07d4e)
(see http://mercurial.selenic.com for more information)
Copyright (C) 2005-2015 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Enabled extensions:
blackbox
extdiff
progress
strip
mq
rebase
evolve 5.1.4
> > I'm not sure if this is expected (but undesired) behavior- I don't see
a
> > bug for it. (Or is this a sign of obsolete cycles?) I'm guessing if I
> > enable evolve on the server, it will just fix itself, because I haven't
> > had any issues like this at home using 'hg serve' between VirtualBox
VMs
> > with evolve on both sides. Is there any debugging I should do before
> > trying that, in case it hides/alters the problem?
>
> First time I hear about it, could be useful to find out where is comes
> from (but moderably interested if it goes away with bundle2)
With bundle2, it sailed through very quickly. I saved the original
obsstore files on both sides if you have other ideas to pursue this
further.
Here's the bundle2 exchange, with the certificate and auth messages
stripped out, in case that offers a clue. (Interesting the double
branchmap command, unless that is done once per branch.)
$ ./hg push -r tip home --config experimental.bundle2-exp=True --debug
pushing to https://mharbison@...
sending capabilities command
query 1; heads
sending batch command
searching for changes
taking initial sample
query 2; still undecided: 12, sample size is: 12
sending known command
2 total queries
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 402 bytes
preparing listkeys for "namespaces"
sending listkeys command
received listkey for "namespaces": 48 bytes
checking for updated bookmarks
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 42 bytes
sending branchmap command
sending branchmap command
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 42 bytes
2 changesets found
list of changesets:
64ee46ed27022fc1033265f7d08dc48a9c71e91d
1de5b35cecd32a4691d0542d03348c13e5813c95
bundle2-output-bundle: "HG20", 6 parts total
bundle2-output-part: "replycaps" 172 bytes payload
bundle2-output-part: "check:heads" streamed payload
bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
bundle2-output-part: "obsmarkers" streamed payload
bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
sending unbundle command
sending 7986857 bytes
bundle2-input-bundle: with-transaction
bundle2-input-part: "reply:changegroup" (advisory) (params: 0 advisory)
supported
bundle2-input-part: "output" (advisory) (params: 0 advisory) supported
bundle2-input-part: total payload size 100
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 4 changes to 3 files
bundle2-input-part: "reply:pushkey" (params: 0 advisory) supported
bundle2-input-part: "reply:obsmarkers" (params: 0 advisory) supported
bundle2-input-part: "output" (advisory) (params: 0 advisory) supported
bundle2-input-part: total payload size 28
remote: 11 new obsolescence markers
bundle2-input-part: "reply:pushkey" (params: 0 advisory) supported
bundle2-input-part: "output" (advisory) supported
bundle2-input-bundle: 6 parts total
updating bookmark @
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 402 bytes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial-evolve-testers/attachments/20150624/0e4c0637/attachment-0002.html>
More information about the Evolve-testers
mailing list