hg pull runs out of memory

[Matt Mackall]

On Fri, 2012-04-27 at 10:16 +0200, Hertroys A. wrote:
> > > > Because Mercurial is designed for dealing with _source code_ quickly.
> > > > And it's massively more efficient to deal with small files typical of
> > > > source code by reading, writing, and calculating deltas on them in
> > > > memory.
> > >
> > > Of course, but isn't it kind of dumb to attempt the same with a large
> > > binary file?
> > 
> > The engineering effort to change the situation is huge and the demand is
> > small.
> 
> Huge? You're the expert in this matter of course, but that's quite contrary to what I expected.

I expect it to be about a year of my time, retrofitting a much more
complex second path into _every single place where file data is
consulted_ in the code base. Even doing the bare minimum (making
checkout and clone work with arbitrarily large files) would require
adding a second path into the backend storage system and rewriting all
the delta and reconstruction code. And during that year, someone else
will need to do everything else I'm currently doing.

The substantial amount of time sunk in the (already very complex but
still very incomplete) largefiles code was done in part because it was a
short-cut around the above.

So yes, you're definitely underestimating the problem.

> > I'll consider fixing it for about $100k. That same $100k of
> > course will buy you lots of 64-bit systems.
> 
> I know the dollar is a bit low these days, but in euro's that's about 5 to 6 years of decent pay!

Not sure where you live, but before I started working on Mercurial, my
annual salary was about twice the quoted figure and included health
insurance and stock options. And I quit that job to do something more
interesting, largely on my own time. To get me to do something I wasn't
interested in for a whole year would actually cost a lot more than the
quoted figure.

But really, I just threw a number out to make you understand the request
wasn't trivial.

-- 
Mathematics is the supreme nostalgia of our time.