Question on work-flow for big binary files

[Marcus Lindblom]

Douglas Philips wrote:
> On or about 2008 Dec 18, at 8:36 AM, Marcus Lindblom wrote:
> 
>> It boils down to this: is it feasible with a distributed versioning
>> system of really large files? Perhaps I should run some tests, but  
>> it's
>> hard to predict performance a few years down the line.
>>
>> Anyone with experience in that field? How much data can one feasibly
>> work with in a DVCS? Am I trying to use the wrong tool for the job?
> 
> 
> You might consider something like:
> 	http://wiki.netbeans.org/HgExternalBinaries

Very Interesting!

I actually handcrafted something similar using wget and some scripts for 
use with third-party libraries only (not content), that don't change 
much at all. (Currently I just keep them in a dir on the webserver, 
that's available for writing via SMB). That worked well with svn and 20 
developers using various branches and about 1 gig of external libaries 
(packed) for Java and C++.

This means I can throw my own stuff out and use HgExternalBinaries for 
the third-party libs. Yay!

Will it work for content? Perhaps, perhaps not.

At the very least, this separates the content versioning from the 
problem of connecting the right binaries with the right source revision.
(Except that one might want to define a external binary diretory, rather 
than individual files, but that could be solved via a small utility on 
top of hgextbinaries)

Essentially, content needs to be versioned, but not diffed. So a simple 
store-every-version (and some garbage) system with the latest files 
served via http(s) seems to be a good way.

Cheers,
/Marcus