git-annex vs largefiles (was: Re: Mercurial popularity is stagnant)

[Augie Fackler]

On Jul 1, 2014, at 5:31 PM, Michael McNeil Forbes <michael.forbes at gmail.com> wrote:

> On Jul 1, 2014, at 1:50 PM, Bruce Cran <bruce at cran.org.uk> wrote:
>> On 7/1/2014 1:57 PM, Angel Ezquerra wrote:
>>> I may be missing something but isn't the git-annex git's equivalent to mercurial's built in largefiles extension?
>> Or the 3rd-party BigFiles (http://mercurial.selenic.com/wiki/BigfilesExtension) or Snap (http://mercurial.selenic.com/wiki/SnapExtension)?
> 
> The big advantage of git-annex is the files can be distributed: you do not need
> to have all of the files in a single location.  It keeps track of where the
> various files are.  Thus, you can checkout the source repository on your laptop
> and the get only a few of the data files associated with a run for analysis.
> The killer feature is that git-annex keeps track of where the various files are
> located, making sure that you keep at least n reliable copies somewhere, but
> not demanding that all of the files be kept anywhere in specific.

I'm unfamiliar with git-annex, so please bear with me. Doesn't there have to be a place where all the data files are present? That is, don't you necessarily have to have some central place where all the files exist for reliability purposes?

It sounds like git-annex adds some limited sparse checkout capabilities on top of largefiles. You may be interested in the work Durham at Facebook has been doing: https://bitbucket.org/DurhamG/hg/src/de9df54e2667d80f78b15e7b754033b550b6f1a8/hgext/sparse.py

> As far as I can see, not of the mercurial extensions comes even close to
> providing this type of functionality.  (Either all the files need to be
> stored on some remote server, or all the files need to be stored locally.)
> 
> Michael.
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20140701/dcabfc13/attachment.asc>