Mercurial as a database backend

[Greg Ward]

On 10 August 2012, Adam B said:
> Hello all,
> 
> I have a question about Mercurial's internals.  I'm interested in using
> Mercurial as a database backend for a peer-to-peer application.  One of the
> main challenges I see with this is maintaining efficiency with hundreds of
> thousands of records.  Logically each database "record" should be a
> versioned file in the repository.  However, storing 100k+ files is a really
> inefficient use of the disk, especially if the records are small.
> 
> What I wonder is, how difficult would it be to modify Mercurial to store
> the files in, for example, a SQLite database?  Not the revlog mind you,
> that would stay as-is.  I'm just talking about the working copy of each
> file.  Is the Mercurial code base tightly coupled with the file system or
> is it's interface with file system nicely factored into a few key
> functions/classes?
> 
> I would only need to support the essential mercurial functions like: add,
> remove, commit, revert, push, pull, update, merge, and log.

Version control is not a database.

revlog is a pretty clever file format when the data is delta-friendly
and compressible. (E.g. it would suck for encrypted data, but it's
great for uncompressed images and plain text.) I really think you want
to peel back a few layers and look directly at revlog.

(And if you want a fast, low-memory, pure C implementation of revlog,
I've got just the thing for you: clone http://hg.gerg.ca/xrevlog and
poke around.)

       Greg
-- 
Greg Ward                                http://www.gerg.ca/
OUR PLAN HAS FAILED STOP JOHN DENVER IS NOT TRULY DEAD
STOP HE LIVES ON IN HIS MUSIC STOP PLEASE ADVISE FULL STOP