hg repositories and Windows DFS Replication
Matt Brown - nyc
MatthewBrown at gmail.com
Tue Sep 4 19:46:20 UTC 2012
Guys, before misinfo spreads further, DFS-R does deal with large files like
any other file.
But... you have to keep the method DFS-R uses in mind considering:
1) a file is written to a folder monitored for replication on SERVER A
2) SERVER A then produces a delta check sum of the already generated check
sum of the file (which is a version of the originally generated checksum of
the file when DFS-R was configured to monitor this folder).
3) SERVER A communicates to all receiving servers (we'll call this entity
simply SERVER B) to figure out if SERVER B has any data "blocks" in its
existing cache that will match this file delta... If so, then it replicates
a small amount, if not, it replicates a large amount, up to the total size
of the delta.
4) SERVER B receives the delta data
5) SERVER B "stages" the file==writes the ENTIRE contents of the file in to
a staging area [this is where the bottleneck exists in the instance of
pre-existing large files that have many little changes taking place]
6) If during the staging on SERVER B, SERVER A sends another change to the
same file, SERVER B cancels the current staging operation on CHANGE 1 and
starts processing CHANGE 2... this "loop" can introduce huge problems...
because... you got it... ORIGINAL FILE will always be ORIGINAL FILE until
CHANGE N*94 is allowed to finish (see step 7)
7) When SERVER B is finally allowed to finish writing the file completely,
it simply moves the file (aka changes the block location(s) entry in the
file system table for the file) to it's final production location (and
"staging" is finished.
http://mbrownnyc.wordpress.com/2011/08/24/method-to-really-monitor-dfs-replication/
Due to these necessary steps performed, generally, it is NOT recommended to
use DFS-R to replicate extremely large files, but as you already gathered,
that truly relies on what your receiving servers' write capabilities.
On Tuesday, May 31, 2011 9:19:22 PM UTC-4, Robert Altman wrote:
>
> Has anyone ever used mercurial on Windows servers with DFS replicating the
> repositories to another server. (This would essentially server to "mirror"
> the repositories to another computer.)
>
> As long as the changes are only made on one server (the web server), is
> there any risk to the data itself? (I wouldn't think so, but I'm curious
> if anyone has tried it.)
>
> Thanks.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial/attachments/20120904/add6363f/attachment-0002.html>
More information about the Mercurial
mailing list