hgwatchman: inotify that works

[Martin Geisler]

Siddharth Agarwal <sid0 at fb.com> writes:

Hey!

> We've just released the fast status Mercurial extension that we're
> using at FB.
>
> https://bitbucket.org/facebook/hgwatchman
>
> This went through a beta test of over a month, and has been turned on
> for everyone using Mercurial at FB for a couple of weeks.
>
> For our 200,000 file repo, this brings hg status down from over 3
> seconds to under 0.6.

That's really cool! Thanks for the hard work you guys are putting into
Mercurial.

I tried it (after fixing some small 32-bit build errors in watchman)
with the OpenOffice repo where there are 69k files in the working copy.
The repo is on a SSD and the time went from 0.5 sec to 0.17 sec.

For fun, I tried with the statdaemon extension I wrote a long time ago.
There I didn't see any performance improvement when I turned it on.

I haven't looked at watchman in detail, but do they do something like I
had to do here to make sure I get a consistent view of the filesystem:

  https://bitbucket.org/aragost/statdaemon/src/tip/shared.py#cl-60

That is, touch a well-known file and wait until the event thus generated
is seen. Assuming that FS events are delivered in FIFO order, the daemon
knows that it has seen all events up to the time it was called when it
sees the touched file.

As I recall it, this was to guard against a situation where you run

  echo foo > a.txt; hg status

and where 'hg status' gets FS info from the daemon without the update to
a.txt because that event is pending in the kernel.

I don't remember I my assumption above was ever validated so I would be
interested in any info you have about this, i.e., what does inotify
promise about the delivery order of events and can an event even be
delayed in the kernel so that the two applications ("echo foo > a.txt"
and "hg status") gets inconsistent views of the filesystem.

-- 
Martin Geisler
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mercurial-scm.org/pipermail/mercurial-devel/attachments/20130902/ecd7a282/attachment.asc>