www.kernel.org/hg/linux-2.6 corrupted?

[Matt Mackall]

On Wed, Dec 06, 2006 at 02:30:58AM -0800, Paul Jackson wrote:
> Benoit wrote:
> > The problem is that the repo in kernel.org is rsync'ed, and we use a
> > lockless scheme for readers.
> 
> Thanks for the quick reply.
> 
> A couple of questions:
> 
>  1) Can I fix my corrupted repositories easily, or is it better
>     to just make new ones afresh?

Yes. Do a local clone -r to a revision that's, say, 95% of the way
through the history. This will get rid of the broken changes. Then
pull from kernel.org again.
 
>  2) When will this be working again?  Is it some transient
>     in kernel.org that will just automatically be ok in an hour;
>     is it some corruption in kernel.org that requires human
>     intervention (perhaps in a day); is it something that
>     requires recoding to ever work again (perhaps in a week)?

Here's the problem:

The repository is mirrored on a machine called master.kernel.org. For
security purposes, master has an automated process to mirror all
users' public information to www.kernel.org with rsync. Only admins
have any other access to www.kernel.org.

If a user on www.kernel.org does a pull while this is happening, they
may race with rsync and see an inconsistent state because the
changelog file is mirrored before the rest of the files. This means
the CGI server will see changesets that are not fully committed.
Judging from stats from the mirror on hg.intevation.org, this race
happens < 1% of the time.

Recent rsync happens to sort things so that files (sorted) are sent
before directories (sorted). Git happens to avoid this problem by
having their topmost pointers in refs/ which is rsynced after their
other data which lives in objects/. On the other hand, Mercurial
orders things such that changes to 00changelog.i are always written
last but that file ends up getting rsynced first.
 
>  3) How will I know when it is working again?

We're going to try to fix this while we make changes to deal with the
case-folding problem. It will be in the upcoming release.

The ideal fix, of course, is not to use rsync. It's significantly less
efficient here than Mercurial's own protocol and because it doesn't
respect Mercurial's locking or ordering rules, is likely to race. But
kernel.org's policy is not something I can control.

-- 
Mathematics is the supreme nostalgia of our time.