hg verify exceptions and missed errors

[Sanjoy Mahajan]

In trying to reproduce a problem I noticed long ago with 'hg verify', I
made the following test script.  It makes a simple repository, then adds
a newline to the end of each of the .hg/ files and runs 'hg verify'
(each add and test done from a clean copy).

  #!/bin/bash

  d=`mktemp -d`
  cd $d

  hg init ; touch a ; hg add a ; hg commit -m 'Added a'
  echo === testing master copy: =====
  hg verify

  files=`find .hg -type f`
  for f in $files ; do
    work=`mktemp -d`
    cp -a . $work/
    echo ; echo === testing perturbation to $f: =====
    ( cd $work ; echo "" >> $f ; hg verify )
  done

I'm using v0.8 (0fc1b588976f), python 2.3.5, Debian Linux i386

The summary results (full log below sig):

'hg verify' spots the change to these files:
  .hg/data/a.d (i.e. when adding newline to .hg/data/a.d)
  .hg/00manifest.d
  .hg/00changelog.d

It generates an exception when these files are changed:
  .hg/data/a.i (unpack str size does not match format)
  .hg/00manifest.i (ditto)
  .hg/00changelog.i (ditto)

It does not detect the change to these files:
  .hg/dirstate
  .hg/undo
  .hg/undo.dirstate

-Sanjoy

`A society of sheep must in time beget a government of wolves.'
   - Bertrand de Jouvenal


The full log from running the script:

=== testing master copy: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions

=== testing perturbation to .hg/data/a.d: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
a file data short -1 bytes
1 files, 1 changesets, 1 total revisions
1 integrity errors encountered!

=== testing perturbation to .hg/data/a.i: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
** unknown exception encountered, details follow
** report bug details to mercurial at selenic.com
** Mercurial Distributed SCM (version 0fc1b588976f)
Traceback (most recent call last):
  File "/usr/bin/hg", line 13, in ?
    commands.run()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2550, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2718, in dispatch
    return d()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2703, in <lambda>
    d = lambda: func(u, repo, *args, **cmdoptions)
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2245, in verify
    return repo.verify()
  File "/usr/lib/python2.3/site-packages/mercurial/localrepo.py", line 1773, in verify
    fl = self.file(f)
  File "/usr/lib/python2.3/site-packages/mercurial/localrepo.py", line 167, in file
    return filelog.filelog(self.opener, f)
  File "/usr/lib/python2.3/site-packages/mercurial/filelog.py", line 17, in __init__
    os.path.join("data", self.encodedir(path + ".d")))
  File "/usr/lib/python2.3/site-packages/mercurial/revlog.py", line 218, in __init__
    e = struct.unpack(indexformat, i[f:f + s])
struct.error: unpack str size does not match format

=== testing perturbation to .hg/dirstate: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions

=== testing perturbation to .hg/00manifest.d: =====
checking changesets
checking manifests
manifest data short -1 bytes
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions
1 integrity errors encountered!

=== testing perturbation to .hg/00manifest.i: =====
** unknown exception encountered, details follow
** report bug details to mercurial at selenic.com
** Mercurial Distributed SCM (version 0fc1b588976f)
Traceback (most recent call last):
  File "/usr/bin/hg", line 13, in ?
    commands.run()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2550, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2699, in dispatch
    repo = hg.repository(ui=u, path=path)
  File "/usr/lib/python2.3/site-packages/mercurial/hg.py", line 28, in repository
    return localrepo.localrepository(ui, path, create)
  File "/usr/lib/python2.3/site-packages/mercurial/localrepo.py", line 34, in __init__
    self.manifest = manifest.manifest(self.opener)
  File "/usr/lib/python2.3/site-packages/mercurial/manifest.py", line 18, in __init__
    revlog.__init__(self, opener, "00manifest.i", "00manifest.d")
  File "/usr/lib/python2.3/site-packages/mercurial/revlog.py", line 218, in __init__
    e = struct.unpack(indexformat, i[f:f + s])
struct.error: unpack str size does not match format

=== testing perturbation to .hg/00changelog.d: =====
checking changesets
changeset data short -1 bytes
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions
1 integrity errors encountered!

=== testing perturbation to .hg/00changelog.i: =====
** unknown exception encountered, details follow
** report bug details to mercurial at selenic.com
** Mercurial Distributed SCM (version 0fc1b588976f)
Traceback (most recent call last):
  File "/usr/bin/hg", line 13, in ?
    commands.run()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2550, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2699, in dispatch
    repo = hg.repository(ui=u, path=path)
  File "/usr/lib/python2.3/site-packages/mercurial/hg.py", line 28, in repository
    return localrepo.localrepository(ui, path, create)
  File "/usr/lib/python2.3/site-packages/mercurial/localrepo.py", line 35, in __init__
    self.changelog = changelog.changelog(self.opener)
  File "/usr/lib/python2.3/site-packages/mercurial/changelog.py", line 15, in __init__
    revlog.__init__(self, opener, "00changelog.i", "00changelog.d")
  File "/usr/lib/python2.3/site-packages/mercurial/revlog.py", line 218, in __init__
    e = struct.unpack(indexformat, i[f:f + s])
struct.error: unpack str size does not match format

=== testing perturbation to .hg/undo: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions

=== testing perturbation to .hg/undo.dirstate: =====
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 1 changesets, 1 total revisions