[PATCH v3] debug: automate the process of truncating a damaged obsstore (issue5265)
Yuya Nishihara
yuya at tcha.org
Thu Jul 7 14:03:22 UTC 2016
On Mon, 4 Jul 2016 08:48:33 -0700, Simon Farnsworth wrote:
> # HG changeset patch
> # User Simon Farnsworth <simonfar at fb.com>
> # Date 1467646999 25200
> # Mon Jul 04 08:43:19 2016 -0700
> # Node ID 31eea7188911ca17b36e69170dfcec07f4da37ae
> # Parent fd93b15b5c30d16fd9c9eba61402d07fc4085db3
> debug: automate the process of truncating a damaged obsstore (issue5265)
>
> We occasionally see users who've had a system crash damage the obsstore file
> in their .hg/store directory; this makes all `hg` commands fail until we go
> in and remove the damaged section of the obsstore by hand.
>
> Automate the process we use when this happens, as a debug command because it
> loses the corrupted data. We only use it in rare circumstances when it's
> important to retrieve a user's work and apply it to a fresh clone.
>
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -15,6 +15,7 @@
> import re
> import shlex
> import socket
> +import struct
> import sys
> import tempfile
> import time
> @@ -3702,6 +3703,61 @@
> displayer.show(repo[r], **props)
> displayer.close()
>
> + at command('debugtruncatestore',
> + [('', 'obsolete', None, _('truncate bad markers in obsstore'))],
> + _('[OPTION]'))
> +def debugtruncatestore(ui, repo, **opts):
> + """Fix up repository corruption by truncating damaged files
> +
> + Most on-disk data structures are designed to be append-only. A failed write
> + (e.g. due to an unexpected power failure) can leave the file corrupted.
> +
> + This command attempts to recover from that situation by replacing the
> + corrupted file with a version that only contains the valid records from the
> + broken file. It is not guaranteed to remove all corrupt records - it will
> + only remove corrupt records where normal use of the repo would result in a
> + crash.
> +
> + Corrupt files will be renamed with a .corrupt extension before the fixed
> + version is written out, so that you can examine the corruption and/or undo
> + this command.
> +
> + You should normally use :hg:`recover` before resorting to this command.
> + """
> +
> + if 'obsolete' in opts:
Better to take a lock here. Otherwise obsstore could theoretically be modified
while finding the first bad position.
> + data = repo.svfs.tryread('obsstore')
> + if data:
> + # Slow algorithm - but this is an emergency debug operation
> + version = None
> + corrupt = False
> + while version is None and len(data) > 0:
> + try:
> + (version, markers) = obsolete._readmarkers(data)
> + # Force evaluation of all the markers - the pure
> + # implementation returns a generator which won't detonate
> + # until you evaluate the bad marker.
> + for marker in markers:
> + pass
> + except (ValueError, struct.error, error.Abort):
> + corrupt = True
> + version = None
> + data = data[:-1]
> + continue
> + break
> + if corrupt:
> + with repo.lock():
> + repo.svfs.rename('obsstore', 'obsstore.corrupt')
> + if len(data) > 0:
> + repo.svfs.write('obsstore', data)
> + ui.write(_('truncated damaged obsstore\n'))
> + else:
> + ui.write(_('deleted unreadable obsstore\n'))
> + else:
> + ui.write(_('no damage to obsstore\n'))
> + else:
> + ui.write(_('no obsstore\n'))
More information about the Mercurial-devel
mailing list