How do I restore missing largefiles to a repository?

[Matt Harbison]

On Tue, 20 Aug 2019 10:20:25 -0400, Ansis Māliņš <ansis.malins at gmail.com>  
wrote:

> Here’s what happened:
> 1. Our central repository with the full complement of largefiles went  
> down
> 2. I published my personal copy as a new central repository
> 3. People pushed to it and now it’s ahead of the old repository,  
> including new largefiles
> 4. I’ve regained access to the old repository
>
> Question: How do I transfer largefiles from the old central repository  
> to the new central repository so it has a full complement? I must do so  
> while preserving any new largefiles that have since been pushed to the  
> new repository.

The blobs are stored in ".hg/largefiles".  So as long as you don't delete  
anything from there on the server, you should be able to add the missing  
ones from your local repo.

Alternately if you want, you can `hg serve` your full repo, and then  
update to each rev in the repo on the server.  (You will need to  
explicitly set paths.default to your full repo).  This will fetch the  
missing files, and cache them in the global store as well as the repo  
itself.

> Alternatively: what is the right thing to do in this situation such that  
> I end up with a fully complete central repository with all new and old  
> largefiles?
>
> The problem with the new central repository is that cloning to a machine  
> that has an empty local largefile cache crashes server side (and hangs  
> client side at “getting changed largefiles”) with:
>
> 2019-08-19 22:19:23.846 [cgi-pool-722] WARN  
> sonia.scm.web.cgi.DefaultCGIExecutor - Traceback (most recent call  
> last): File "/home/scm-server/.scm/lib/python/hgweb.py", line 51, in  
> <module> wsgicgi.launch(application) File  
> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/wsgicgi.py", line 87,  
> in launch for chunk in content: File  
> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line  
> 310, in run_wsgi for r in self._runwsgi(req, repo): File  
> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line  
> 350, in _runwsgi return protocol.call(rctx.repo, req, cmd) File  
> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/protocol.py", line  
> 111, in call rsp = wireproto.dispatch(repo, p, cmd) File  
> "/usr/lib/python2.7/dist-packages/mercurial/wireproto.py", line 575, in  
> dispatch return func(repo, proto, *args) File  
> "/usr/lib/python2.7/dist-packages/mercurial/wireproto.py", line 651, in  
> batch result = func(repo, proto, *[data[k] for k in keys]) File  
> "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 88,  
> in statlfile filename = lfutil.findfile(repo, sha) File  
> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 102,  
> in findfile path, exists = findstorepath(repo, hash) File  
> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 212,  
> in findstorepath if instore(repo, hash): File  
> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 193,  
> in instore return os.path.exists(storepath(repo, hash, forcelocal)) File  
> "/usr/lib/python2.7/genericpath.py", line 26, in exists os.stat(path)  
> TypeError: stat() argument 1 must be encoded string without null bytes,  
> not str 2019-08-19 22:19:23.850 [qtp1763847188-30] WARN  
> sonia.scm.web.HgCGIExceptionHandler - Mercurial/Python process ends with  
> return code 1
>
> Server side mercurial version is 4.0 (from Debian 9’s default repo).

You might want to upgrade the server version, even if you have to build  
 from source.  I can't say that a lot has changed in largefiles in the last  
2.5+ years, but I'm sure there have been some fixes and changes.

> Thank you for your time,
> Ansis.