Status of lfs and lock extensions

[Daniele Benegiamo]

On 2021/01/19 03:54, Matt Harbison wrote:
 >  > I would like to know if there are any news (or future plans) about the
 >  > "lfs" extension and any sort of support for a "centralized file
 >  > locking" feature.
 >
 > The LFS extension is shipped with Mercurial.  Don't let the fact that 
it is marked experimental scare you off- I use it in production.  The 
experimental tag is mostly so that we can change some of the 
fileset/revset/template functionality without worrying about backward 
compatibility.  I have no intention of changing the storage layout in an 
incompatible way.  There is a TODO list of future plans that I hope to 
get to some day:
 >
 >      https://www.mercurial-scm.org/repo/hg/file/5.6.1/hgext/lfs/TODO.rst

Hi Matt!

thanks a lot for all the shared information! And I'm very happy 
development on it is going forward.

I'll test it for sure. Can I follow the instructions in the extension 
documentation to set it up?

     https://www.mercurial-scm.org/repo/hg/file/5.6.1/hgext/lfs/__init__.py

(later few questions about the server)

(sorry for some basic questions about the setup - but we're new on the 
lfs protocol)


 > [...]
 >
 >  > While the "largefiles" extension is well-established, I would like to
 >  > know if there's any update about the "lfs" extension
 >  > (https://www.mercurial-scm.org/wiki/LfsPlan).
 >
 > If you are settled on using one or the other, I would *strongly* 
recommend using LFS instead of largefiles.  It is much simpler, and less 
likely to break commands in subtle ways.  You can convert between LFS 
and a normal repo using the convert extension without changing hashes, 
and the LFS and normal repo will be able to do exchanges with each 
other.  (The convert extension does have edge cases where it can change 
hashes when it probably shouldn't, but I've been planning to try to use 
the repo upgrade command to do a conversion that shouldn't change hashes.)

Thanks!

Could you give some more clues about the "break commands in subtle ways" 
about the "largefiles" extension? We just started testing such 
extensions, so we still don't have complete and detailed pros/cons.

If I understand correctly, the "convert" extension supports 
bi-directional conversions: normal <-> largefiles and normal <-> lfs. Is 
it right? Or there are limitations? (it would be very helpful to run our 
tests)


 >
 >  > The plan was to "re-use' the Git LFS protocol, and as such protocol
 >  > supports file locking
 >  > (https://github.com/git-lfs/git-lfs/wiki/File-Locking), it could be an
 >  > ideal solution for us.
 >
 > It does use the git-lfs protocol, though I didn't implement the 
VERIFY command.  (That's not about checking the file, it's about 
supporting uploading files to a 3rd party server.)  It also doesn't 
support ssh.

Thanks for listing them. Fortunately they're not important limitations 
for our current network setup.


 > Before the lfs serving functionality was added the Mercurial, I was 
using a couple of 3rd party packages that implemented lfs for the 
server.  Most of the issues I had were around the server trying to be 
clever with the User-Agent string it was sent, but I think it ended up 
working with the two I tried.  I since moved off to the native Mercurial 
server, so IDK what the state of things is.  The intention is that it 
should be interoperable, so if you find something that doesn't work, 
file a bug.  (Though it's been awhile since I did LFS work, and the 
issues have generally been 3rd party, so no promises to fix it.)

I don't know very well how git-lfs works. It seems you need a dedicated 
server process that deals with the protocol and (I suppose) mediates 
with the underlying git repository. Or it can run independently by git?

Just to be sure to have understood correctly, the current version of 
"hgweb" can be used as a remote git-lfs endpoint, and so it can be used 
directly with the "lfs" extension on the clients? Or we need other 
specialized software on server to run the tests?


 > I didn't try any file locking (and it's not implemented in Mercurial 
client), but maybe if you've already got a 3rd party server and can send 
it the right commands to lock it on the server, it will (mostly) work? I 
didn't look at any of the lfs extensions or proposals when working on 
the server, because I needed the bare bones implementation.

So the "lfs" extension could work with the current git-lfs v2 reference 
implementation (https://git-lfs.github.com)?

I see the locking API is documented here: 
https://github.com/git-lfs/git-lfs/blob/master/docs/api/locking.md

If I have understood, an hypothetical client "lfs-lock" extension on 
Mercurial could use that API to manage locks on server (if supported by 
the server of course). The server should then enforce the locks on push.


Thanks for all the shared info!
     Daniele.