Completely baffled

[Liam Routt]

Perhaps this is for another day, but...

In trying to convert my existing repositories, I've confirmed (from my 
perspective) another issue with largefiles we have suspected has been 
happening.

I have run hg convert on a repository, but the new repository does not 
have all of what were the largefiles in the original repository, ie. my 
converted repository does not have all the files I expect to be in the 
repository it came from.

We've seen situations where an update has failed up grab some of the 
latest largefiles for users. I suspect that this is another example of 
the same thing, just that the convert process was equally unable to grab 
files which is should have been putting into the converted repository. 
Note that no messages indicated any problem with the conversion process 
at all.

Obviously in order to properly address this issue someone is going to 
want a test case to work with, and I'm not sure that I can pass on my 
repository, nor do I have any confidence I know how to reproduce this. 
I'll see what I can manage next week.

Thanks again for the help!

Take care,

Liam Routt
Media Saints

On 6/12/2011 11:10 AM, Liam Routt wrote:
> Can I say how pleased I was to find this morning (my time) that others
> had chased this down for me, and that the issue is being progressed.
> Thanks to all involved.
>
> I'm new to Mercurial (having fled another DVCS which had a game-breaking
> flaw in it), so I'm somewhat distressed that what seems to be a fairly
> straightforward use case somehow seems to have slipped through the
> cracks. Not being able to do an update without losing local changes to
> largefiles (assuming that I understand the simple test case) would seem
> to be a huge failing of basic functionality that should have been caught
> before the feature was released to the public, I would have thought. But
> I am conscious that it is REALLY EASY to be critical from the outside,
> and I am not interested at belaboring the point (indeed I'm sorry that
> I'm mentioning it). At this point I intend to stick with Mercurial, for
> what it is worth.
>
>
> In the short-term I have to work out what to do for my project. Today
> all our work is being manually copied and merged into a known good
> directory, so that we can get through and make our delivery.
>
> Beyond that, I am considering re-converting the repositories to remove
> the largefiles support. I am expecting that the only penalty we will pay
> for doing that is that we won't get the repository size savings which
> largefiles was helping us with, but otherwise we should be able to just
> work with basic Mercurial. And then when we feel the largefiles are
> going to handle our use case we should be able to go back to using them
> by lgconverting the repositories again.
>
> I assume that is a sane plan of attack?
>
> Take care,
>
> Liam
>
> On 6/12/2011 7:09 AM, Na'Tosha Bard wrote:
>> 2011/12/5 Mads Kiilerich <mads at kiilerich.com <mailto:mads at kiilerich.com>>
>>
>> On 12/05/2011 06:59 PM, Haszlakiewicz, Eric wrote:
>>
>> -----Original Message-----
>> From: mercurial-bounces at selenic.com
>> <mailto:mercurial-bounces at selenic.com> [mailto:mercurial-
>> <mailto:mercurial->
>>
>> 2011/12/5 Liam Routt<liam.routt at mediasaints.__com
>> <mailto:liam.routt at mediasaints.com>>
>>
>> Just as a followup, we managed to lose the last
>> two days of work
>> by one of our developers today with Mercurial, which
>> really hurts.
>>
>>
>> My team uses Largefiles heavily and we have not seen this.
>> Are you
>>
>>
>> I think I was able to reproduce this, here's a minimal set of
>> commands that causes the modified large file to no longer be
>> modified:
>>
>>
>> Yes, there is a bug here. A simpler test case:
>>
>> $ hg init foo
>> $ cd foo
>> $ echo "[extensions]" >> .hg/hgrc
>> $ echo "largefiles=" >> .hg/hgrc
>>
>> $ touch thisfileislarge normal
>> $ hg add --large thisfileislarge
>> $ hg add normal
>> $ hg commit -m0
>>
>> $ hg tag t
>>
>> $ hg up -qr 0
>>
>> $ echo BLUE > thisfileislarge
>> $ echo blue > normal
>> $ hg stat
>> M normal
>> M thisfileislarge
>>
>> $ hg up
>>
>> 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
>> getting changed largefiles
>> 0 largefiles updated, 0 removed
>>
>> $ hg stat # should also show 'M thisfileislarge'
>> M normal
>> $ cat thisfileislarge
>> BLUE
>>
>> The problem seems to be related to the synchronization between the
>> working directory, the old standin in .hglf and the (potentially)
>> new standin.
>>
>> The test behaviour is not stable - run the tests several times
>> before you trust the output.
>>
>> It was unreliable behaviour like this that tricked me into
>> introducing a stupid test in e89385e4ef8d.
>>
>> I think something fishy is going on with all the invocations of
>> lfdirstate.normal (for example in openlfdirstate) and how we can't
>> mark a file as normal within the second it was updated, but I
>> haven't figured out what the intention with the code is and I thus
>> can't tell if it works correctly.
>>
>>
>> Hi Mads,
>>
>> Can you file a bug and assign it to me so that I don't forget? I'm
>> currently working on speeding up the speed of "hg status" with
>> largefiles, but this would be the next thing on my list. and, based on
>> my knowledge of largefiles, I don't think it will be hard to fix.
>>
>> Cheers,
>> Na'Tosha
>>
>>
>> --
>> *Na'Tosha Bard*
>> Build & Infrastructure Developer | Unity Technologies
>>
>> *E-Mail:* natosha at unity3d.com <mailto:natosha at unity3d.com>
>> *Skype:* natosha.bard
>>