dirstate and improving the performance of hg-add

[Adrian Buehlmann]

(reordering the answer, as we usually don't top post)

On 2012-06-11 23:06, Joshua Redstone wrote:
> On 6/11/12 3:58 PM, "Adrian Buehlmann" <adrian at cadifra.com> wrote:
>> On 2012-06-11 21:08, Joshua Redstone wrote:
>>> Hi mercurial-devel,
>>> I've been looking into how to improve the performance of hg-add and
>>> wanted to get people's thoughts on removing dirstate._dirs and adding a
>>> sorted list of entries to dirstate to mirror the stuff in dirstate._map.
>>>
>>> Background:
>>>
>>> hg-add is fast for small repos, but for larger repos, we've been seeing
>>> the time to add a file grow to over 1.5 seconds ...

[..]

>>> A few observations:
>>>
>>> - the number of case-insensitive comparisons could be dramatically
>>> reduced by indexing into the dirstate rather than exhaustively iterating
>>> through the whole thing.  One way to do this is to keep a sorted list of
>>> entries and doing a binary search for the path being added.  It suffices
>>> to do a case-insensitive comparison with only the directory components
>>> of the entries in the list surrounded where the path would be added.
>>
>> Do you actually need the default value of the config setting
>> ui.portablefilenames?
>>
>> The default is "warn", which does that case-insensitive checking. But if
>> you don't need that warning, then setting it to "ignore" or "false"
>> should not do this case-insensitive comparison.
>>
> If we can afford it, it seems like it's nice to have that warning to
> maintain portability.  And, at least in principle, it seems like we should
> be able to afford it with ease.

You are sidestepping my question :-). The question is about whether the
default value of "warn" is actually relevant to your very demanding use
patterns.

So I assume:
(a) You want to keep that warning
(b) The 1.5 seconds was measured with ui.portablefilenames=True

Pure Linux shops hardly ever need that warning. And if you happend to
later change your mind, you can always rename files, making a revision
and it's children able to checkout on Windows. The names in the store
are encoded, so the store survives on all supported platforms, no matter
how silly your file names or your platforms used are.

That default value was the result of relatively short discussion (IIRC)
and I think I voiced then to have it on "ignore", but I was defeated. I
still think the default should be the fast setting. Even though I am a
Windows hacker.

IMHO, it suffices to check on checkout, if the platform is restricted
(that is, on Windows).

Also, I'm not arguing that your proposal should not be followed. I was
just curious about what your needs really are, and I suspect there is a
possible speed gain by giving up that warning in the settings.

I also think it is not a question of portability. The store is portable
across all platforms. It's only a specific revision that might cause
problems when you try to check it out on a specific platform.

It's also only a question about user interface, that's also probably why
that setting is in the ui config section. But having it at "warn" has a
cost.