Tag caching, at last

[Matt Mackall]

On Mon, 2009-07-13 at 21:11 -0400, Greg Ward wrote:
> On Mon, Jul 13, 2009 at 5:00 PM, Matt Mackall<mpm at selenic.com> wrote:
> > Which is why I think strip and friends should simply shoot the cache for
> > now, until we can figure out how to be more clever.
> 
> I'm worried about the effect that will have on MQ users.  If I'm
> qpushing and qpopping on top of a 100-head repo, then I'm also going
> to run "hg tip" quite frequently.  And if that takes 10 sec after a
> qpop or qrefresh, that would suck.
> 
> Now, if "shoot the cache" really means "kindly inform the cache that
> head X is being destroyed", that probably wouldn't be so bad.  But
> that smacks of cleverness, and I'm getting very leery of clever tag
> caching.
> 
> > Even though I've said I don't want version numbers and that it's ok in
> > the future to just switch to a new format/filename, I -do- want to have
> > the format more or less pinned down. Even if you don't actually -use-
> > the tags you write out for now, I'd rather write them out so we don't
> > have to burn a disk format when we do figure out how to use them. Does
> > that make sense?
> 
> Makes total sense.  But I don't see how caching tags in the form of
> "hg tags" output helps.  We've discussed strip/rollback to death, but
> they are only part of the problem.
> 
> The final straw for me was tag rank.  If we only cache (say) the
> output of "hg tags", then the info needed to compute tag rank is gone.
>  The only place to find that info is in the .hgtags file on each head.
>  So I think caching tag info only works if we cache the same info
> that's in .hgtags on every head... in which case, why not just read
> .hgtags directly, since we've already avoided the expensive part and
> cached all those fnodes.

Uh, why do we care about rank? 

Let's back up a bit. There are three speeds:

slow - no caching, have to do heads() and visit all manifests
fast - cache has all or almost all .hgtags pointers needed
instant - no need to parse any .hgtags revisions

You've gotten the fast level working, yes? That means there's no reason
that we should ever need to be slower than fast. That means strip and
rollback can always be -at least- fast. Let's put those aside, because
fast is good enough. 

Now, the question is: can we easily design for some things (not ALL
things, but SOME things) to be instant? The answer is yes.
Those some things are tag lookup for tags/parents/id/log/hgweb/etc when
NOTHING has changed in history since the last lookup. Taking 0 seconds
rather than 1 second here is a big win because these commands are so
common.

We do that by:

- taking your fast solution
- writing out cache of final tag to node mapping
- if NOTHING has changed on read by (tip revision/hash match), return
cached tag list

Super simple. Now, you might say: but nasty commands like
strip/rollback/etc might be able to somehow create a situation where
tip's revision and hash are unchanged but something else in history
changed. But that's actually very easy to deal with: just have strip
invalidate the cache... by simply reading current tags when it's done,
thereby causing the cache to be updated! Again, this is still 'fast'
without any extra cleverness.

So if we do:

qpop   # strip updates tags, looks up one head ('fast')

tags   # instant

qpush  

tags   # tip changed, 'fast'

tags   # tip is current, instant

-- 
http://selenic.com : development and support for Mercurial and Linux