[PATCH 3 of 4 V2] chgcache: implement the background preloading thread

[Jun Wu]

Excerpts from Yuya Nishihara's message of 2017-02-28 01:02:23 +0900:
> On Wed, 22 Feb 2017 18:16:10 -0800, Jun Wu wrote:
> > # HG changeset patch
> > # User Jun Wu <quark at fb.com>
> > # Date 1487813979 28800
> > #      Wed Feb 22 17:39:39 2017 -0800
> > # Node ID 28571825744fb4f4b424385f55afa9484532ef43
> > # Parent  630457e88fa0485bce7822345ea640b5cdcb9b8e
> > # Available At https://bitbucket.org/quark-zju/hg-draft 
> > #              hg pull https://bitbucket.org/quark-zju/hg-draft  -r 28571825744f
> > chgcache: implement the background preloading thread
> > 
> > The background preloading thread is meant to used by chg master, and it sets
> > up the IPC channel.
> > 
> > diff --git a/mercurial/chgcache.py b/mercurial/chgcache.py
> > --- a/mercurial/chgcache.py
> > +++ b/mercurial/chgcache.py
> > @@ -13,4 +13,5 @@ import select
> >  import socket
> >  import termios
> > +import threading
> >  import time
> >  
> > @@ -87,2 +88,51 @@ def reportrepopaths():
> >          for path in _localrepopaths:
> >              _ipc.send('%s %s' % (now, path))
> > +
> > +# -- only used by the master --------------------------------------------------
> > +
> > +def _initipc():
> > +    """initialize and return the global IPC object"""
> > +    global _ipc
> > +    if _ipc is None:
> > +        _ipc = socketipc()
> > +    return _ipc
> > +
> > +class preloader(object):
> > +    """background thread doing the actual preloading work"""
> > +
> > +    def __init__(self):
> > +        self._ipc = _initipc()
> > +        self._shouldexit = threading.Event()
> > +        self._thread = None
> > +
> > +    def start(self):
> > +        if self._thread is not None:
> > +            self.stop()
> > +        self._thread = threading.Thread(target=self._preloadloop,
> > +                                        name='chgcache.preloader')
> > +        self._thread.daemon = True
> > +        self._thread.start()
> 
> If start() designed to be called more than once, _shouldexit should be cleared.

Practically, it's probably not called more than once.

> > +    def stop(self):
> > +        if self._thread is None:
> > +            return
> > +        self._shouldexit.set()
> > +        self._ipc.send('') # unblock (possibly blocking) recv()
> 
> Ok, that will work though I'm not a fan of it.
> 
> Perhaps a cleaner way is to move recv() to the main loop of the server,
> and deliver received messages via thread-safe queue + event. But that will
> need more codes.

That sounds over complicated. Maybe shm is a better approach (no atime
needed, no blocking / close issues, need extra code to write "atomically"
but that's not hard)

> > +    def _preloadloop(self, interval=0.5):
> > +        while not self._shouldexit.is_set():
> > +            try:
> > +                atimestr, path = self._ipc.recv().split(' ', 1)
> > +                atime = float(atimestr)
> 
> recv() will block the thread with no timeout, so the interval is useless?

Sorry - interval was left from the shm code. It should be removed.