D8051: worker: Use buffered input from the pickle stream

[yuja (Yuya Nishihara)]

yuja added a comment.


  >   for rfd, wfd in pipes:
  >       os.close(wfd)
  >
  > - selector.register(os.fdopen(rfd, 'rb', 0), selectors.EVENT_READ)
  >
  > +        selector.register(os.fdopen(rfd, 'rb'), selectors.EVENT_READ)
  
  Using buffered I/O can cause a deadlock (until the worker process exits.)
  The master process expects EVENT_READ will be asserted (i.e. level-triggered)
  if there are more than one readable items, but buffered file won't
  since almost all readable items will be moved to its internal buffer.
  
    import time
    from mercurial import (
        ui as uimod,
        worker,
    )
    
    def some_work(n):
        # send back many items at once
        for i in range(10):
            yield (n, i)
        # and don't close() the pipe for a while
        time.sleep(10)
    
    ui = uimod.ui()
    ui.setconfig(b'worker', b'numcpus', b'2')
    gen = worker._posixworker(ui, some_work, staticargs=(), args=[0, 1],
                              hasretval=False)
    for x in gen:
        ui.write(b'%r\n' % (x,))
        ui.flush()

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8051/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8051

To: heftig, #hg-reviewers
Cc: yuja, sheehan, mercurial-devel