[PATCH STABLE] setdiscovery: limit the size of all sample (issue4411)

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David <pierre-yves.david at fb.com>
# Date 1414885973 0
#      Sat Nov 01 23:52:53 2014 +0000
# Branch stable
# Node ID e9b8dd716dcd3aede2c0781dcf14d8553a934ae1
# Parent  eb315418224c330a412538e13876e5cecff83da5
setdiscovery: limit the size of all sample (issue4411)

Further digging on this issue show that the limit on the sample size used in
discovery never works for heads. Here is a quote from the code itself:

  desiredlen = size - len(always)
  if desiredlen <= 0:
      # This could be bad if there are very many heads, all unknown to the
      # server. We're counting on long request support here.

The long request support never landed and evolution make the "very many heads,
all unknown to the server" case quite common.

We implement a simple and stupid hard limit of sample size for all query. This
should prevent HTTP 414 error with the current state of the code.

diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -201,19 +201,23 @@ def findcommonheads(ui, local, remote,
             break
 
         if full:
             ui.note(_("sampling from both directions\n"))
             sample = _takefullsample(dag, undecided, size=fullsamplesize)
+            targetsize = fullsamplesize
         elif common:
             # use cheapish initial sample
             ui.debug("taking initial sample\n")
             sample = _takefullsample(dag, undecided, size=fullsamplesize)
+            targetsize = fullsamplesize
         else:
             # use even cheaper initial sample
             ui.debug("taking quick initial sample\n")
             sample = _takequicksample(dag, undecided, size=initialsamplesize,
                                       initial=True)
+            targetsize = initialsamplesize
+        sample = _limitsample(sample, targetsize)
 
         roundtrips += 1
         ui.progress(_('searching'), roundtrips, unit=_('queries'))
         ui.debug("query %i; still undecided: %i, sample size is: %i\n"
                  % (roundtrips, len(undecided), len(sample)))
diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
--- a/tests/test-setdiscovery.t
+++ b/tests/test-setdiscovery.t
@@ -309,21 +309,27 @@ One with >200 heads, which used to use u
   comparing with b
   query 1; heads
   searching for changes
   taking quick initial sample
   searching: 2 queries
-  query 2; still undecided: 1240, sample size is: 260
+  query 2; still undecided: 1240, sample size is: 100
   sampling from both directions
   searching: 3 queries
-  query 3; still undecided: 980, sample size is: 260
+  query 3; still undecided: 1140, sample size is: 200
   sampling from both directions
   searching: 4 queries
-  query 4; still undecided: 720, sample size is: 260
+  query 4; still undecided: 940, sample size is: 200
   sampling from both directions
   searching: 5 queries
-  query 5; still undecided: 460, sample size is: 200
-  5 total queries
+  query 5; still undecided: 740, sample size is: 200
+  sampling from both directions
+  searching: 6 queries
+  query 6; still undecided: 540, sample size is: 200
+  sampling from both directions
+  searching: 7 queries
+  query 7; still undecided: 44, sample size is: 44
+  7 total queries
   common heads: 3ee37d65064a
 
 Test actual protocol when pulling one new head in addition to common heads
 
   $ hg clone -U b c