MEDIUM: sched: fix the default runqueue-depth

The scheduler's run queue depth had been empirically limited to around
280/sqrt(nbhread) when the scheduler was shared between all threads and
the number of threads was very low (4-16), because by then it looked
like a good balance between contention and latency.

Nowadays, each thread has its own scheduler and the number of threads
has exploded, but this calculus remains in place. It causes a noticeable
scheduling overhead on large systems because the number of tasks each
scheduler handles at once can become ridiculously low (e.g. 14 for 256
threads), forcing some users to work around this by forcing the value
in the configuration.

Let's get rid of this outdated calculation and keep the fixed default
value of 280 that was once considered correct and that is still applied
to a single thread, since each scheduler continues to run on a single
thread. Nothing prevents changing the default again later but at least
now the value in defaults.h corresponds to the one we're running on by
default.

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 77a1e4f77c2f98cff80432c3dd5ac950ec7b38d7..418379ef704bbf12b74e2b2b10ab41ceba578739 100644 (file)
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -5787,13 +5787,11 @@ tune.ring.queues <number>
 
 tune.runqueue-depth <number>
   Sets the maximum amount of task that can be processed at once when running
-  tasks. The default value depends on the number of threads but sits between 35
-  and 280, which tend to show the highest request rates and lowest latencies.
-  Increasing it may incur latency when dealing with I/Os, making it too small
-  can incur extra overhead. Higher thread counts benefit from lower values.
-  When experimenting with much larger values, it may be useful to also enable
-  tune.sched.low-latency and possibly tune.fd.edge-triggered to limit the
-  maximum latency to the lowest possible.
+  tasks. The default value is 280, which tends to show high request rates and
+  low latencies. Increasing it may incur latency when dealing with I/Os, making
+  it too small can incur extra overhead. When experimenting with much larger
+  values, it may be useful to also enable tune.sched.low-latency and possibly
+  tune.fd.edge-triggered to limit the maximum latency to the lowest possible.
 
 tune.sched.low-latency { on | off }
   Enables ('on') or disables ('off') the low-latency task scheduler. By default

diff --git a/include/haproxy/defaults.h b/include/haproxy/defaults.h
index 2b1fed3d1af9261eb3d86aec040d15a2d491890d..29bf2f3c08c87bca9a7e78440d5cd148a9ead0fc 100644 (file)
--- a/include/haproxy/defaults.h
+++ b/include/haproxy/defaults.h
@@ -255,9 +255,7 @@
 #endif
 
 // The base max number of tasks to run at once to be used when not set by
-// tune.runqueue-depth. It will automatically be divided by the square root
-// of the number of threads for better fairness. As such, 64 threads will
-// use 35 and a single thread will use 280.
+// tune.runqueue-depth.
 #ifndef RUNQUEUE_DEPTH
 #define RUNQUEUE_DEPTH 280
 #endif

diff --git a/src/haproxy.c b/src/haproxy.c
index 046b289a134cb7136b019655259838ef2dd8b93d..50584f415d63e20aea442401a854b381eba571d3 100644 (file)
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -2484,14 +2484,8 @@ static void step_init_2(int argc, char** argv)
        if (global.tune.max_rules_at_once <= 0)
                global.tune.max_rules_at_once = MAX_RULES_AT_ONCE;
 
-       if (global.tune.runqueue_depth <= 0) {
-               /* tests on various thread counts from 1 to 64 have shown an
-                * optimal queue depth following roughly 1/sqrt(threads).
-                */
-               int s = my_flsl(global.nbthread);
-               s += (global.nbthread / s); // roughly twice the sqrt.
-               global.tune.runqueue_depth = RUNQUEUE_DEPTH * 2 / s;
-       }
+       if (global.tune.runqueue_depth <= 0)
+               global.tune.runqueue_depth = RUNQUEUE_DEPTH;
 
        if (global.tune.recv_enough == 0)
                global.tune.recv_enough = MIN_RECV_AT_ONCE_ENOUGH;