DEV: sse-latency: add a latency measuring tool for SSE response streams

DEV: sse-latency: add a latency measuring tool for SSE response streams

Common HTTP load generation tools are not convenient to measure latency
through haproxy when it comes to server-sent events, because the
measurements do not apply to the partial contents that SSE are. These
are particularly important for AI environments where agents need to
minimize latency.

This commit introduces a new tool that is both a client and a server,
and that is meant to measure an intermediary's added latency. For this,
it connects to itself to measure its own inflicted latency (loopback
test) then does it again through the configured proxy, and reports both
as well as the traversal time. The principle is to note the timestamp
at which each SSE is sent, and measure the difference when received by
the client, since both sides are the same process and use the same clock
reference. The output shows the reference time (loopback), the gateway
time, and the gateway overhead as the difference between the two.

The tool supports adding more clients, doing more requests, delaying the
first token response, and setting the inter-token latency.

An example haproxy config is provided (gateway.cfg) that measures both
with and without option http-no-delay.

Without http-no-delay (+ ~100ms, max 200 due to Nagle):

  === loopback (direct to listen address): 1 requests, 1 ok, 0 failed, 200 tokens ===
  TTFT (client)            n=1  mean=0.04ms  min=0.04ms  p50=0.04ms  p95=0.04ms  p99=0.04ms  max=0.04ms
  TTFT emit->recv(1st)     n=1  mean=0.00ms  min=0.00ms  p50=0.00ms  p95=0.00ms  p99=0.00ms  max=0.00ms
  inter-token (client)     n=199  mean=5.00ms  min=4.07ms  p50=5.13ms  p95=5.33ms  p99=5.36ms  max=5.38ms
  emit->recv (tokens)      n=199  mean=0.04ms  min=0.01ms  p50=0.03ms  p95=0.07ms  p99=0.18ms  max=0.19ms

  === gateway: 1 requests, 1 ok, 0 failed, 1000 tokens ===
  TTFT (client)            n=1  mean=207.17ms  min=207.17ms  p50=207.17ms  p95=207.17ms  p99=207.17ms  max=207.17ms
  TTFT emit->recv(1st)     n=1  mean=206.86ms  min=206.86ms  p50=206.86ms  p95=206.86ms  p99=206.86ms  max=206.86ms
  inter-token (client)     n=999  mean=4.80ms  min=0.00ms  p50=0.00ms  p95=0.00ms  p99=207.98ms  max=208.97ms
  emit->recv (tokens)      n=999  mean=104.04ms  min=1.02ms  p50=103.88ms  p95=197.67ms  p99=205.89ms  max=208.07ms

  === gateway overhead (gateway minus loopback; negative = loopback noise) ===
  emit->recv (1st token)   mean +206.851ms  p50 +206.851ms  p95 +206.851ms  p99 +206.851ms
  emit->recv (tokens)      mean +104.002ms  p50 +103.851ms  p95 +197.599ms  p99 +205.709ms
  TTFT (client)            mean +207.122ms  p50 +207.122ms  p95 +207.122ms  p99 +207.122ms
  inter-token (client)     mean -0.205ms  p50 -5.128ms  p95 -5.331ms  p99 +202.624ms

With http-no-delay (+ ~40 microseconds max):

  === loopback (direct to listen address): 1 requests, 1 ok, 0 failed, 200 tokens ===
  TTFT (client)            n=1  mean=0.05ms  min=0.05ms  p50=0.05ms  p95=0.05ms  p99=0.05ms  max=0.05ms
  TTFT emit->recv(1st)     n=1  mean=0.01ms  min=0.01ms  p50=0.01ms  p95=0.01ms  p99=0.01ms  max=0.01ms
  inter-token (client)     n=199  mean=5.00ms  min=4.03ms  p50=5.06ms  p95=5.27ms  p99=5.34ms  max=5.59ms
  emit->recv (tokens)      n=199  mean=0.04ms  min=0.01ms  p50=0.03ms  p95=0.07ms  p99=0.18ms  max=0.49ms

  === gateway: 1 requests, 1 ok, 0 failed, 1000 tokens ===
  TTFT (client)            n=1  mean=0.18ms  min=0.18ms  p50=0.18ms  p95=0.18ms  p99=0.18ms  max=0.18ms
  TTFT emit->recv(1st)     n=1  mean=0.03ms  min=0.03ms  p50=0.03ms  p95=0.03ms  p99=0.03ms  max=0.03ms
  inter-token (client)     n=999  mean=5.00ms  min=4.04ms  p50=5.10ms  p95=5.31ms  p99=5.37ms  max=5.45ms
  emit->recv (tokens)      n=999  mean=0.07ms  min=0.03ms  p50=0.07ms  p95=0.13ms  p99=0.20ms  max=0.28ms

  === gateway overhead (gateway minus loopback; negative = loopback noise) ===
  emit->recv (1st token)   mean +0.026ms  p50 +0.026ms  p95 +0.026ms  p99 +0.026ms
  emit->recv (tokens)      mean +0.034ms  p50 +0.036ms  p95 +0.059ms  p99 +0.029ms
  TTFT (client)            mean +0.135ms  p50 +0.135ms  p95 +0.135ms  p99 +0.135ms
  inter-token (client)     mean -0.002ms  p50 +0.042ms  p95 +0.041ms  p99 +0.035ms