MINOR: http-ana: automatically enable http-no-delay for streaming events (SSE)

Some people get trapped using plain HTTP in AI gateway environments which
rely on server-sent events (SSE, documented at
  https://html.spec.whatwg.org/multipage/server-sent-events.html).

This mechanism consists in pushing body updates and expecting intermediaries
not to buffer them. The way to detect this is to check the advertised
content-type in the response, which is "text/event-stream" in this case.

What happens by default is that the Nagle algorithm delays short messages
by up to ~200ms hoping to aggregate them. But when all messages are only
a few tens of bytes large, sent every 5-10ms, this shows a jerky traffic
that's not pleasant to use.

The current patch simply adds a check for the aforementioned content-type
on the response path, and automatically enables http-no-delay for sending
to the client when found. It's sufficient to address the case without
requiring the user to adjust the config, as measured with sse-latency:

Before:
  === gateway overhead (gateway minus loopback; negative = loopback noise) ===
  emit->recv (1st token)   mean +206.851ms  p50 +206.851ms  p95 +206.851ms  p99 +206.851ms
  emit->recv (tokens)      mean +104.002ms  p50 +103.851ms  p95 +197.599ms  p99 +205.709ms
  TTFT (client)            mean +207.122ms  p50 +207.122ms  p95 +207.122ms  p99 +207.122ms
  inter-token (client)     mean -0.205ms  p50 -5.128ms  p95 -5.331ms  p99 +202.624ms

After:
  === gateway overhead (gateway minus loopback; negative = loopback noise) ===
  emit->recv (1st token)   mean +0.026ms  p50 +0.026ms  p95 +0.026ms  p99 +0.026ms
  emit->recv (tokens)      mean +0.034ms  p50 +0.036ms  p95 +0.059ms  p99 +0.029ms
  TTFT (client)            mean +0.135ms  p50 +0.135ms  p95 +0.135ms  p99 +0.135ms
  inter-token (client)     mean -0.002ms  p50 +0.042ms  p95 +0.041ms  p99 +0.035ms

It would make sense to backport this to 3.4 to help users who forget to
adjust their config to handle the streamed events.

diff --git a/src/http_ana.c b/src/http_ana.c
index bf24821783b062f7012d34dfd681ae197b24fa08..397680595627260a689cdaca52ecb9e9ef8ec1b4 100644 (file)
--- a/src/http_ana.c
+++ b/src/http_ana.c
@@ -1666,6 +1666,17 @@ int http_wait_for_response(struct stream *s, struct channel *rep, int an_bit)
                txn->flags |= TX_CON_WANT_TUN;
        }
 
+       /* check if we need to switch to no-delay for SSE */
+       if (!(s->scf->flags & SC_FL_SND_NEVERWAIT)) {
+               struct http_hdr_ctx ctx = { .blk = NULL };
+
+               if (http_find_header(htx, ist("content-type"), &ctx, 0) &&
+                   isteqi(ctx.value, ist("text/event-stream"))) {
+                       s->scf->flags |= SC_FL_SND_NEVERWAIT;
+                       s->scb->flags |= SC_FL_SND_NEVERWAIT;
+               }
+       }
+
        /* Check for NTML authentication headers in 401 (WWW-Authenticate) and
         * 407 (Proxy-Authenticate) responses and set the connection to
         * private.