In 3.5-dev6, commit
1a99a7c0f4 ("MINOR: flt-http-comp: Check request
headers for response compression earlier") was needed to make sure
that processing of the accept-encoding request header happened in the
suitable order when both compression offload and decompression are
used at the same time. While theoretically it's no longer needed now
with the updated filter sequences merged into -dev9, it still uncovered
an interesting vestigal construct that we could carbon-14 date to 2014
in 1.5-dev24, precisely with this commit:
70730dddbd ("MEDIUM: http:
enable analysers to have keep-alive on stats").
By then, applets were not yet autonomous objects outside of the stream,
but were entirely part of the stream and were not much compatible with
analyzer processing, with a few exceptions, and the stream (by then
called a session) was reset to a pre-parsing state after each response
to process the next one, resulting in great difficulties explaining
why only select analyzers were kept and the other ones dropped.
In 3.5 all of this is old history and it's a bit surprising and even
refreshing to find living witnesses of this antique era, having managed
to both adapt to and resist natural evolution.
What happened with the 3.5-dev6 patch above is that function
select_compression_request_header() is no longer called by the
compression code during requests, and is expected to be called by the
rest of the natural request processing flow. But the antique code that
only keeps a subset of analyzers didn't care about ANA_REQ_HTTP_INNER
by then since applets didn't need that, and since it's the only place
which can still make sure that this function is called, the result
is that the accept-encoding header is no longer inspected for applets,
because either it was a frontend and the flag was already missing, or
it was a backend and that flag which was set for the backend was reset
by that code. Because of this, compression of applet responses (stats,
cache, promex, Lua services) is silently disabled since 3.5-dev6.
Upon careful inspecting, it turns out that all flags that were added
by the antique code (AN_REQ_FLT_HTTP_HDRS, AN_REQ_HTTP_XFER_BODY etc)
are all properly set by http_process_common() which is called on
AN_REQ_HTTP_INNER. Thus the fix consists in removing all that old
code to stop purging flags, and only enabling AN_REQ_HTTP_INNER when
applets are branched from frontends, so that the flag is always set,
and that the rest of the processing is properly performed in both
cases.
With this patch, sending curl --compressed to a config with only
a frontend or only a backend containing "stats uri /" properly
returns compressed contents again.
No backport is needed since it only affects 3.5-dev6 (3.4 and older
still work fine thanks to the code that the patch above dropped).
Incidentally, regtest legacy-implicit-order.vtc needed to be fixed
because it was written after the change above, when the cache's
output wasn't compressed by the existing rules. Now with the fix,
the cache's output gets compressed again, so adding a separate
proxy for this allows to distinguish the request processing.
http-response cache-store my_cache
http-response set-header X-Hit %[res.cache_hit]
server s1 ${s1_addr}:${s1_port}
+
+ listen li2
+ bind "fd@${li2}"
+ # Explicit filter mixed with implicitly declared core filters:
+ # previously a configuration error, now automatically ordered with
+ # the cache filter before the trace filter and the compression
+ # filter after it.
+ filter trace quiet
+ http-request cache-use my_cache
+ http-response cache-store my_cache
+ http-response set-header X-Hit %[res.cache_hit]
+ server s1 ${s1_addr}:${s1_port}
} -start
# First request: the response is stored in the cache then compressed for the
# Second request: served from the cache. A cache hit is only possible if the
# uncompressed body was stored, proving the cache filter was evaluated
# before the compression filter. And the body is still delivered
-# uncompressed, confirming the stored body was not the compressed one.
+# compressed, because it was recompressed by the compression rules when
+# leaving the cache.
client c2 -connect ${h1_li1_sock} {
+ txreq -url /a -hdr "Accept-Encoding: gzip"
+ rxresp
+ expect resp.status == 200
+ expect resp.http.x-hit == 1
+ expect resp.http.content-encoding == gzip
+ gunzip
+} -run
+
+# Third request: served from the cache. A cache hit is only possible if the
+# uncompressed body was stored, proving the cache filter was evaluated
+# before the compression filter. Here we request via a distinct proxy that
+# does not compress but shares the same cache. By retrieving an uncompressed
+# cached object, it will prove that the cached body was indeed uncompressed.
+client c3 -connect ${h1_li2_sock} {
txreq -url /a -hdr "Accept-Encoding: gzip"
rxresp
expect resp.status == 200
expect resp.http.x-hit == 1
expect resp.bodylen == 500
+ expect resp.http.content-encoding == <undef>
} -run
s->flags |= SF_ERR_LOCAL; // to mark that it comes from the proxy
http_set_term_flags(s);
- if (HAS_FILTERS(s))
- req->analysers |= AN_REQ_FLT_HTTP_HDRS;
-
- /* enable the minimally required analyzers to handle keep-alive and compression on the HTTP response */
- req->analysers &= (AN_REQ_HTTP_BODY | AN_REQ_FLT_HTTP_HDRS | AN_REQ_FLT_END);
- req->analysers &= ~AN_REQ_FLT_XFER_DATA;
- req->analysers |= AN_REQ_HTTP_XFER_BODY;
+ /* inner is usually set only during backend processing, and is
+ * also needed for applets branched from frontends in order to
+ * enable compression, cookie processing, unique-id generation,
+ * or whatever occasional feature they might rely on.
+ */
+ if (s->be == sess->fe)
+ req->analysers |= AN_REQ_HTTP_INNER;
s->scb->flags |= SC_FL_SND_ASAP;
s->flags |= SF_ASSIGNED;
projects / haproxy.git / commitdiff