MINOR: flt-http-comp: size the compression reserve using SLZ_MAX_OVERHEAD()

htx_compression_buffer_init() reserves room for the deflater's output based
on what it would emit as stored blocks. libslz now publishes in slz.h the
room each call to slz_encode() needs beyond its input length,
SLZ_MAX_OVERHEAD(len), which includes the stored blocks, the end of the
stream, the bits queued by previous calls and the up to 40 bytes its fixed
huffman encoding may lag behind stored blocks. Let's use it so that the
reserve follows slz's own settings instead of a local estimate.

All the DATA blocks of a pass are queued by the slz backend and compressed
in a single call when the last one is flushed, so the call covers at most
the whole buffer. The reserve is thus based on the buffer size, which keeps
it constant (52 bytes for buffers up to 64 kB), plus 14 bytes for the finish
(queue, last block and gzip trailer, a flush needs 10) and 10 bytes for the
gzip header: 76 bytes for 16 kB buffers.

The previous computation is kept for zlib, which publishes no such bound,
and when linking with an older external libslz which doesn't define the
macro. In all cases the result of htx_replace_blk_value() remains checked
since commit 5ab337485 ("BUG/MEDIUM: flt-http-comp: check the block
returned by htx_replace_blk_value()"). Crafted bodies made of runs of
literals >= 144 separated by short references, as well as random data,
sent in chunks of 256 bytes to 64 kB, all decompress correctly.

This is not a bug fix, only a way to better tie haproxy to slz's settings,
and it must not be backported.

diff --git a/src/flt_http_comp.c b/src/flt_http_comp.c
index f76fe212ea05e5893454a7967abd8b66ca4104cc..71dffb859b2907d77bb87a3a0e7eb880f0e06196 100644 (file)
--- a/src/flt_http_comp.c
+++ b/src/flt_http_comp.c
@@ -807,12 +807,30 @@ select_compression_response_header(struct comp_state *st, struct stream *s, stru
 static int
 htx_compression_buffer_init(struct htx *htx, struct buffer *out)
 {
+#if defined(USE_SLZ) && defined(SLZ_MAX_OVERHEAD)
+       /* SLZ publishes the room each call needs beyond its input length,
+        * including the stored blocks and the lag of its huffman encoding. All
+        * the DATA blocks of a pass are queued and compressed in a single call
+        * when the last one is flushed, so the call covers at most the whole
+        * buffer, hence the use of its size, which keeps the reserve constant
+        * (52 bytes for buffers up to 64 kB). It is followed by the finish (up
+        * to 14 bytes for the queue, the last block and the gzip trailer) or a
+        * flush (10 bytes), and the gzip header (10 bytes) is sent with the
+        * first output.
+        */
+       if (htx_free_space(htx) < 24 + SLZ_MAX_OVERHEAD(htx->size))
+               return -1;
+#else
        /* output stream requires at least 10 bytes for the gzip header, plus
         * at least 8 bytes for the gzip trailer (crc+len), plus at most
-        * 5 bytes per 32kB block and 2 bytes to close the stream.
+        * 5 bytes per 32kB block and 2 bytes to close the stream. This only
+        * covers data sent as stored blocks, and no deflater is guaranteed to
+        * emit less than its input, so the result of htx_replace_blk_value()
+        * must still be checked.
         */
        if (htx_free_space(htx) < 20 + 5 * ((htx->data + 32767) >> 15))
                return -1;
+#endif
        b_reset(out);
        return 0;
 }