OPTIM: flt-http-comp: only flush the compressor after the last DATA block

comp_http_payload() used to call the compressor's flush() after each DATA
block, which emits a synchronization point (an EOB and an empty stored
block, up to 5 bytes) and closes the current huffman block each time.
Worse, with slz, and before recent commit a0b37fb470 ("BUG/MEDIUM:
compression: fix incompatibility between compression and large-buffers"),
add_data() only queued the data and flush() is where they were compressed,
so each block was compressed on its own, without being able to reference
the previous ones, since slz keeps no context between calls.

Now flush() is only called when no other DATA block follows in this
pass, so that the client still gets everything while we're waiting for
more data. With zlib, which compresses in add_data(), the last bits are
simply kept for the next block. With slz, all DATA blocks of the pass
are queued (the first one by reference, the following ones copied into
a contiguous buffer, as add_data() already does) and compressed at once
with their whole context. The first block's data are still referenced
after the block is removed or emptied, which is fine since this does not
touch the data area, and they are copied when the second block is added.

The difference is only visible when several DATA blocks are processed
in the same pass, e.g. a 1 MB HTML response made of 256-byte chunks
shrinks from 405057 to 402030 bytes (-0.7%) at level 1.

diff --git a/src/flt_http_comp.c b/src/flt_http_comp.c
index 07012b9794fb96cc33dbe2d19c7c6c52cc1404af..3a3a6c60da833138e27515191d7d7d20157013bb 100644 (file)
--- a/src/flt_http_comp.c
+++ b/src/flt_http_comp.c
@@ -291,7 +291,7 @@ comp_http_payload(struct stream *s, struct filter *filter, struct http_msg *msg,
        struct htx *htx = htxbuf(&msg->chn->buf);
        struct htx_ret htxret = htx_find_offset(htx, offset);
        struct htx_blk *blk, *next;
-       int ret, consumed = 0, to_forward = 0, last = 0;
+       int ret, consumed = 0, to_forward = 0, last = 0, flush;
 
        blk = htxret.blk;
        offset = htxret.ret;
@@ -325,8 +325,20 @@ comp_http_payload(struct stream *s, struct filter *filter, struct http_msg *msg,
                                        v.len = b_size(&trash);
                                }
 
+                               /* The output only needs to be flushed when no other
+                                * DATA block follows in this pass, so that the
+                                * client gets everything while we wait for more.
+                                * Otherwise zlib keeps the last bits for the next
+                                * block, and slz queues the data so that all blocks
+                                * of the pass are compressed at once with their
+                                * whole context. Both save a synchronization point
+                                * per block, which costs up to 5 bytes and closes
+                                * the current huffman block.
+                                */
+                               flush = last || v.len == len || !next || htx_get_blk_type(next) != HTX_BLK_DATA;
+
                                ret = htx_compression_buffer_add_data(st, v.ptr, v.len, &trash, dir);
-                               if (ret < 0 || htx_compression_buffer_end(st, &trash, last, dir) < 0)
+                               if (ret < 0 || (flush && htx_compression_buffer_end(st, &trash, last, dir) < 0))
                                        goto error;
                                BUG_ON(v.len != ret);