BUG/MEDIUM: compression: use a trash chunk as large as the buffer for the output
Since commit
a0b37fb47 ("BUG/MEDIUM: compression: fix incompatibility
between compression and large-buffers"), slz_encode() writes directly into
the output chunk passed by the compression filter, which is the global
trash, sized like regular buffers. But with large buffers, a DATA block may
be larger than this trash. The input is then cut to the trash size by the
test added in commit
f82ace414 ("MEDIUM: compression: Be sure to never
compress more than a chunk at once"), leaving no room for the deflater's
overhead. On incompressible data, the first call emits the input plus 5
bytes for the stored block plus the 10 bytes of the gzip header, which
overflows the trash:
FATAL: bug condition "b->data + count > b->size" matched at include/haproxy/buf.h:395
#0 b_add (b=..., count=32783) at include/haproxy/buf.h:395
#1 comp_rfc195x_add_data (..., in_len=32768, out=...) at src/compression.c:404
This was seen in production on a PDF served with 32 kB buffers and
"http-response wait-for-body use-large-buffer", where the body formed a
37 kB DATA block. Without DEBUG_STRICT=2, the overflow goes unnoticed.
The output replaces the input in the HTX, so the right size for the output
chunk is the one of the buffer it comes from. It is now obtained with
get_best_trash_chunk(), which returns a large trash chunk for large buffers
and the regular one otherwise. htx_compression_buffer_init() still ensures
that the HTX has the room for the deflater's overhead, so a DATA block
plus this overhead always fits in such a chunk. A side effect is that the
data of a large buffer are now compressed in a single call, as intended by
the use of large buffers for compression.
The reg-test compression/large_buffers.vtc reproduces the crash with
unpatched versions built with DEBUG_STRICT=2.
This must be backported wherever
a0b37fb47 is backported.