BUG/MEDIUM: log: encode the whole non-printable range in the +json output
The +json encoder only escaped the double quote and the backslash, which
left two problems behind. First, RFC 8259 #7 requires every character
below 0x20 to be escaped, so a control character coming from a request,
such as a TAB, made the record invalid JSON:
log-format "{\"hdr\":%{+json}[capture.req.hdr(0)]}"
$ jq . < record
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 1, column 8
Second, RFC 8259 #8.1 requires the text to be valid UTF-8, while a header
value is made of arbitrary bytes. A byte such as 0xAA, or a sequence such
as 0xC0 0x41, is perfectly legal on the wire and made the record
undecodable:
$ python3 -m json.tool < record
'utf-8' codec can't decode byte 0xaa in position 8: invalid start byte
In both cases a client can invalidate the log records of its own requests,
and possibly a whole batch depending on how the collector reacts to a
parse error.
Section 8.6 of the configuration manual makes it pretty clear that only
characters from 32 to 126 are emitted as-is, everything else being
encoded as "#" plus two hex digits, "#" itself included so that the
result is unambiguous. There is no reason for the json encoding to bypass
this rule especially when json decoders may face problems, so let's tag
that whole range in json_escape_map and escape it, using the two-character
form for the characters which have one and the six-character one for the
others.
This also protects readers from a possible misinterpretation of some
UTF-8 sequences (e.g. direction change) that could fool a reader on
screen if rendered as-is. Now it will be unabmbiguous since bytes will
be logged one at a time as is already done for the plain text logs.
This should be backported after some observation time to 3.0, where +json
was introduced by commit
3f7c8387c0 ("MINOR: log: add +json encoding
option").