BUG/MINOR: intops: reject varints that do not fit in 64 bits in decode_varint()
decode_varint() accepts any number of continuation bytes. From the 11th
byte on, the shift count exceeds 63, which is undefined behaviour (UBSan:
"shift exponent 67 is too large for 64-bit type"), and a 10-byte varint
whose last byte is above 15 silently wraps: a value >= 2^64 is decoded
modulo 2^64. For example an SPOP agent sending the AGENT-HELLO
max-frame-size as f0 ff ff ff ff ff ff ff ff 7f has it decoded as
2314921131297212640 instead of being rejected.
In order to avoid adding costly checks in the inner loop, let's only
process the first 9 bytes as usual in the current loop and make a
special case of the 10th byte where we add an overflow check if ever
met.
This should be backported to all stable versions.
Reported-by: Martin Mirchev <mirchevmartin2203@gmail.com>
Co-authored-by: Martin Mirchev <mirchevmartin2203@gmail.com>