BUG/MINOR: ssl: don't pass the address of a size_t as an int* to OpenSSL
aes_process() passes &out->data, which is a size_t, to the EVP update and
final routines through an (int*) cast. OpenSSL writes an int there, so on
a 64-bit big-endian platform (s390x, ppc64) the resulting length lands in
the high half of the size_t, and the buffer's data count becomes the real
length shifted left by 32 bits. The subsequent write at out->area +
out->data then targets a wild address.
Concretely, on such builds the "aes_gcm_dec" / "aes_cbc_dec" converters
and the JWE decryption paths corrupt memory as soon as they process any
input, so any client able to submit a token to them can crash the worker.
Little-endian builds, which are the vast majority, happen to work by
chance since the count starts at zero and only the low half is written.
Let's use a proper local int for the OpenSSL output length and assign it
back to the buffer's data count.
The shared helper comes from commit
f0e64de75 ("MINOR: ssl: Factorize AES
GCM data processing") in 3.4 but the same cast existed in the AES
converters before that. It must be backported to all stable versions.
Reported-by: Claude (ANT-2026-74N8W6SJ)