diff options
| author | Tom Lane <tgl@sss.pgh.pa.us> | 2023-03-31 16:29:55 -0400 |
|---|---|---|
| committer | Tom Lane <tgl@sss.pgh.pa.us> | 2023-03-31 16:29:55 -0400 |
| commit | a2a0c7c29e47f39da905577659e66b0086b769cc (patch) | |
| tree | 9c8eb5a7f98d349edd8c60eb43984bc751c9d8e7 /src/backend/utils/adt/numeric.c | |
| parent | f0d65c0eaf05d6acd3ae05cde4a31465eb3992b2 (diff) | |
| download | postgresql-a2a0c7c29e47f39da905577659e66b0086b769cc.tar.gz postgresql-a2a0c7c29e47f39da905577659e66b0086b769cc.zip | |
Further tweaking of width_bucket() edge cases.
I realized that the third overflow case I posited in commit b0e9e4d76
actually should be handled in a different way: rather than tolerating
the idea that the quotient could round to 1, we should clamp so that
the output cannot be more than "count" when we know that the operand is
less than bound2. That being the case, we don't need an overflow-aware
increment in that code path, which leads me to revert the movement of
the pg_add_s32_overflow() call. (The diff in width_bucket_float8
might be easier to read by comparing against b0e9e4d76^.)
What's more, width_bucket_numeric also has this problem of the quotient
potentially rounding to 1, so add a clamp there too.
As before, I'm not quite convinced that a back-patch is warranted.
Discussion: https://postgr.es/m/391415.1680268470@sss.pgh.pa.us
Diffstat (limited to 'src/backend/utils/adt/numeric.c')
| -rw-r--r-- | src/backend/utils/adt/numeric.c | 17 |
1 files changed, 14 insertions, 3 deletions
diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index a83feea3967..3c3184f15b5 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -1907,7 +1907,7 @@ width_bucket_numeric(PG_FUNCTION_ARGS) } /* - * If 'operand' is not outside the bucket range, determine the correct + * 'operand' is inside the bucket range, so determine the correct * bucket for it to go. The calculations performed by this function * are derived directly from the SQL2003 spec. Note however that we * multiply by count before dividing, to avoid unnecessary roundoff error. @@ -1940,8 +1940,19 @@ compute_bucket(Numeric operand, Numeric bound1, Numeric bound2, operand_var.dscale + count_var->dscale); div_var(&operand_var, &bound2_var, result_var, select_div_scale(&operand_var, &bound2_var), true); - add_var(result_var, &const_one, result_var); - floor_var(result_var, result_var); + + /* + * Roundoff in the division could give us a quotient exactly equal to + * "count", which is too large. Clamp so that we do not emit a result + * larger than "count". + */ + if (cmp_var(result_var, count_var) >= 0) + set_var_from_var(count_var, result_var); + else + { + add_var(result_var, &const_one, result_var); + floor_var(result_var, result_var); + } free_var(&bound1_var); free_var(&bound2_var); |