diff options
author | Dean Rasheed <dean.a.rasheed@gmail.com> | 2021-10-06 13:20:23 +0100 |
---|---|---|
committer | Dean Rasheed <dean.a.rasheed@gmail.com> | 2021-10-06 13:20:23 +0100 |
commit | 9ab94ccb15af288f9ff295b4f04b28d86115b803 (patch) | |
tree | 4f3afc1877a2b0e52cd0ade6cf142ee4633a3d95 /src/backend/utils/adt/numeric.c | |
parent | d6d68e223379985661df45c728c32c3b6d462629 (diff) | |
download | postgresql-9ab94ccb15af288f9ff295b4f04b28d86115b803.tar.gz postgresql-9ab94ccb15af288f9ff295b4f04b28d86115b803.zip |
Fix corner-case loss of precision in numeric_power().
This fixes a loss of precision that occurs when the first input is
very close to 1, so that its logarithm is very small.
Formerly, during the initial low-precision calculation to estimate the
result weight, the logarithm was computed to a local rscale that was
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
as close as 1e-16383 to 1, hence its logarithm may be as small as
1e-16383, and so the local rscale needs to be allowed to exceed 16383,
otherwise all precision is lost, leading to a poor choice of rscale
for the full-precision calculation.
Fix this by removing the cap on the local rscale during the initial
low-precision calculation, as we already do in the full-precision
calculation. This doesn't change the fact that the initial calculation
is a low-precision approximation, computing the logarithm to around 8
significant digits, which is very fast, especially when the base is
very close to 1.
Patch by me, reviewed by Alvaro Herrera.
Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
Diffstat (limited to 'src/backend/utils/adt/numeric.c')
-rw-r--r-- | src/backend/utils/adt/numeric.c | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index 9f16f09c110..885d50f5fc3 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -9245,9 +9245,13 @@ power_var(const NumericVar *base, const NumericVar *exp, NumericVar *result) */ ln_dweight = estimate_ln_dweight(base); + /* + * Set the scale for the low-precision calculation, computing ln(base) to + * around 8 significant digits. Note that ln_dweight may be as small as + * -SHRT_MAX, so the scale may exceed NUMERIC_MAX_DISPLAY_SCALE here. + */ local_rscale = 8 - ln_dweight; local_rscale = Max(local_rscale, NUMERIC_MIN_DISPLAY_SCALE); - local_rscale = Min(local_rscale, NUMERIC_MAX_DISPLAY_SCALE); ln_var(base, &ln_base, local_rscale); |