aboutsummaryrefslogtreecommitdiff
path: root/src/backend/access/gist/gistbuildbuffers.c
diff options
context:
space:
mode:
authorTom Lane <tgl@sss.pgh.pa.us>2020-06-13 13:43:24 -0400
committerTom Lane <tgl@sss.pgh.pa.us>2020-06-13 13:43:40 -0400
commit33dd9bb3b0a88981f18a10d89720b4e40d8876ba (patch)
treea7420614e71b5c70ef907f4cdb46f8a68e1808cc /src/backend/access/gist/gistbuildbuffers.c
parente745bcc00149fe3c35ba1123800e0beb948e3678 (diff)
downloadpostgresql-33dd9bb3b0a88981f18a10d89720b4e40d8876ba.tar.gz
postgresql-33dd9bb3b0a88981f18a10d89720b4e40d8876ba.zip
Fix behavior of float aggregates for single Inf or NaN inputs.
When there is just one non-null input value, and it is infinity or NaN, aggregates such as stddev_pop and covar_pop should produce a NaN result, because the calculation is not well-defined. They used to do so, but since we adopted Youngs-Cramer aggregation in commit e954a727f, they produced zero instead. That's an oversight, so fix it. Add tests exercising these edge cases. Affected aggregates are var_pop(double precision) stddev_pop(double precision) var_pop(real) stddev_pop(real) regr_sxx(double precision,double precision) regr_syy(double precision,double precision) regr_sxy(double precision,double precision) regr_r2(double precision,double precision) regr_slope(double precision,double precision) regr_intercept(double precision,double precision) covar_pop(double precision,double precision) corr(double precision,double precision) Back-patch to v12 where the behavior change was accidentally introduced. Report and patch by me; thanks to Dean Rasheed for review. Discussion: https://postgr.es/m/353062.1591898766@sss.pgh.pa.us
Diffstat (limited to 'src/backend/access/gist/gistbuildbuffers.c')
0 files changed, 0 insertions, 0 deletions