diff options
| author | Neil Conway <neilc@samurai.com> | 2007-09-19 22:31:51 +0000 |
|---|---|---|
| committer | Neil Conway <neilc@samurai.com> | 2007-09-19 22:31:51 +0000 |
| commit | d5e6f4f82802ddb18ec02780d21d0e50d2b30eba (patch) | |
| tree | 8c2a95049cb1ac2c7eb40dbe6f25ec6679d9e518 /src/backend/utils/adt/float.c | |
| parent | eb4f4f5b7cb0b95fed42fe44d8b91c21529e25d5 (diff) | |
| download | postgresql-d5e6f4f82802ddb18ec02780d21d0e50d2b30eba.tar.gz postgresql-d5e6f4f82802ddb18ec02780d21d0e50d2b30eba.zip | |
Prevent corr() from returning the wrong results for negative correlation
values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.
Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
Diffstat (limited to 'src/backend/utils/adt/float.c')
| -rw-r--r-- | src/backend/utils/adt/float.c | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 161e02ba273..81d724e499e 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.130 2006/10/05 01:40:45 tgl Exp $ + * $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.130.2.1 2007/09/19 22:31:51 neilc Exp $ * *------------------------------------------------------------------------- */ @@ -2469,8 +2469,7 @@ float8_corr(PG_FUNCTION_ARGS) if (numeratorX <= 0 || numeratorY <= 0) PG_RETURN_NULL(); - PG_RETURN_FLOAT8(sqrt((numeratorXY * numeratorXY) / - (numeratorX * numeratorY))); + PG_RETURN_FLOAT8(numeratorXY / sqrt(numeratorX * numeratorY)); } Datum |