Fix GIN's shimTriConsistentFn to not corrupt its input.

Commit 0f21db36d made an assumption that GIN triConsistentFns
would not modify their input entryRes[] arrays.  But in fact,
the "shim" triConsistentFn that we use for opclasses that don't
supply their own did exactly that, potentially leading to wrong
answers from a GIN index search.  Through bad luck, none of the
test cases that we have for such opclasses exposed the bug.

One response to this could be that the assumption of consistency check
functions not modifying entryRes[] arrays is a bad one, but it still
seems reasonable to me.  Notably, shimTriConsistentFn is itself
assuming that with respect to the underlying boolean consistentFn,
so it's sure being self-centered in supposing that it gets to do so.

Fortunately, it's quite simple to fix shimTriConsistentFn to restore
the entry-time state of entryRes[], so let's do that instead.

This issue doesn't affect any core GIN opclasses, since they all
supply their own triConsistentFns.  It does affect contrib modules
btree_gin, hstore, and intarray.

Along the way, I (tgl) noticed that shimTriConsistentFn failed to
pick up on a "recheck" flag returned by its first call to the boolean
consistentFn.  This may be only a latent problem, since it would be
unlikely for a consistentFn to set recheck for the all-false case
and not any other cases.  (Indeed, none of our contrib modules do
that.)  Nonetheless, it's formally wrong.

Reported-by: Vinod Sridharan <vsridh90@gmail.com>
Author: Vinod Sridharan <vsridh90@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAFMdLD7XzsXfi1+DpTqTgrD8XU0i2C99KuF=5VHLWjx4C1pkcg@mail.gmail.com
Backpatch-through: 13

1 files changed, 15 insertions, 5 deletions

diff --git a/src/backend/access/gin/ginlogic.c b/src/backend/access/gin/ginlogic.c
index 665ff9b9810..ff456247cef 100644
--- a/src/backend/access/gin/ginlogic.c
+++ b/src/backend/access/gin/ginlogic.c
@@ -140,7 +140,9 @@ shimBoolConsistentFn(GinScanKey key)
  * every combination is O(n^2), so this is only feasible for a small number of
  * MAYBE inputs.
  *
- * NB: This function modifies the key->entryRes array!
+ * NB: This function modifies the key->entryRes array.  For now that's okay
+ * so long as we restore the entry-time contents before returning.  This may
+ * need revisiting if we ever invent multithreaded GIN scans, though.
  */
 static GinTernaryValue
 shimTriConsistentFn(GinScanKey key)
@@ -149,7 +151,7 @@ shimTriConsistentFn(GinScanKey key)
 	int			maybeEntries[MAX_MAYBE_ENTRIES];
 	int			i;
 	bool		boolResult;
-	bool		recheck = false;
+	bool		recheck;
 	GinTernaryValue curResult;
 
 	/*
@@ -169,8 +171,8 @@ shimTriConsistentFn(GinScanKey key)
 	}
 
 	/*
-	 * If none of the inputs were MAYBE, so we can just call consistent
-	 * function as is.
+	 * If none of the inputs were MAYBE, we can just call the consistent
+	 * function as-is.
 	 */
 	if (nmaybe == 0)
 		return directBoolConsistentFn(key);
@@ -179,6 +181,7 @@ shimTriConsistentFn(GinScanKey key)
 	for (i = 0; i < nmaybe; i++)
 		key->entryRes[maybeEntries[i]] = GIN_FALSE;
 	curResult = directBoolConsistentFn(key);
+	recheck = key->recheckCurItem;
 
 	for (;;)
 	{
@@ -200,13 +203,20 @@ shimTriConsistentFn(GinScanKey key)
 		recheck |= key->recheckCurItem;
 
 		if (curResult != boolResult)
-			return GIN_MAYBE;
+		{
+			curResult = GIN_MAYBE;
+			break;
+		}
 	}
 
 	/* TRUE with recheck is taken to mean MAYBE */
 	if (curResult == GIN_TRUE && recheck)
 		curResult = GIN_MAYBE;
 
+	/* We must restore the original state of the entryRes array */
+	for (i = 0; i < nmaybe; i++)
+		key->entryRes[maybeEntries[i]] = GIN_MAYBE;
+
 	return curResult;
 }