Remove arbitrary 10MB limit on two-phase state file size. It's not that hard

to go beoynd 10MB, as demonstrated by Gavin Sharry's example of dropping a
schema with ~25000 objects. The really bogus thing about the limit was that
it was enforced when a state file file was read in, not when it was written,
so you would end up with a prepared transaction that you can't commit or
abort, and the only recourse was to shut down the server and remove the file
by hand.

Raise the limit to MaxAllocSize, and enforce it also when a state file is
written. We could've removed the limit altogether, but reading in a file
larger than MaxAllocSize would fail anyway because we read it into a
palloc'd buffer.

Backpatch down to 8.1, where 2PC and this issue was introduced.

1 files changed, 15 insertions, 3 deletions

diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 36f5514b69e..c38ce14289f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -7,7 +7,7 @@
  * Portions Copyright (c) 1994, Regents of the University of California
  *
  * IDENTIFICATION
- *		$PostgreSQL: pgsql/src/backend/access/transam/twophase.c,v 1.39.2.1 2008/04/26 23:35:33 tgl Exp $
+ *		$PostgreSQL: pgsql/src/backend/access/transam/twophase.c,v 1.39.2.2 2008/05/19 18:16:46 heikki Exp $
  *
  * NOTES
  *		Each global transaction is associated with a global transaction
@@ -56,6 +56,7 @@
 #include "storage/procarray.h"
 #include "storage/smgr.h"
 #include "utils/builtins.h"
+#include "utils/memutils.h"
 
 
 /*
@@ -866,6 +867,15 @@ EndPrepare(GlobalTransaction gxact)
 	hdr->total_len = records.total_len + sizeof(pg_crc32);
 
 	/*
+	 * If the file size exceeds MaxAllocSize, we won't be able to read it in
+	 * ReadTwoPhaseFile. Check for that now, rather than fail at commit time.
+	 */
+	if (hdr->total_len > MaxAllocSize)
+		ereport(ERROR,
+				(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+				 errmsg("two-phase state file maximum length exceeded")));
+
+	/*
 	 * Create the 2PC state file.
 	 *
 	 * Note: because we use BasicOpenFile(), we are responsible for ensuring
@@ -1045,7 +1055,9 @@ ReadTwoPhaseFile(TransactionId xid)
 
 	/*
 	 * Check file length.  We can determine a lower bound pretty easily. We
-	 * set an upper bound mainly to avoid palloc() failure on a corrupt file.
+	 * set an upper bound to avoid palloc() failure on a corrupt file, though
+	 * we can't guarantee that we won't get an out of memory error anyway,
+	 * even on a valid file.
 	 */
 	if (fstat(fd, &stat))
 	{
@@ -1060,7 +1072,7 @@ ReadTwoPhaseFile(TransactionId xid)
 	if (stat.st_size < (MAXALIGN(sizeof(TwoPhaseFileHeader)) +
 						MAXALIGN(sizeof(TwoPhaseRecordOnDisk)) +
 						sizeof(pg_crc32)) ||
-		stat.st_size > 10000000)
+		stat.st_size > MaxAllocSize)
 	{
 		close(fd);
 		return NULL;