diff options
| author | Tomas Vondra <tomas.vondra@postgresql.org> | 2021-06-08 20:22:18 +0200 |
|---|---|---|
| committer | Tomas Vondra <tomas.vondra@postgresql.org> | 2021-06-08 20:28:31 +0200 |
| commit | cb92703384e2bb3fa0a690e5dbb95ad333c2b44c (patch) | |
| tree | 644c14c6f86391b0ae60736b36330b0caa387864 /contrib/postgres_fdw/sql/postgres_fdw.sql | |
| parent | d1f0aa7696917213485c03b076b573497a535076 (diff) | |
| download | postgresql-cb92703384e2bb3fa0a690e5dbb95ad333c2b44c.tar.gz postgresql-cb92703384e2bb3fa0a690e5dbb95ad333c2b44c.zip | |
Adjust batch size in postgres_fdw to not use too many parameters
The FE/BE protocol identifies parameters with an Int16 index, which
limits the maximum number of parameters per query to 65535. With
batching added to postges_fdw this limit is much easier to hit, as
the whole batch is essentially a single query, making this error much
easier to hit.
The failures are a bit unpredictable, because it also depends on the
number of columns in the query. So instead of just failing, this patch
tweaks the batch_size to not exceed the maximum number of parameters.
Reported-by: Hou Zhijie <houzj.fnst@cn.fujitsu.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com
Diffstat (limited to 'contrib/postgres_fdw/sql/postgres_fdw.sql')
| -rw-r--r-- | contrib/postgres_fdw/sql/postgres_fdw.sql | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index 191efbf7c24..8cb2148f1f6 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -3026,6 +3026,13 @@ SELECT COUNT(*) FROM ftable; TRUNCATE batch_table; DROP FOREIGN TABLE ftable; +-- try if large batches exceed max number of bind parameters +CREATE FOREIGN TABLE ftable ( x int ) SERVER loopback OPTIONS ( table_name 'batch_table', batch_size '100000' ); +INSERT INTO ftable SELECT * FROM generate_series(1, 70000) i; +SELECT COUNT(*) FROM ftable; +TRUNCATE batch_table; +DROP FOREIGN TABLE ftable; + -- Disable batch insert CREATE FOREIGN TABLE ftable ( x int ) SERVER loopback OPTIONS ( table_name 'batch_table', batch_size '1' ); EXPLAIN (VERBOSE, COSTS OFF) INSERT INTO ftable VALUES (1), (2); |