aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Re-bin segment when memory pages are freed.Thomas Munro2023-07-04
| | | | | | | | | | | | | | | | | | | | | It's OK to be lazy about re-binning memory segments when allocating, because that can only leave segments in a bin that's too high. We'll search higher bins if necessary while allocating next time, and also eventually re-bin, so no memory can become unreachable that way. However, when freeing memory, the largest contiguous range of free pages might go up, so we should re-bin eagerly to make sure we don't leave the segment in a bin that is too low for get_best_segment() to find. The re-binning code is moved into a function of its own, so it can be called whenever free pages are returned to the segment's free page map. Back-patch to all supported releases. Author: Dongming Liu <ldming101@gmail.com> Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier version) Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CAL1p7e8LzB2LSeAXo2pXCW4%2BRya9s0sJ3G_ReKOU%3DAjSUWjHWQ%40mail.gmail.com
* Fix race in SSI interaction with gin fast path.Thomas Munro2023-07-04
| | | | | | | | | | | | | | | | | | The ginfast.c code previously checked for conflicts in before locking the relevant buffer, leaving a window where a RW conflict could be missed. Re-order. There was also a place where buffer ID and block number were confused while trying to predicate-lock a page, noted by visual inspection. Back-patch to all supported releases. Fixes one more problem discovered with the reproducer from bug #17949, in this case when Dmitry tried other index types. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reported-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
* Fix race in SSI interaction with bitmap heap scan.Thomas Munro2023-07-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When performing a bitmap heap scan, we don't want to miss concurrent writes that occurred after we observed the heap's rs_nblocks, but before we took predicate locks on index pages. Therefore, we can't skip fetching any heap tuples that are referenced by the index, because we need to test them all with CheckForSerializableConflictOut(). The old optimization that would ignore any references to blocks >= rs_nblocks gets in the way of that requirement, because it means that concurrent writes in that window are ignored. Removing that optimization shouldn't affect correctness at any isolation level, because any new tuples shouldn't be visible to an MVCC snapshot. There also shouldn't be any error-causing references to heap blocks past the end, because we should have held at least an AccessShareLock on the table before the index scan. It can't get smaller while our transaction is running. For now, though, we'll keep the optimization at lower levels to avoid making unnecessary changes in a bug fix. Back-patch to all supported releases. In release 11, the code is in a different place but not fundamentally different. Fixes one aspect of bug #17949. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
* Fix race in SSI interaction with empty btrees.Thomas Munro2023-07-04
| | | | | | | | | | | | | | | | | When predicate-locking btrees, we have a special case for completely empty btrees, since there is no page to lock. This was racy, because, without buffer lock held, a matching key could be inserted between the _bt_search() and the PredicateLockRelation() calls. Fix, by rechecking _bt_search() after taking the relation-level SIREAD lock, if using SERIALIZABLE isolation and an empty btree is discovered. Back-patch to all supported releases. Fixes one aspect of bug #17949. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
* Remove expensive test of postgres_fdw batch insertsTomas Vondra2023-07-03
| | | | | | | | | | | | | The test inserted 70k rows into a foreign table, in order to verify correct behavior with more than 65535 parameters, and was added in response to a bug report. However, this is rather expensive, especially when running the tests under valgrind, CLOBBER_CACHE_ALWAYS etc. It doesn't seem worth it to keep running the test, so remove it from all branches (14+). Backpatch-through: 14 Discussion: https://postgr.es/m/2131017.1623451468@sss.pgh.pa.us
* Improve pg_basebackup long file name test Windows robustnessAndrew Dunstan2023-07-03
| | | | | | | | | | Creation of a file with a very long name can create problems on Windows due to its file path limits. Work around that by creating the file via a symlink with a shorter name. Error displayed by buildfarm animal fairywren.o Backpatch to all live branches
* Make PG_TEST_NOCLEAN work for temporary directories in TAP testsMichael Paquier2023-07-03
| | | | | | | | | | | | | | | | When set, this environment variable was only effective for data directories but not for all the other temporary files created by PostgreSQL::Test::Utils. Keeping the temporary files after a successful run can be useful for debugging purposes. The documentation is updated to reflect the new behavior, with contents available in doc/ since v16 and in src/test/perl/README since v15. Author: Jacob Champion Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CAAWbhmgHtDH1SGZ+Fw05CsXtE0mzTmjbuUxLB9mY9iPKgM6cUw@mail.gmail.com Discussion: https://postgr.es/m/YyPd9unV14SX2bLF@paquier.xyz Backpatch-through: 11
* Silence "missing contrecord" error.Thomas Munro2023-07-03
| | | | | | | | | | | | | | | | | Commit dd38ff28ad added a new error message "missing contrecord" when we fail to reassemble a record. Unfortunately that caused noisy messages to be logged by pg_waldump at end of segment, and by walsender when asked to shut down on a segment boundary. Remove the new error message, so that this condition signals end-of- WAL without a message. It's arguably a reportable condition that should not be silenced while performing crash recovery, but fixing that without introducing noise in the other cases will require more research. Back-patch to 15. Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Discussion: https://postgr.es/m/6a1df56e-4656-b3ce-4b7a-a9cb41df8189%40enterprisedb.com
* Fix oversight in handling of modifiedCols since f24523672dTomas Vondra2023-07-02
| | | | | | | | | | | | | | | | | | | Commit f24523672d fixed a memory leak by moving the modifiedCols bitmap into the per-row memory context. In the case of AFTER UPDATE triggers, the bitmap is however referenced from an event kept until the end of the query, resulting in a use-after-free bug. Fixed by copying the bitmap into the AfterTriggerEvents memory context, which is the one where we keep the trigger events. There's only one place that needs to do the copy, but the memory context may not exist yet. Doing that in a separate function seems more readable. Report by Alexander Pyhalov, fix by me. Backpatch to 13, where the bitmap was added to the event by commit 71d60e2aa0. Reported-by: Alexander Pyhalov Backpatch-through: 13 Discussion: https://postgr.es/m/acddb17c89b0d6cb940eaeda18c08bbe@postgrespro.ru
* Fix memory leak in Incremental Sort rescansTomas Vondra2023-07-02
| | | | | | | | | | | | | | | | | | | | | | | | The Incremental Sort had a couple issues, resulting in leaking memory during rescans, possibly triggering OOM. The code had a couple of related flaws: 1. During rescans, the sort states were reset but then also set to NULL (despite the comment saying otherwise). ExecIncrementalSort then sees NULL and initializes a new sort state, leaking the memory used by the old one. 2. Initializing the sort state also automatically rebuilt the info about presorted keys, leaking the already initialized info. presorted_keys was also unnecessarily reset to NULL. Patch by James Coleman, based on patches by Laurenz Albe and Tom Lane. Backpatch to 13, where Incremental Sort was introduced. Author: James Coleman, Laurenz Albe, Tom Lane Reported-by: Laurenz Albe, Zu-Ming Jiang Backpatch-through: 13 Discussion: https://postgr.es/m/b2bd02dff61af15e3526293e2771f874cf2a3be7.camel%40cybertec.at Discussion: https://postgr.es/m/db03c582-086d-e7cd-d4a1-3bc722f81765%40inf.ethz.ch
* Fix marking of indisvalid for partitioned indexes at creationMichael Paquier2023-06-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The logic that introduced partitioned indexes missed a few things when invalidating a partitioned index when these are created, still the code is written to handle recursions: 1) If created from scratch because a mapping index could not be found, the new index created could be itself invalid, if for example it was a partitioned index with one of its leaves invalid. 2) A CCI was missing when indisvalid is set for a parent index, leading to inconsistent trees when recursing across more than one level for a partitioned index creation if an invalidation of the parent was required. This could lead to the creation of a partition index tree where some of the partitioned indexes are marked as invalid, but some of the parents are marked valid, which is not something that should happen (as validatePartitionedIndex() defines, indisvalid is switched to true for a partitioned index iff all its partitions are themselves valid). This patch makes sure that indisvalid is set to false on a partitioned index if at least one of its partition is invalid. The flag is set to true if *all* its partitions are valid. The regression test added in this commit abuses of a failed concurrent index creation, marked as invalid, that maps with an index created on its partitioned table afterwards. Reported-by: Alexander Lakhin Reviewed-by: Alexander Lakhin Discussion: https://postgr.es/m/14987634-43c0-0cb3-e075-94d423607e08@gmail.com Backpatch-through: 11
* Fix pg_depend entry to AMs after ALTER TABLE .. SET ACCESS METHODMichael Paquier2023-06-30
| | | | | | | | | | | | | | | | | | | | | | ALTER TABLE .. SET ACCESS METHOD was not registering a dependency to the new access method with the relation altered in its rewrite phase, making possible the drop of an access method even if there are relations that depend on it. During the rewrite, a temporary relation is created to build the new relation files before swapping the new and old files, and, while the temporary relation was registering a correct dependency to the new AM, the old relation did not do that. A dependency on the access method is added when the relation files are swapped, which is the point where pg_class is updated. Materialized views and tables use the same code path, hence both were impacted. Backpatch down to 15, where this command has been introduced. Reported-by: Alexander Lakhin Reviewed-by: Nathan Bossart, Andres Freund Discussion: https://postgr.es/m/18000-9145c25b1af475ca@postgresql.org Backpatch-through: 15
* Fix order of operations in ExecEvalFieldStoreDeForm().Tom Lane2023-06-29
| | | | | | | | | | | | | | | | | | | | | | | | | If the given composite datum is toasted out-of-line, DatumGetHeapTupleHeader will perform database accesses to detoast it. That can invalidate the result of get_cached_rowtype, as documented (perhaps not plainly enough) in that function's API spec; which leads to strange errors or crashes when we try to use the TupleDesc to read the tuple. In short then, trying to update a field of a composite column could fail intermittently if the overall column value is wide enough to require toasting. We can fix the bug at no cost by just changing the order of operations, since we don't need the TupleDesc until after detoasting. (Other callers of get_cached_rowtype appear to get this right already, so there's only one bug.) Note that the added regression test case reveals this bug reliably only with debug_discard_caches/CLOBBER_CACHE_ALWAYS. Per bug #17994 from Alexander Lakhin. Sadly, this patch does not fix the missing-values issue revealed in the bug discussion; we'll need some more work to cover that. Discussion: https://postgr.es/m/17994-5c7100b51b4790e9@postgresql.org
* Remove inappropriate raw_expression_tree_walker() codePeter Eisentraut2023-06-29
| | | | | | | | It was walking into the ColumnDef->compression field, which is not a node but a string. This code is currently not reachable (because the compression field is only set in situations that don't go through raw_expression_tree_walker()), but if it had been, this could have behaved erratically.
* pg_stat_statements: Fix second comment related to entry resetsMichael Paquier2023-06-29
| | | | | | | | | This should have been part of dc73db6, but it got lost in the mix. Oversight in 6b4d23f. Author: Japin Li Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM Backpatch-through: 14
* pg_stat_statements: Fix incorrect comment with entry resetsMichael Paquier2023-06-29
| | | | | | | | Oversight in 6b4d23f. Author: Japin Li, Richard Guo Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM Backpatch-through: 14
* Ignore invalid indexes when enforcing index rules in ALTER TABLE ATTACH ↵Michael Paquier2023-06-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PARTITION A portion of ALTER TABLE .. ATTACH PARTITION is to ensure that the partition being attached to the partitioned table has a correct set of indexes, so as there is a consistent index mapping between the partitioned table and its new-to-be partition. However, as introduced in 8b08f7d, the current logic could choose an invalid index as a match, which is something that can exist when dealing with more than two levels of partitioning, like attaching a partitioned table (that has partitions, with an index created by CREATE INDEX ON ONLY) to another partitioned table. A partitioned index with indisvalid set to false is equivalent to an incomplete partition tree, meaning that an invalid partitioned index does not have indexes defined in all its partitions. Hence, choosing an invalid partitioned index can create inconsistent partition index trees, where the parent attaching to is valid, but its partition may be invalid. In the report from Alexander Lakhin, this showed up as an assertion failure when validating an index. Without assertions enabled, the partition index tree would be actually broken, as indisvalid should be switched to true for a partitioned index once all its partitions are themselves valid. With two levels of partitioning, the top partitioned table used a valid index and was able to link to an invalid index stored on its partition, itself a partitioned table. I have studied a few options here (like the possibility to switch indisvalid to false for the parent), but came down to the conclusion that we'd better rely on a simple rule: invalid indexes had better never be chosen, so as the partition attached uses and creates indexes that the parent expects. Some regression tests are added to provide some coverage. Note that the existing coverage is not impacted. This is a problem since partitioned indexes exist, so backpatch all the way down to v11. Reported-by: Alexander Lakhin Discussion: https://postgr.es/14987634-43c0-0cb3-e075-94d423607e08@gmail.com Backpatch-through: 11
* Fix comment on clearing padding.Heikki Linnakangas2023-06-27
| | | | | Author: Japin Li Discussion: https://www.postgresql.org/message-id/MEYP282MB16696317B5DA7D0D92306149B627A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
* Check for interrupts and stack overflow in TParserGet().Tom Lane2023-06-24
| | | | | | | | | | | | | | | TParserGet() recurses for some token types, meaning it's possible to drive it to stack overflow. Since this is a minority behavior, I chose to add the check_stack_depth() call to the two places that recurse rather than doing it during every single call. While at it, add CHECK_FOR_INTERRUPTS(), because this can run unpleasantly long for long inputs. Per bug #17995 from Zuming Jiang. This is old, so back-patch to all supported branches. Discussion: https://postgr.es/m/17995-9f20ff3e6389db4c@postgresql.org
* doc: rename "decades" to be more genericBruce Momjian2023-06-23
| | | | | | | | Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZJTzwD2rTbHWWQ9g@paquier.xyz Backpatch-through: 11
* Fix incorrect error message in libpq_pipelineMichael Paquier2023-06-23
| | | | | | | | | | One of the tests for the pipeline mode with portal description expects a non-NULL PQgetResult, but used an incorrect error message on failure, telling that PQgetResult being NULL was the expected result. Author: Jelte Fennema Discussion: https://postgr.es/m/CAGECzQTkShHecFF+EZrm94Lbsu2ej569T=bz+PjMbw9Aiioxuw@mail.gmail.com Backpatch-through: 14
* Doc: Clarify the behavior of triggers/rules in a logical subscriber.Amit Kapila2023-06-22
| | | | | | | | | | | | | By default, triggers and rules do not fire on a logical replication subscriber based on the "session_replication_role" GUC being set to "replica". However, the docs in the logical replication section assumed that the reader understood how this GUC worked. This modifies the docs to be more explicit and links back to the GUC itself. Author: Jonathan Katz, Peter Smith Reviewed-by: Vignesh C, Euler Taveira Backpatch-through: 11 Discussion: https://postgr.es/m/5bb2c9a2-499f-e1a2-6e33-5ce96b35cc4a@postgresql.org
* Doc: mention that extended stats aren't used for joinsDavid Rowley2023-06-22
| | | | | | | | | | Statistics defined by the CREATE STATISTICS command are only used to assist with the selectivity estimations of base relations, never for joins. Here we mention this fact in the notes section of the CREATE STATISTICS command. Discussion: https://postgr.es/m/CAApHDvrMuVgDOrmg_EtFDZ=AOovq6EsJNnHH1ddyZ8EqL4yzMw@mail.gmail.com Backpatch-through: 11
* nbtree VACUUM: cope with topparent inconsistencies.Peter Geoghegan2023-06-21
| | | | | | | | | | | | | | | | | | Avoid "right sibling %u of block %u is not next child" errors when vacuuming a corrupt nbtree index. Just LOG the issue and press on. That way VACUUM will have a decent chance of finishing off all required processing for the index (and for the table as a whole). This is similar to recent work from commit 5abff197, as well as work from commit 5b861baa (later backpatched as commit 43e409ce), which taught nbtree VACUUM to keep going when its "re-find" check fails. The hardening added by this commit takes place directly after the "re-find" check, right before the critical section for the first stage of page deletion. Author: Peter Geoghegan <pg@bowt.ie> Discussion: https://postgr.es/m/CAH2-Wz=dayg0vjs4+er84TS9ami=csdzjpuiCGbEw=idhwqhzQ@mail.gmail.com Backpatch: 11- (all supported versions).
* doc: update PG history as over "three decades"Bruce Momjian2023-06-21
| | | | | | | | Reported-by: Pierre <pbaumard@gmail.com> Discussion: https://postgr.es/m/168724660637.399156.7642965215720120947@wrigleys.postgresql.org Backpatch-through: 11
* Avoid Assert failure when processing empty statement in aborted xact.Tom Lane2023-06-21
| | | | | | | | | | | | | | | | | | | | | exec_parse_message() wants to create a cached plan in all cases, including for empty input. The empty-input path does not have a test for being in an aborted transaction, making it possible that plancache.c will fail due to trying to do database lookups even though there's no real work to do. One solution would be to throw an aborted-transaction error in this path too, but it's not entirely clear whether the lack of such an error was intentional or whether some clients might be relying on non-error behavior. Instead, let's hack plancache.c so that it treats empty statements with the same logic it already had for transaction control commands, ensuring that it can soldier through even in an already-aborted transaction. Per bug #17983 from Alexander Lakhin. Back-patch to all supported branches. Discussion: https://postgr.es/m/17983-da4569fcb878672e@postgresql.org
* Disable use of archiving in 009_twophase.plMichael Paquier2023-06-21
| | | | | | | | | | | | | | | | | This partially reverts 68cb5af, as using archiving to enforce the rename of the last partial segment of the old timeline at promotion to use .partial as suffix is impacting the tests when it does switchovers. As showed by the logs gathered by the CI in the tests that failed, a new standby may fail to find the WAL segment it needs to follow a promoted instance with its timeline jump, as it got renamed to .partial. This problem would manifest as a run timeout with 009_twophase.pl, as the new standby repeatedly requests a segment from the promoted primary that it would not find. Reported-by: Nathan Bossart Discussion: https://postgr.es/m/20230621043345.GA787473@nathanxps13 Backpatch-through: 13
* Fix the errhint message and docs for drop subscription failure.Amit Kapila2023-06-21
| | | | | | | | | | The existing errhint message and docs were missing the fact that we can't disassociate from the slot unless the subscription is disabled. Author: Robert Sjöblom, Peter Smith Reviewed-by: Peter Eisentraut, Amit Kapila Backpatch-through: 11 Discussion: https://postgr.es/m/807bdf85-61ea-88e2-5712-6d9fcd4eabff@fortnox.se
* Fix hash join when inner hashkey expressions contain Params.Tom Lane2023-06-20
| | | | | | | | | | | | | | | | | | | | | | | | If the inner-side expressions contain PARAM_EXEC Params, we must re-hash whenever the values of those Params change. The executor mechanism for that exists already, but we failed to invoke it because finalize_plan() neglected to search the Hash.hashkeys field for Params. This allowed a previous scan's hash table to be re-used when it should not be, leading to rows missing from the join's output. (I believe incorrectly-included join rows are impossible however, since checking the real hashclauses would reject false matches.) This bug is very ancient, dating probably to d24d75ff1 of 7.4. Sadly, this simple fix depends on the plan representational changes made by 2abd7ae9b, so it will only work back to v12. I thought about trying to make some kind of hack for v11, but I'm leery of putting code significantly different from what is used in the newer branches into a nearly-EOL branch. Seeing that the bug escaped detection for a full twenty years, problematic cases must be rare; so I don't feel too awful about leaving v11 as-is. Per bug #17985 from Zuming Jiang. Back-patch to v12. Discussion: https://postgr.es/m/17985-748b66607acd432e@postgresql.org
* docs: adjust tag indenting and add MERGE mentionBruce Momjian2023-06-20
| | | | | | | | Discussion: https://postgr.es/m/CAMpnoC4_WsY3gsY+ud-Z0GDbafR=K7t7cXn2gatEqFnsRNY3yQ@mail.gmail.com Author: Will Mortensen Backpatch-through: 15
* Enable archiving in recovery TAP test 009_twophase.plMichael Paquier2023-06-20
| | | | | | | | | | | | | | | | | This is a follow-up of f663b00, that has been committed to v13 and v14, tweaking the TAP test for two-phase transactions so as it provides coverage for the bug that has been fixed. This change is done in its own commit for clarity, as v15 and HEAD did not show the problematic behavior, still missed coverage for it. While on it, this adds a comment about the dependency of the last partial segment rename and RecoverPreparedTransactions() at the end of recovery, as that can be easy to miss. Author: Michael Paquier Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/743b9b45a2d4013bd90b6a5cba8d6faeb717ee34.camel@cybertec.at Backpatch-through: 13
* Don't use partial unique indexes for unique proofs in the plannerDavid Rowley2023-06-19
| | | | | | | | | | | | | | | | | | | Here we adjust relation_has_unique_index_for() so that it no longer makes use of partial unique indexes as uniqueness proofs. It is incorrect to use these as the predicates used by check_index_predicates() to set predOK makes use of not only baserestrictinfo quals as proofs, but also qual from join conditions. For relation_has_unique_index_for()'s case, we need to know the relation is unique for a given set of columns before any joins are evaluated, so if predOK was only set to true due to some join qual, then it's unsafe to use such indexes in relation_has_unique_index_for(). The final plan may not even make use of that index, which could result in reading tuples that are not as unique as the planner previously expected them to be. Bug: #17975 Reported-by: Tor Erik Linnerud Backpatch-through: 11, all supported versions Discussion: https://postgr.es/m/17975-98a90c156f25c952%40postgresql.org
* Fix typo in comment.Amit Langote2023-06-16
| | | | | | | Back-patch down to 11. Author: Sho Kato (<kato-sho@fujitsu.com>) Discussion: https://postgr.es/m/TYCPR01MB68499042A33BC32241193AAF9F5BA%40TYCPR01MB6849.jpnprd01.prod.outlook.com
* intarray: Prevent out-of-bound memory reads with gist__int_opsMichael Paquier2023-06-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As gist__int_ops stands in intarray, it is possible to store GiST entries for leaf pages that can cause corruptions when decompressed. Leaf nodes are stored as decompressed all the time by the compression method, and the decompression method should map with that, retrieving the contents of the page without doing any decompression. However, the code authorized the insertion of leaf page data with a higher number of array items than what can be supported, generating a NOTICE message to inform about this matter (199 for a 8k page, for reference). When calling the decompression method, a decompression would be attempted on this leaf node item but the contents should be retrieved as they are. The NOTICE message generated when dealing with the compression of a leaf page and too many elements in the input array for gist__int_ops has been introduced by 08ee64e, removing the marker stored in the array to track if this is actually a leaf node. However, it also missed the fact that the decompression path should do nothing for a leaf page. Hence, as the code stand, a too-large array would be stored as uncompressed but the decompression path would attempt a decompression rather that retrieving the contents as they are. This leads to various problems. First, even if 08ee64e tried to address that, it is possible to do out-of-bound chunk writes with a large input array, with the backend informing about that with WARNINGs. On decompression, retrieving the stored leaf data would lead to incorrect memory reads, leading to crashes or even worse. Perhaps somebody would be interested in expanding the number of array items that can be handled in a leaf page for this operator in the future, which would require revisiting the choice done in 08ee64e, but based on the lack of reports about this problem since 2005 it does not look so. For now, this commit prevents the insertion of data for leaf pages when using more array items that the code can handle on decompression, switching the NOTICE message to an ERROR. If one wishes to use more array items, gist__intbig_ops is an optional choice. While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a limit is reached, because that's what the module is facing in such cases. Author: Ankit Kumar Pandey, Alexander Lakhin Reviewed-by: Richard Guo, Michael Paquier Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org Backpatch-through: 11
* Fix make_etags breakage on certain platforms.Tatsuo Ishii2023-06-14
| | | | | | | | | | make_etags produced wrong format TAGS files on platforms such as Mac, which uses non-Exuberant ctags. Author: Masahiko Sawada Reviewed-by: Tatsuo Ishii Backpatch-through: 15 Discussion: https://postgr.es/m/CAD21AoDmCqpS%2BU6b9Bc-b4OFx3tz%3DNv6O2KVkoVg7sHk60spjA%40mail.gmail.com
* Correctly update hasSubLinks while mutating a rule action.Tom Lane2023-06-13
| | | | | | | | | | | | | | | | | | | | | rewriteRuleAction neglected to check for SubLink nodes in the securityQuals of range table entries. This could lead to failing to convert such a SubLink to a SubPlan, resulting in assertion crashes or weird errors later in planning. In passing, fix some poor coding in rewriteTargetView: we should not pass the source parsetree's hasSubLinks field to ReplaceVarsFromTargetList's outer_hasSubLinks. ReplaceVarsFromTargetList knows enough to ignore that when a Query node is passed, but it's still confusing and bad precedent: if we did try to update that flag we'd be updating a stale copy of the parsetree. Per bug #17972 from Alexander Lakhin. This has been broken since we added RangeTblEntry.securityQuals (although the presented test case only fails back to 215b43cdc), so back-patch all the way. Discussion: https://postgr.es/m/17972-f422c094237847d0@postgresql.org
* Accept fractional seconds in jsonpath's datetime() method.Tom Lane2023-06-12
| | | | | | | | | | | | | | | | | | | | | | Commit 927d9abb6 purported to make datetime() accept any string that could be output for a datetime value by to_jsonb(). But it overlooked the possibility of fractional seconds being present, so that cases as simple as to_jsonb(now()) would defeat it. Fix by adding formats that include ".US" to the list in executeDateTimeMethod(). (Note that while this is nominally microseconds, it'll do the right thing for fractions with fewer than six digits.) In passing, re-order the list to restore the datatype ordering specified in its comment. The violation accidentally did not break anything; but the next edit might be less lucky, so add more comments. Per report from Tim Field. Back-patch to v13 where datetime() was added, like the previous patch. Discussion: https://postgr.es/m/014A028B-5CE6-4FDF-AC24-426CA6FC9CEE@mohiohio.com
* hstore: Tighten key/value parsing check for whitespacesMichael Paquier2023-06-12
| | | | | | | | | | | | | | | | | isspace() can be locale-sensitive depending on the platform, causing hstore to consider as whitespaces characters it should not see as such. For example, U+0105, being decoded as 0xC4 0x85 in UTF-8, would be discarded from the input given. This problem is similar to 9ae2661, though it was missed that hstore can also manipulate non-ASCII inputs, so replace the existing isspace() calls with scanner_isspace(). This problem exists for a long time, so backpatch all the way down. Author: Evan Jones Discussion: https://postgr.es/m/CA+HWA9awUW0+RV_gO9r1ABZwGoZxPztcJxPy8vMFSTbTfi4jig@mail.gmail.com Backpatch-through: 11
* Refactor routine to find single log content pattern in TAP testsMichael Paquier2023-06-09
| | | | | | | | | | | | | | | | | | | | | | | The same routine to check if a specific pattern can be found in the server logs was copied over four different test scripts. This refactors the whole to use a single routine located in PostgreSQL::Test::Cluster, named log_contains, to grab the contents of the server logs and check for a specific pattern. On HEAD, the code previously used assumed that slurp_file() could not handle an undefined offset, setting it to zero, but slurp_file() does do an extra fseek() before retrieving the log contents only if an offset is defined. In two places, the test was retrieving the full log contents with slurp_file() after calling substr() to apply an offset, ignoring that slurp_file() would be able to handle that. Backpatch all the way down to ease the introduction of new tests that could rely on the new routine. Author: Vignesh C Reviewed-by: Andrew Dunstan, Dagfinn Ilmari Mannsåker, Michael Paquier Discussion: https://postgr.es/m/CALDaNm0YSiLpjCmajwLfidQrFOrLNKPQir7s__PeVvh9U3uoTQ@mail.gmail.com Backpatch-through: 11
* Refactor log check logic for connect_ok/fails in PostgreSQL::Test::ClusterMichael Paquier2023-06-09
| | | | | | | | | | | | | | This commit refactors a bit the code in charge of checking for log patterns when connections fail or succeed, by moving the log pattern checks into their own routine, for clarity. This has come up as something to improve while discussing the refactoring of find_in_log(). Backpatch down to 14 where these routines are used, to ease the introduction of new tests that could rely on them. Author: Vignesh C, Michael Paquier Discussion: https://postgr.es/m/CALDaNm0YSiLpjCmajwLfidQrFOrLNKPQir7s__PeVvh9U3uoTQ@mail.gmail.com Backpatch-through: 14
* doc: Fix example command for ALTER FOREIGN TABLE ... OPTIONS.Fujii Masao2023-06-08
| | | | | | | | | | | | | | | In the documentation, previously the example command for ALTER FOREIGN TABLE ... OPTIONS incorrectly included both the option name and value with the DROP operation. The correct syntax for the DROP operation requires only the name of the option to be specified. This commit fixes the example by removing the option value from the DROP operation. Back-patch to all supported versions. Author: Mehmet Emin KARAKAS <emin100@gmail.com> Reviewed-by: Fujii Masao Discussion: https://postgr.es/m/CANQrdXAHzbcEYhjGoe5A42OmfvdQhHFJzyKj9gJvHuDKyOF5Ng@mail.gmail.com
* Use per-tuple context in ExecGetAllUpdatedColsTomas Vondra2023-06-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | Commit fc22b6623b (generated columns) replaced ExecGetUpdatedCols() with ExecGetAllUpdatedCols() in a couple places handling UPDATE (triggers and lock mode). However, ExecGetUpdatedCols() did exec_rt_fetch() while ExecGetAllUpdatedCols() also allocates memory through bms_union() without paying attention to the memory context and happened to use the long-lived ExecutorState, leaking the memory until the end of the query. The amount of leaked memory is proportional to the number of (updated) attributes, types of UPDATE triggers, and the number of processed rows (which for UPDATE ... FROM ... may be much higher than updated rows). Fixed by switching to the per-tuple context in GetAllUpdatedColumns(). This is fine for all in-core callers, but external callers may need to copy the result. But we're not aware of any such callers. Note the issue was introduced by fc22b6623b, but the macros were later renamed by f50e888990. Backpatch to 12, where the issue was introduced. Reported-by: Tomas Vondra Reviewed-by: Andres Freund, Tom Lane, Jakub Wartak Backpatch-through: 12 Discussion: https://postgr.es/m/222a3442-7f7d-246c-ed9b-a76209d19239@enterprisedb.com
* Initialize 'recordXtime' to silence compiler warning.Heikki Linnakangas2023-06-06
| | | | | | | | | | In reality, recordXtime will always be set by the getRecordTimestamp call, but the compiler doesn't necessarily see that. Back-patch to all supported versions. Author: Tristan Partin Discussion: https://www.postgresql.org/message-id/CT5MN8E11U0M.1NYNCHXYUHY41@gonk
* Doc: explain about dependency tracking for new-style SQL functions.Tom Lane2023-06-04
| | | | | | | | | | 5.14 Dependency Tracking was not updated when we added new-style SQL functions. Improve that. Noted by Sami Imseih. Back-patch to v14 where new-style SQL functions came in. Discussion: https://postgr.es/m/2C1933AB-C2F8-499B-9D18-4AC1882256A0@amazon.com
* Fix pg_dump's failure to honor dependencies of SQL functions.Tom Lane2023-06-04
| | | | | | | | | | | | | | | | | | | A new-style SQL function can contain a parse-time dependency on a unique index, much as views and matviews can (such cases arise from GROUP BY and ON CONFLICT clauses, for example). To dump and restore such a function successfully, pg_dump must postpone the function until after the unique index is created, which will happen in the post-data part of the dump. Therefore we have to remove the normal constraint that functions are dumped in pre-data. Add code similar to the existing logic that handles this for matviews. I added test cases for both as well, since code coverage tests showed that we weren't testing the matview logic. Per report from Sami Imseih. Back-patch to v14 where new-style SQL functions came in. Discussion: https://postgr.es/m/2C1933AB-C2F8-499B-9D18-4AC1882256A0@amazon.com
* Fix misuse of pg_log_info() for details/hints.Tom Lane2023-06-04
| | | | | | | | | | | | | | | | | Two places in pg_dump_sort.c were using pg_log_info() to add more details to a message printed with pg_log_warning(). This is bad, because at default verbosity level we would print the warning line but not the details. One should use pg_log_warning_detail() or pg_log_warning_hint() instead. Commit 9a374b77f got rid of most such abuses, but unaccountably missed these. Noted while studying a bug report from Sami Imseih. Back-patch to v15 where 9a374b77f came in. (Prior versions don't have the missing-details misbehavior, for reasons I didn't bother to track down.) Discussion: https://postgr.es/m/2C1933AB-C2F8-499B-9D18-4AC1882256A0@amazon.com
* doc: add missing "the" in LATERAL sentence.Bruce Momjian2023-06-01
| | | | Backpatch-through: 11
* nbtree VACUUM: cope with right sibling link corruption.Peter Geoghegan2023-05-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid "right sibling's left-link doesn't match" errors when vacuuming a corrupt nbtree index. Just LOG the issue and press on. That way VACUUM will have a decent chance of finishing off all required processing for the index (and for the table as a whole). This error was seen in the field from time to time (it's more than a theoretical risk), so giving VACUUM the ability to press on like this has real value. Nothing short of a REINDEX is expected to fix the underlying index corruption, so giving up (by throwing an error) risks making a bad situation far worse. Anything that blocks forward progress by VACUUM like this might go unnoticed for a long time. This could eventually lead to a wraparound/xidStopLimit outage. Note that _bt_unlink_halfdead_page() has always been able to bail on page deletion when the target page's left sibling page was in an inconsistent state. It now does the same thing (returns false to back out of the second phase of deletion) when it notices sibling link corruption in the target page's right sibling page. This is similar to the work from commit 5b861baa (later backpatched as commit 43e409ce), which taught nbtree to press on with vacuuming an index when page deletion fails to "re-find" a downlink in the target page's parent page. The "re-find" check seems to make VACUUM bail on page deletion more often in practice, but there is no reason to take any chances here. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/CAH2-Wzko2q2kP1+UvgJyP9g0mF4hopK0NtQZcxwvMv9_ytGhkQ@mail.gmail.com Backpatch: 11- (all supported versions).
* doc: Fix example query for pg_walinspectDaniel Gustafsson2023-05-25
| | | | | | | | | The LIMIT clause had ended up in the wrong place in the query. Backpatch to v15 where pg_walinspect was introduced. Reported-by: Jian He <jian.universality@gmail.com> Discussion: https://postgr.es/m/CACJufxHqXDr4NnmwmR6pEiVPAg54J0dgwMuYQzrH5BX6+NtF1g@mail.gmail.com Backpatch-through: 15
* Fix pgbench in prepared mode with an empty pipelineAlvaro Herrera2023-05-25
| | | | | | | | It crashes because it references memory that's not allocated in that particular case. Fix by allocating it. Reported-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/bcf802a6-afc1-95b9-7bf4-c5dd868ec144@gmail.com
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-30 23:12:15 +0000