aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Avoid compiler warning in non-assert buildsAmit Langote2023-10-26
| | | | | | | | | | | After 01575ad788e3, expand_single_inheritance_child()'s parentOID variable is read only in an Assert, provoking a compiler warning in non-assert builds. Fix that by marking the variable with PG_USED_FOR_ASSERTS_ONLY. Per report and suggestion from David Rowley Discussion: https://postgr.es/m/CAApHDvpjA_8Wxu4DCTRVAvPxC9atwMe6N%2ByvrcGsgb7mrfdpJA%40mail.gmail.com
* Add trailing commas to enum definitionsPeter Eisentraut2023-10-26
| | | | | | | | | | | | | | | | | | | | Since C99, there can be a trailing comma after the last value in an enum definition. A lot of new code has been introducing this style on the fly. Some new patches are now taking an inconsistent approach to this. Some add the last comma on the fly if they add a new last value, some are trying to preserve the existing style in each place, some are even dropping the last comma if there was one. We could nudge this all in a consistent direction if we just add the trailing commas everywhere once. I omitted a few places where there was a fixed "last" value that will always stay last. I also skipped the header files of libpq and ecpg, in case people want to use those with older compilers. There were also a small number of cases where the enum type wasn't used anywhere (but the enum values were), which ended up confusing pgindent a bit, so I left those alone. Discussion: https://www.postgresql.org/message-id/flat/386f8c45-c8ac-4681-8add-e3b0852c1620%40eisentraut.org
* Introduce the concept of read-only StringInfosDavid Rowley2023-10-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were various places in our codebase which conjured up a StringInfo by manually assigning the StringInfo fields and setting the data field to point to some existing buffer. There wasn't much consistency here as to what fields like maxlen got set to and in one location we didn't correctly ensure that the buffer was correctly NUL terminated at len bytes, as per what was documented as required in stringinfo.h Here we introduce 2 new functions to initialize StringInfos. One allows callers to initialize a StringInfo passing along a buffer that is already allocated by palloc. Here the StringInfo code uses this buffer directly rather than doing any memcpying into a new allocation. Having this as a function allows us to verify the buffer is correctly NUL terminated. StringInfos initialized this way can be appended to and reset just like any other normal StringInfo. The other new initialization function also accepts an existing buffer, but the given buffer does not need to be a pointer to a palloc'd chunk. This buffer could be a pointer pointing partway into some palloc'd chunk or may not even be palloc'd at all. StringInfos initialized this way are deemed as "read-only". This means that it's not possible to append to them or reset them. For the latter of the two new initialization functions mentioned above, we relax the requirement that the data buffer must be NUL terminated. Relaxing this requirement is convenient in a few places as it can save us from having to allocate an entire new buffer just to add the NUL terminator or save us from having to temporarily add a NUL only to have to put the original char back again later. Incompatibility note: Here we also forego adding the NUL in a few places where it does not seem to be required. These locations are passing the given StringInfo into a type's receive function. It does not seem like any of our built-in receive functions require this, but perhaps there's some UDT out there in the wild which does require this. It is likely worthy of a mention in the release notes that a UDT's receive function mustn't rely on the input StringInfo being NUL terminated. Author: David Rowley Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAApHDvorfO3iBZ%3DxpiZvp3uHtJVLyFaPBSvcAhAq2HPLnaNSwQ%40mail.gmail.com
* Prevent duplicate RTEPermissionInfo for plain-inheritance parentsAmit Langote2023-10-26
| | | | | | | | | | | | | | | | | | | Currently, expand_single_inheritance_child() doesn't reset perminfoindex in a plain-inheritance parent's child RTE, because prior to 387f9ed0a0, the executor would use the first child RTE to locate the parent's RTEPermissionInfo. That in turn causes add_rte_to_flat_rtable() to create an extra RTEPermissionInfo belonging to the parent's child RTE with the same content as the one belonging to the parent's original ("root") RTE. In 387f9ed0a0, we changed things so that the executor can now use the parent's "root" RTE for locating its RTEPermissionInfo instead of the child RTE, so the latter's perminfoindex need not be set anymore, so make it so. Reported-by: Tom Lane Discussion: https://postgr.es/m/839708.1698174464@sss.pgh.pa.us Backpatch-through: 16
* Migrate logical slots to the new node during an upgrade.Amit Kapila2023-10-26
| | | | | | | | | | | | | | | | | | | | | | | | While reading information from the old cluster, a list of logical slots is fetched. At the later part of upgrading, pg_upgrade revisits the list and restores slots by executing pg_create_logical_replication_slot() on the new cluster. Migration of logical replication slots is only supported when the old cluster is version 17.0 or later. If the old node has invalid slots or slots with unconsumed WAL records, the pg_upgrade fails. These checks are needed to prevent data loss. The significant advantage of this commit is that it makes it easy to continue logical replication even after upgrading the publisher node. Previously, pg_upgrade allowed copying publications to a new node. With this patch, adjusting the connection string to the new publisher will cause the apply worker on the subscriber to connect to the new publisher automatically. This enables seamless continuation of logical replication, even after an upgrade. Author: Hayato Kuroda, Hou Zhijie Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
* Doc: remove misleading info about ecpg's CONNECT/DISCONNECT DEFAULT.Tom Lane2023-10-25
| | | | | | | | | | | | | | | | | | | | | | | | | As far as I can see, ecpg has no notion of a "default" open connection. You can do "CONNECT TO DEFAULT" but that just specifies letting libpq use all its default connection parameters --- the resulting connection is not special subsequently. In particular, SET CONNECTION = DEFAULT and DISCONNECT DEFAULT simply act on a connection named DEFAULT, if you've made one; they do not have special lookup rules. But the documentation of these commands makes it look like they do. Simplest fix, I think, is just to remove the paras suggesting that DEFAULT is special here. Also, SET CONNECTION *does* have one special lookup rule, which is that it recognizes CURRENT as an alias for the currently selected connection. SET CONNECTION = CURRENT is a no-op, so it's pretty useless, but nonetheless it does something different from selecting a connection by name; so we'd better document it. Per report from Sylvain Frandaz. Back-patch to all supported versions. Discussion: https://postgr.es/m/169824721149.1769274.1553568436817652238@wrigleys.postgresql.org
* Remove dead code in pg_ctl.c.Nathan Bossart2023-10-25
| | | | | | | Missed in 39969e2a1e. Author: David Steele Discussion: https://postgr.es/m/0c742f0c-d663-419d-b5a7-4fe867f5566c%40pgmasters.net
* Doc fix: Interfacing Extensions to IndexesJeff Davis2023-10-25
| | | | | | | | | Refer to CREATE ACCESS METHOD rather than suggesting direct changes to pg_am. Also corrects index-specific language that predated table access methods. Discussion: https://postgr.es/m/20231025172551.685b7799455f9a6addcf5afa@sraoss.co.jp Reported-by: Yugo NAGATA <nagata@sraoss.co.jp>
* Fix some regression tests for d3d55ce57136Alexander Korotkov2023-10-25
| | | | Add missing (cost off) to explain.
* Remove useless self-joinsAlexander Korotkov2023-10-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Self Join Elimination (SJE) feature removes an inner join of a plain table to itself in the query tree if is proved that the join can be replaced with a scan without impacting the query result. Self join and inner relation are replaced with the outer in query, equivalence classes, and planner info structures. Also, inner restrictlist moves to the outer one with removing duplicated clauses. Thus, this optimization reduces the length of the range table list (this especially makes sense for partitioned relations), reduces the number of restriction clauses === selectivity estimations, and potentially can improve total planner prediction for the query. The SJE proof is based on innerrel_is_unique machinery. We can remove a self-join when for each outer row: 1. At most one inner row matches the join clause. 2. Each matched inner row must be (physically) the same row as the outer one. In this patch we use the next approach to identify a self-join: 1. Collect all merge-joinable join quals which look like a.x = b.x 2. Add to the list above the baseretrictinfo of the inner table. 3. Check innerrel_is_unique() for the qual list. If it returns false, skip this pair of joining tables. 4. Check uniqueness, proved by the baserestrictinfo clauses. To prove the possibility of self-join elimination inner and outer clauses must have an exact match. The relation replacement procedure is not trivial and it is partly combined with the one, used to remove useless left joins. Tests, covering this feature, were added to join.sql. Some regression tests changed due to self-join removal logic. Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru Author: Andrey Lepikhov, Alexander Kuzmenkov Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina Reviewed-by: Alexander Korotkov
* Use snprintf instead of sprintf in pg_regress.Daniel Gustafsson2023-10-25
| | | | | | | | To avoid static analyzers sounding the alarm, move to using snprintf instead of sprintf. This was an oversight in 66d6086cbcbfc8dee789a6. Reported-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/849588.1698179694@sss.pgh.pa.us
* Add missing include dir and references to libpq for MSVC buildDavid Rowley2023-10-25
| | | | | 66d6086cb adjusted pg_regress to require this but forgot to adjust the Visual Studio build script.
* doc: Fix some typos and grammarMichael Paquier2023-10-25
| | | | | | | Author: Ekaterina Kiryanova, Elena Indrupskaya, Oleg Sibiryakov, Maxim Yablokov Discussion: https://postgr.es/m/7aad518b-3e6d-47f3-9184-b1d69cb412e7@postgrespro.ru Backpatch-through: 11
* Log OpenSSL version in ./configure outputMichael Paquier2023-10-25
| | | | | | | | | | | | This information is useful to know when scanning buildfarm results, and it is already displayed in Meson. The output of `openssl version` is logged, with the command retrieved from PATH. This depends on c8e4030d1bdd, so backpatch down to 16. Reviewed-by: Peter Eisentraut, Daniel Gustafsson, Tom Lane Discussion: https://postgr.es/m/ZTW9yOlZaSVoFhTz@paquier.xyz Backpatch-through: 16
* Reword memory terminology for PQresultMemorySizeDaniel Gustafsson2023-10-24
| | | | | | | | | | | | | Rather than using the generic word "space" we might as well use "memory" since that's precisely what we're dealing with here. This was extracted from a larger patch around terminology changes where the remaining hunks were rejected. Author: Gurjeet Singh <gurjeet@singh.im> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/CABwTF4UHO_NtcsOL6_XZfnpKg_0XBFKa7B-7_x5zs3MRZm3-Tg@mail.gmail.com
* Speed up pg_regress server readiness testing.Daniel Gustafsson2023-10-24
| | | | | | | | | | | | | | | Instead of connecting to the server with psql to check if it is ready for running tests, this changes pg_regress to use PQPing which avoids performing system() calls which are expensive on some platforms, like Windows. The frequency of tests is also increased in order to connect to the server faster. This patch is part of a larger effort to make testing consume fewer resources in order to be able to fit more tests into the available CI system constraints. Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/20230823192239.jxew5s3sjru63lio@awork3.anarazel.de
* Fix problems when a plain-inheritance parent table is excluded.Tom Lane2023-10-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an UPDATE/DELETE/MERGE's target table is an old-style inheritance tree, it's possible for the parent to get excluded from the plan while some children are not. (I believe this is only possible if we can prove that a CHECK ... NO INHERIT constraint on the parent contradicts the query WHERE clause, so it's a very unusual case.) In such a case, ExecInitModifyTable mistakenly concluded that the first surviving child is the target table, leading to at least two bugs: 1. The wrong table's statement-level triggers would get fired. 2. In v16 and up, it was possible to fail with "invalid perminfoindex 0 in RTE with relid nnnn" due to the child RTE not having permissions data included in the query plan. This was hard to reproduce reliably because it did not occur unless the update triggered some non-HOT index updates. In v14 and up, this is easy to fix by defining ModifyTable.rootRelation to be the parent RTE in plain inheritance as well as partitioned cases. While the wrong-triggers bug also appears in older branches, the relevant code in both the planner and executor is quite a bit different, so it would take a good deal of effort to develop and test a suitable patch. Given the lack of field complaints about the trigger issue, I'll desist for now. (Patching v11 for this seems unwise anyway, given that it will have no more releases after next month.) Per bug #18147 from Hans Buschmann. Amit Langote and Tom Lane Discussion: https://postgr.es/m/18147-6fc796538913ee88@postgresql.org
* Doc: indexUnchanged is strictly a hint.Peter Geoghegan2023-10-24
| | | | | | | | | | | | | | Clearly spell out the limitations of aminsert()'s indexUnchanged hinting mechanism in the index AM documentation. Oversight in commit 9dc718bd, which added the "logically unchanged index" hint (which is used to trigger bottom-up index deletion). Author: Peter Geoghegan <pg@bowt.ie> Reported-By: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/CAH2-WzmU_BQ=-H9L+bxTSMQBqHMjp1DSwGypvL0gKs+dTOfkKg@mail.gmail.com Backpatch: 14-, where indexUnchanged hinting was introduced.
* Assert that buffers are marked dirty before XLogRegisterBuffer().Jeff Davis2023-10-23
| | | | | | | | | | | Enforce the rule from transam/README in XLogRegisterBuffer(), and update callers to follow the rule. Hash indexes sometimes register clean pages as a part of the locking protocol, so provide a REGBUF_NO_CHANGE flag to support that use. Discussion: https://postgr.es/m/c84114f8-c7f1-5b57-f85a-3adc31e1a904@iki.fi Reviewed-by: Heikki Linnakangas
* doc: Improve example query related to pg_wait_eventsMichael Paquier2023-10-24
| | | | | Author: Pavel Luzanov Discussion: https://postgr.es/m/4f79be75-7e30-4817-b0da-4a691ea5427f@postgrespro.ru
* Fix typos in wait_event.cMichael Paquier2023-10-24
| | | | Noticed while working on a different patch. Introduced in af720b4c50a1.
* Change struct tablespaceinfo's oid member from 'char *' to 'Oid'Robert Haas2023-10-23
| | | | | | | | | | | | | | | | | | | | | | | | | | This shouldn't change behavior except in the unusual case where there are file in the tablespace directory that have entirely numeric names but are nevertheless not possible names for a tablespace directory, either because their names have leading zeroes that shouldn't be there, or the value is actually zero, or because the value is too large to represent as an OID. In those cases, the directory would previously have made it into the list of tablespaceinfo objects and no longer will. Thus, base backups will now ignore such directories, instead of treating them as legitimate tablespace directories. Similarly, if entries for such tablespaces occur in a tablespace_map file, they will now be rejected as erroneous, instead of being honored. This is infrastructure for future work that wants to be able to know the tablespace of each relation that is part of a backup *as an OID*. By strengthening the up-front validation, we don't have to worry about weird cases later, and can more easily avoid repeated string->integer conversions. Patch by me, reviewed by David Steele. Discussion: http://postgr.es/m/CA+TgmoZNVeBzoqDL8xvr-nkaepq815jtDR4nJzPew7=3iEuM1g@mail.gmail.com
* Refactor parse_filename_for_nontemp_relation to parse more.Robert Haas2023-10-23
| | | | | | | | | | | | | | | | | | | | | | | | | | Instead of returning the number of characters in the RelFileNumber, return the RelFileNumber itself. Continue to return the fork number, as before, and additionally return the segment number. parse_filename_for_nontemp_relation now rejects a RelFileNumber or segment number that begins with a leading zero. Before, we accepted such cases as relation filenames, but if we continued to do so after this change, the function might return the same values for two different files (e.g. 1234.5 and 001234.5 or 1234.005) which could be annoying for callers. Since we don't actually ever generate filenames with leading zeroes in the names, any such files that we find must have been created by something other than PostgreSQL, and it is therefore reasonable to treat them as non-relation files. Along the way, change unlogged_relation_entry to store a RelFileNumber rather than an OID. This update should have been made in 851f4cc75cdd8c831f1baa9a7abf8c8248b65890, but it was overlooked. It's trivial to make the update as part of this commit, perhaps more trivial than it would have been without it, so do that. Patch by me, reviewed by David Steele. Discussion: http://postgr.es/m/CA+TgmoZNVeBzoqDL8xvr-nkaepq815jtDR4nJzPew7=3iEuM1g@mail.gmail.com
* Remove unnecessary break in pg_logical_replication_slot_advance()Michael Paquier2023-10-23
| | | | | | | | | | | | | | | pg_logical_replication_slot_advance() included a break condition to stop when a targeted LSN is reached, when processing a series of WAL records with XLogReadRecord(). Since 38a957316d7e, it matched with the check of its main while loop. This condition saved from an extra CFI check, actually pointless, so let's remove this condition and simplify the code. In passing, fix an incorrect comment. Author: Bharath Rupireddy Reviewed-by: Tom Lane, Gurjeet Singh Discussion: https://postgr.es/m/CALj2ACWfGDLQ2cy7ZKwxnJqbDkO6Yvqqrqxne5ZN4HYm=PRTGg@mail.gmail.com
* doc: Fix some grammar and inconsistent tagsMichael Paquier2023-10-23
| | | | | | Author: Ekaterina Kiryanova, Elena Indrupskaya, Oleg Sibiryakov, Maxim Yablokov Discussion: https://postgr.es/m/4c2a430b-32e2-44e2-aeca-03b7db6824e4@postgrespro.ru
* Log LLVM library version in configure output.Thomas Munro2023-10-22
| | | | | | | | | | When scanning build farm results, it's useful to be able to see which version is in use. For the Meson build system, this information was already displayed. Back-patch to all supported branches. Discussion: https://postgr.es/m/4022690.1697852728%40sss.pgh.pa.us
* Fix min_dynamic_shared_memory on Windows.Thomas Munro2023-10-22
| | | | | | | | | | | | | | | | | | When min_dynamic_shared_memory is set above 0, we try to find space in a pre-allocated region of the main shared memory area instead of calling dsm_impl_XXX() routines to allocate more. The dsm_pin_segment() and dsm_unpin_segment() routines had a bug: they called dsm_impl_XXX() routines even for main region segments. Nobody noticed before now because those routines do nothing on Unix, but on Windows they'd fail while attempting to duplicate an invalid Windows HANDLE. Add the missing gating. Back-patch to 14, where commit 84b1c63a added this feature. Fixes pgsql-bugs bug #18165. Reported-by: Maxime Boyer <maxime.boyer@cra-arc.gc.ca> Tested-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/18165-bf4f525cea6e51de%40postgresql.org
* Allow ALTER SYSTEM to set unrecognized custom GUCs.Tom Lane2023-10-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, ALTER SYSTEM failed if the target GUC wasn't present in the session's GUC hashtable. That is a reasonable behavior for core (single-part) GUC names, and for custom GUCs for which we have loaded an extension that's reserved the prefix. But it's unnecessarily restrictive otherwise, and it also causes inconsistent behavior: you can "ALTER SYSTEM SET foo.bar" only if you did "SET foo.bar" earlier in the session. That's fairly silly. Hence, refactor things so that we can execute ALTER SYSTEM even if the variable doesn't have a GUC hashtable entry, as long as the name meets the custom-variable naming requirements and does not have a reserved prefix. (It's safe to do this even if the variable belongs to an extension we currently don't have loaded. A bad value will at worst cause a WARNING when the extension does get loaded.) Also, adjust GRANT ON PARAMETER to have the same opinions about whether to allow an unrecognized GUC name, and to throw the same errors if not (it previously used a one-size-fits-all message for several distinguishable conditions). By default, only a superuser will be allowed to do ALTER SYSTEM SET on an unrecognized name, but it's possible to GRANT the ability to do it. Patch by me, pursuant to a documentation complaint from Gavin Panella. Arguably this is a bug fix, but given the lack of other complaints I'll refrain from back-patching. Discussion: https://postgr.es/m/2617358.1697501956@sss.pgh.pa.us Discussion: https://postgr.es/m/169746329791.169914.16613647309012285391@wrigleys.postgresql.org
* Make some error strings more genericAlvaro Herrera2023-10-20
| | | | | It's undesirable to have SQL commands or configuration options in a translatable error string, so take some of these out.
* meson: Make detection of python more robustAndres Freund2023-10-20
| | | | | | | | | | | Previously we errored out if no python installation could be found (but we did handle not having enough of python installed to build plpython against). Presumably nobody hit this so far, as python is likely installed due to meson requiring python. Author: Tristan Partin <tristan@neon.tech> Discussion: https://postgr.es/m/CSPIJVUDZFKX.3KHMOAVGF94RV@c3po Backpatch: 16-, where meson support was added
* Doc: update CREATE OPERATOR's statement about => as an operator.Tom Lane2023-10-20
| | | | | | | This doco said that use of => as an operator "is deprecated". It's been fully disallowed since 865f14a2d back in 9.5, but evidently that commit missed updating this statement. Do so now.
* Extend ALTER OPERATOR to allow setting more optimization attributes.Tom Lane2023-10-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allow the COMMUTATOR, NEGATOR, MERGES, and HASHES attributes to be set by ALTER OPERATOR. However, we don't allow COMMUTATOR/NEGATOR to be changed once set, nor allow the MERGES/HASHES flags to be unset once set. Changes like that might invalidate plans already made, and dealing with the consequences seems like more trouble than it's worth. The main use-case we foresee for this is to allow addition of missed properties in extension update scripts, such as extending an existing operator to support hashing. So only transitions from not-set to set states seem very useful. This patch also causes us to reject some incorrect cases that formerly resulted in inconsistent catalog state, such as trying to set the commutator of an operator to be some other operator that already has a (different) commutator. While at it, move the InvokeObjectPostCreateHook call for CREATE OPERATOR to not occur until after we've fixed up commutator or negator links as needed. The previous ordering could only be justified by thinking of the OperatorUpd call as a kind of ALTER OPERATOR step; but we don't call InvokeObjectPostAlterHook therein. It seems better to let the hook see the final state of the operator object. In the documentation, move the discussion of how to establish commutator pairs from xoper.sgml to the CREATE OPERATOR ref page. Tommy Pavlicek, reviewed and editorialized a bit by me Discussion: https://postgr.es/m/CAEhP-W-vGVzf4udhR5M8Bdv88UYnPrhoSkj3ieR3QNrsGQoqdg@mail.gmail.com
* Remove unnecessary dependencies to wal_level=logical in TAP testsMichael Paquier2023-10-20
| | | | | | | | | | | | | A couple of TAP tests make use of wal_level=logical for nodes that do not need to do any kind of logical decoding operations, like subscription nodes on which changes are only applied. This can be confusing when reading these tests as setup examples, so let's remove this configuration where not required (contrary to two-way logical replication and similar more complex cases). This simplifies the tests a bit, making them slightly cheaper with less WAL generated overall. Author: Hayato Kuroda Discussion: https://postgr.es/m/TYAPR01MB5866946BCEB747ABE513ACC6F5D5A@TYAPR01MB5866.jpnprd01.prod.outlook.com
* During online checkpoints, insert XLOG_CHECKPOINT_REDO at redo point.Robert Haas2023-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | This allows tools that read the WAL sequentially to identify (possible) redo points when they're reached, rather than only being able to detect them in retrospect when XLOG_CHECKPOINT_ONLINE is found, possibly much later in the WAL stream. There are other possible applications as well; see the discussion links below. Any redo location that precedes the checkpoint location should now point to an XLOG_CHECKPOINT_REDO record, so add a cross-check to verify this. While adjusting the code in CreateCheckPoint() for this patch, I made it call WALInsertLockAcquireExclusive a bit later than before, since there appears to be no need for it to be held while checking whether the system is idle, whether this is an end-of-recovery checkpoint, or what the current timeline is. Bump XLOG_PAGE_MAGIC. Patch by me, based in part on earlier work from Dilip Kumar. Review by Dilip Kumar, Amit Kapila, Andres Freund, and Michael Paquier. Discussion: http://postgr.es/m/CA+TgmoYy-Vc6G9QKcAKNksCa29cv__czr+N9X_QCxEfQVpp_8w@mail.gmail.com Discussion: http://postgr.es/m/20230614194717.jyuw3okxup4cvtbt%40awork3.anarazel.de Discussion: http://postgr.es/m/CA+hUKG+b2ego8=YNW2Ohe9QmSiReh1-ogrv8V_WZpJTqP3O+2w@mail.gmail.com
* Doc: modernize comment for boolin().Tom Lane2023-10-19
| | | | | | | | | | Most of the behavior described by this comment was moved to parse_bool_with_len() some time ago. Move what's still valuable there too, and drop the rest. Peter Smith Discussion: https://postgr.es/m/CAHut+PtMJURKp=U8Z=Ktp0zV40sEb1f-iEk9FvY2GQe+5ZBnwg@mail.gmail.com
* pg_stat_statements: Add local_blk_{read|write}_timeMichael Paquier2023-10-19
| | | | | | | | | | | | | | This commit adds to pg_stat_statements the two new fields for local buffers introduced by 295c36c0c1fa, adding the time spent to read and write these blocks. These are similar to what is done for temp and shared blocks. This information available only if track_io_timing is enabled. Like for 5a3423ad8ee17, no version bump is required in the module. Author: Nazir Bilal Yavuz Reviewed-by: Robert Haas, Melanie Plageman Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
* Add local_blk_{read|write}_time I/O timing statistics for local blocksMichael Paquier2023-10-19
| | | | | | | | | | | | | There was no I/O timing statistics for counting read and write timings on local blocks, contrary to the counterparts for temp and shared blocks. This information is available when track_io_timing is enabled. The output of EXPLAIN is updated to show this information. An update of pg_stat_statements is planned next. Author: Nazir Bilal Yavuz Reviewed-by: Robert Haas, Melanie Plageman Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
* Rename I/O timing statistics columns to shared_blk_{read|write}_timeMichael Paquier2023-10-19
| | | | | | | | | | | | | | | | | | | | These two counters, defined in BufferUsage to track respectively the time spent while reading and writing blocks have historically only tracked data related to shared buffers, when track_io_timing is enabled. An upcoming patch to add specific counters for local buffers will take advantage of this rename as it has come up that no data is currently tracked for local buffers, and tracking local and shared buffers using the same fields would be inconsistent with the treatment done for temp buffers. Renaming the existing fields clarifies what the block type of each stats field is. pg_stat_statement is updated to reflect the rename. No extension version bump is required as 5a3423ad8ee17 has done one, affecting v17~. Author: Nazir Bilal Yavuz Reviewed-by: Robert Haas, Melanie Plageman Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
* Improve pglz_decompress's defenses against corrupt compressed data.Tom Lane2023-10-18
| | | | | | | | | | | | | | | | When processing a match tag, check to see if the claimed "off" is more than the distance back to the output buffer start. If it is, then the data is corrupt, and what's more we would fetch from outside the buffer boundaries and potentially incur a SIGSEGV. (Although the odds of that seem relatively low, given that "off" can't be more than 4K.) Back-patch to v13; before that, this function wasn't really trying to protect against bad data. Report and fix by Flavien Guedez. Discussion: https://postgr.es/m/01fc0593-e31e-463d-902c-dd43174acee2@oopacity.net
* Install wait_event_types.h in VPATH buildsMichael Paquier2023-10-19
| | | | | | | | | | | An extra rule is needed in src/include/Makefile for VPATH builds to install any generated server-side include files, and wait_event_types.h was forgotten from the set. Issue introduced by fa88928470b5. Reported-by: Christoph Berg Discussion: https://postgr.es/m/ZTAA11u7CtX6NqlK@msg.df7cb.de
* jit: Changes for LLVM 17.Thomas Munro2023-10-19
| | | | | | | | | | | | Changes required by https://llvm.org/docs/NewPassManager.html. Back-patch to 12, leaving the final release of 11 unchanged, consistent with earlier decision not to back-patch LLVM 16 support either. Author: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKG%2BWXznXCyTgCADd%3DHWkP9Qksa6chd7L%3DGCnZo-MBgg9Lg%40mail.gmail.com
* jit: Supply LLVMGlobalGetValueType() for LLVM < 8.Thomas Munro2023-10-19
| | | | | | | | | | | Commit 37d5babb used this C API function while adding support for LLVM 16 and opaque pointers, but it's not available in LLVM 7 and older. Provide it in our own llvmjit_wrap.cpp. It just calls a C++ function that pre-dates LLVM 3.9, our minimum target. Back-patch to 12, like 37d5babb. Discussion: https://postgr.es/m/CA%2BhUKGKnLnJnWrkr%3D4mSGhE5FuTK55FY15uULR7%3Dzzc%3DwX4Nqw%40mail.gmail.com
* jit: Support opaque pointers in LLVM 16.Thomas Munro2023-10-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove use of LLVMGetElementType() and provide the type of all pointers to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM versions[1]. * For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions. * For LLVM == 15, we'll continue to do the same, explicitly opting out of opaque pointer mode. * For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take the extra type argument. The difference is hidden behind some new IR emitting wrapper functions l_load(), l_gep(), l_call() etc. The change is mostly mechanical, except that at each site the correct type had to be provided. In some places we needed to do some extra work to get functions types, including some new wrappers for C++ APIs that are not yet exposed by in LLVM's C API, and some new "example" functions in llvmjit_types.c because it's no longer possible to start from the function pointer type and ask for the function type. Back-patch to 12, because it's a little tricker in 11 and we agreed not to put the latest LLVM support into the upcoming final release of 11. [1] https://llvm.org/docs/OpaquePointers.html Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
* pg_upgrade: Fix test name in 002_pg_upgrade.plMichael Paquier2023-10-18
| | | | | | Author: Hou Zhijie Discussion: https://postgr.es/m/TYAPR01MB5724A40D47E71F4717357EC694D5A@TYAPR01MB5724.jpnprd01.prod.outlook.com Backpatch-through: 15
* Clean up some pg_dump testsPeter Eisentraut2023-10-18
| | | | | | | | | | | | | | | | | 1) Remove useless entries from "unlike" lists. Runs that are not listed in "like" don't need to be excluded in "unlike". 2) Ensure there is always a "like" list, even if it is empty. This makes the test more self-documenting. 3) Use predefined lists such as %full_runs where appropriate, instead of listing all runs separately. Also add code that checks 1 and 2 automatically and dies with an error for violations. Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://www.postgresql.org/message-id/flat/1f8cb371-e84e-434e-0367-6b716fb16fa1@eisentraut.org
* Count write times when extending relation files for shared buffersMichael Paquier2023-10-18
| | | | | | | | | | | | | | | | | | Relation files extended by multiple blocks at a time have been counting the number of blocks written, but forgot to increment the write time in this case, as single-block write and relation extension are treated as two different I/O operations in the shared stats: IOOP_EXTEND vs IOOP_WRITE. In this case IOOP_EXTEND was forgotten for normal (non-temporary) relations, still the number of blocks written was incremented according to the relation extend done. Write times are tracked when track_io_timing is enabled, which is not the case by default. Author: Nazir Bilal Yavuz Reviewed-by: Robert Haas, Melanie Plageman Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com Backpatch-through: 16
* Add flush option to pg_logical_emit_message()Michael Paquier2023-10-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | Since its introduction, LogLogicalMessage() (via the SQL interface pg_logical_emit_message()) has never included a call to XLogFlush(), causing it to potentially lose messages on a crash when used in non-transactional mode. This has come up to me as a problem while playing with ideas to design a test suite for what has become 039_end_of_wal.pl introduced in bae868caf222 by Thomas Munro, because there are no direct ways to force a WAL flush via SQL. The default is false, to not flush messages and influence existing use-cases where this function could be used. If set to true, the message emitted is flushed before returning back to the caller, making the message durable on crash. This new option has no effect when using pg_logical_emit_message() in transactional mode, as the record's flush is guaranteed by the WAL record generated by the transaction committed. Two queries of test_decoding are tweaked to cover the new code path for the flush. Bump catalog version. Author: Michael Paquier Reviewed-by: Andres Freund, Amit Kapila, Fujii Masao, Tung Nguyen, Tomas Vondra Discussion: https://postgr.es/m/ZNsdThSe2qgsfs7R@paquier.xyz
* Dodge a compiler bug affecting timetz_zone/timetz_izone.Tom Lane2023-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | Use a modulo operator instead of implementing the same behavior with a loop. The loop solution is doubtless microscopically faster for the typical case of only wrapping into the very next day, but maybe not so much for large interval values. In any case, timetz is such a backwater that it's doubtful anybody would notice any performance change anyway. This avoids a compiler bug occurring in AIX's xlc, even in pretty late-model revisions. We did not have test coverage for the case where the initial result->time value is negative, so add that. For the moment, install this only in HEAD. My plan is to back-patch the test case, and then the code change assuming that buildfarm testing proves the bug occurs in the back branches. (That seems pretty likely, but let's find out for sure.) Per buildfarm results from commits 97957fdba and 2f0472030. Thanks to Michael Paquier for the idea to use a modulo operation to replace the faulty loop. Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
* Avoid calling proc_exit() in processes forked by system().Nathan Bossart2023-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The SIGTERM handler for the startup process immediately calls proc_exit() for the duration of the restore_command, i.e., a call to system(). This system() call forks a new process to execute the shell command, and this child process inherits the parent's signal handlers. If both the parent and child processes receive SIGTERM, both will attempt to call proc_exit(). This can end badly. For example, both processes will try to remove themselves from the PGPROC shared array. To fix this problem, this commit adds a check in StartupProcShutdownHandler() to see whether MyProcPid == getpid(). If they match, this is the parent process, and we can proc_exit() like before. If they do not match, this is a child process, and we just emit a message to STDERR (in a signal safe manner) and _exit(), thereby skipping any problematic exit callbacks. This commit also adds checks in proc_exit(), ProcKill(), and AuxiliaryProcKill() that verify they are not being called within such child processes. Suggested-by: Andres Freund Reviewed-by: Thomas Munro, Andres Freund Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13 Backpatch-through: 11
* Reword messages about impending (M)XID exhaustion.Robert Haas2023-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | First, we shouldn't recommend switching to single-user mode, because that's terrible advice. Especially on newer versions where VACUUM will enter emergency mode when nearing (M)XID exhaustion, it's perfectly fine to just VACUUM in multi-user mode. Doing it that way is less disruptive and avoids disabling the safeguards that prevent actual wraparound, so recommend that instead. Second, be more precise about what is going to happen (when we're nearing the limits) or what is happening (when we actually hit them). The database doesn't shut down, nor does it refuse all commands. It refuses commands that assign whichever of XIDs and MXIDs are nearly exhausted. No back-patch. The existing hint that advises going to single-user mode is sufficiently awful advice that removing it or changing it might be justifiable even though we normally avoid changing user-facing messages in back-branches, but I (rhaas) felt that it was better to be more conservative and limit this fix to master only. Aside from the usual risk of breaking translations, people might be used to the existing message, or even have monitoring scripts that look for it. Alexander Alekseev, John Naylor, Robert Haas, reviewed at various times by Peter Geoghegan, Hannu Krosing, and Andres Freund. Discussion: http://postgr.es/m/CA+TgmoZBg95FiR9wVQPAXpGPRkacSt2okVge+PKPPFppN7sfnQ@mail.gmail.com
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-04 18:55:41 +0000