aboutsummaryrefslogtreecommitdiff
path: root/src/backend/utils
Commit message (Collapse)AuthorAge
...
* Use more consistent names for wait event objects and typesMichael Paquier2023-09-06
| | | | | | | | | | | | | | | | | The event names use the same case-insensitive characters, hence applying lower() or upper() to the monitoring queries allows the detection of the same events as before this change. It is possible to cross-check the data with the system view pg_wait_events, for instance, with a query like that showing no differences: SELECT lower(type), lower(name), description FROM pg_wait_events ORDER BY 1, 2; This will help in the introduction of more simplifications in the format of wait_event_names. Some of the enum values in the code had to be renamed a bit to follow the same convention naming across the board. Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/ZOxVHQwEC/9X/p/z@paquier.xyz
* Remove the "snapshot too old" feature.Thomas Munro2023-09-05
| | | | | | | | | | | | | | | | | Remove the old_snapshot_threshold setting and mechanism for producing the error "snapshot too old", originally added by commit 848ef42b. Unfortunately it had a number of known problems in terms of correctness and performance, mostly reported by Andres in the course of his work on snapshot scalability. We agreed to remove it, after a long period without an active plan to fix it. This is certainly a desirable feature, and someone might propose a new or improved implementation in the future. Reported-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CACG%3DezYV%2BEvO135fLRdVn-ZusfVsTY6cH1OZqWtezuEYH6ciQA%40mail.gmail.com Discussion: https://postgr.es/m/20200401064008.qob7bfnnbu4w5cw4%40alap3.anarazel.de Discussion: https://postgr.es/m/CA%2BTgmoY%3Daqf0zjTD%2B3dUWYkgMiNDegDLFjo%2B6ze%3DWtpik%2B3XqA%40mail.gmail.com
* Improve description of keys in tsvectorMichael Paquier2023-09-05
| | | | | | | | | | | | | | | | | If all the bits of a key in a tsvector are true (marked with ALLISTRUE), gtsvectorout() would show the following description: "0 true bits, 0 false bits" This is confusing, as all the bits are true, but this would be equivalent to the information if siglen is 0. This commit improves the output so as "all true bits" show instead in this case. Alexander has proposed a regression test for pageinspect, not included here as it is rather expensive compared to its coverage value. Author: Alexander Lakhin Discussion: https://postgr.es/m/17950-6c80a8d2b94ec695@postgresql.org
* Fix out-of-bound read in gtsvector_picksplit()Michael Paquier2023-09-04
| | | | | | | | | | | | | | This could lead to an imprecise choice when splitting an index page of a GiST index on a tsvector, deciding which entries should remain on the old page and which entries should move to a new page. This is wrong since tsearch2 has been moved into core with commit 140d4ebcb46e, so backpatch all the way down. This error has been spotted by valgrind. Author: Alexander Lakhin Discussion: https://postgr.es/m/17950-6c80a8d2b94ec695@postgresql.org Backpatch-through: 11
* Fix handling of shared statistics with dropped databasesMichael Paquier2023-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Dropping a database while a connection is attempted on it was able to lead to the presence of valid database entries in shared statistics. The issue is that MyDatabaseId was getting set too early than it should, as, if the connection attempted on the dropped database fails when renamed or dropped, the shutdown callback of the shared statistics would finish by re-inserting a correct entry related to the database already dropped. As analyzed by the bug reporters, this issue could lead to phantom entries in the database list maintained by the autovacuum launcher (in rebuild_database_list()) if the database dropped was part of the database list when it was still valid. After the database was dropped, it would remain the highest on the list of databases to considered by the autovacuum worker as things to process. This would prevent autovacuum jobs to happen on all the other databases still present. The commit fixes this issue by delaying setting MyDatabaseId until the database existence has been re-checked with the second scan on pg_database after getting a shared lock on it, and by switching pgstat_update_dbstats() so as nothing happens if MyDatabaseId is not valid. Issue introduced by 5891c7a8ed8f, so backpatch down to 15. Reported-by: Will Mortensen, Jacob Speidel Analyzed-by: Will Mortensen, Jacob Speidel Author: Andres Freund Discussion: https://postgr.es/m/17973-bca1f7d5c14f601e@postgresql.org Backpatch-through: 15
* Use actual backend IDs in pg_stat_get_backend_subxact().Nathan Bossart2023-08-30
| | | | | | | | | | | | | | | | | | | | | | Unlike the other pg_stat_get_backend* functions, pg_stat_get_backend_subxact() looks up the backend entry by using its integer argument as a 1-based index in an internal array. The other functions look for the entry with the matching session backend ID. These numbers often match, but that isn't reliably true. This commit resolves this discrepancy by introducing pgstat_get_local_beentry_by_backend_id() and using it in pg_stat_get_backend_subxact(). We cannot use pgstat_get_beentry_by_backend_id() because it returns a PgBackendStatus, which lacks the locally computed additions available in LocalPgBackendStatus that are required by pg_stat_get_backend_subxact(). Author: Ian Barwick Reviewed-by: Sami Imseih, Michael Paquier, Robert Haas Discussion: https://postgr.es/m/CAB8KJ%3Dj-ACb3H4L9a_b3ZG3iCYDW5aEu3WsPAzkm2S7JzS1Few%40mail.gmail.com Backpatch-through: 16
* Rename some support functions for pgstat* views.Nathan Bossart2023-08-30
| | | | | | | | | | | | | | | | | | | | | Presently, pgstat_fetch_stat_beentry() accepts a session's backend ID as its argument, and pgstat_fetch_stat_local_beentry() accepts a 1-based index in an internal array as its argument. The former is typically used wherever a user must provide a backend ID, and the latter is usually used internally when looping over all entries in the array. This difference was first introduced by d7e39d72ca. Before that commit, both functions accepted a 1-based index to the internal array. This commit renames these two functions to make it clear whether they use the backend ID or the 1-based index to look up the entry. This is preparatory work for a follow-up change that will introduce a function for looking up a LocalPgBackendStatus using a backend ID. Reviewed-by: Ian Barwick, Sami Imseih, Michael Paquier, Robert Haas Discussion: https://postgr.es/m/CAB8KJ%3Dj-ACb3H4L9a_b3ZG3iCYDW5aEu3WsPAzkm2S7JzS1Few%40mail.gmail.com Backpatch-through: 16
* Avoid possible overflow with ltsGetFreeBlock() in logtape.cMichael Paquier2023-08-30
| | | | | | | | | | | | | | | | | | nFreeBlocks, defined as a long, stores the number of free blocks in a logical tape. ltsGetFreeBlock() has been using an int to store the value of nFreeBlocks, which could lead to overflows on platforms where long and int are not the same size (in short everything except Windows where long is 4 bytes). The problematic intermediate variable is switched to be a long instead of an int. Issue introduced by c02fdc9223015, so backpatch down to 13. Author: Ranier vilela Reviewed-by: Peter Geoghegan, David Rowley Discussion: https://postgr.es/m/CAEudQApLDWCBR_xmwNjGBrDo+f+S4E87x3s7-+hoaKqYdtC4JQ@mail.gmail.com Backpatch-through: 13
* Rename logical_replication_mode to debug_logical_replication_streamingPeter Eisentraut2023-08-29
| | | | | | | | | | | | | | The logical_replication_mode GUC is intended for testing and debugging purposes, but its current name may be misleading and encourage users to make unnecessary changes. To avoid confusion, renaming the GUC to a less misleading name debug_logical_replication_streaming that casual users are less likely to mistakenly assume needs to be modified in a regular logical replication setup. Author: Hou Zhijie <houzj.fnst@cn.fujitsu.com> Reviewed-by: Peter Smith <smithpb2250@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/d672d774-c44b-6fec-f993-793e744f169a%40eisentraut.org
* Make error messages about WAL segment size more consistentPeter Eisentraut2023-08-28
| | | | | | | | | | | | | | | | Make the primary messages more compact and make the detail messages uniform. In initdb.c and pg_resetwal.c, use the newish option_parse_int() to simplify some of the option parsing. For the backend GUC wal_segment_size, add a GUC check hook to do the verification instead of coding it in bootstrap.c. This might be overkill, but that way the check is in the right place and it becomes more self-documenting. In passing, make pg_controldata use the logging API for warning messages. Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Discussion: https://www.postgresql.org/message-id/flat/9939aa8a-d7be-da2c-7715-0a0b5535a1f7@eisentraut.org
* Fix some typos in wait_event_names.txtMichael Paquier2023-08-28
| | | | | Noticed in passing, while hacking on a different patch touching this area.
* Tighten unit parsing in internal valuesMichael Paquier2023-08-28
| | | | | | | | | | | | | | Interval values now generate an error when the user has multiple consecutive units or a unit without a value. Previously, it was possible to specify multiple units consecutively which is contrary to what the documentation allows, so it was possible to finish with confusing interval values. This is a follow-up of the work done in 165d581f146b. Author: Joseph Koshakow Reviewed-by: Jacob Champion, Gurjeet Singh, Reid Thompson Discussion: https://postgr.es/m/CAAvxfHd-yNO+XYnUxL=GaNZ1n+eE0V-oE0+-cC1jdjdU0KS3iw@mail.gmail.com
* Tighten handling of "ago" in interval valuesMichael Paquier2023-08-28
| | | | | | | | | | | | | | This commit Restrict the unit "ago" to only appear at the end of the interval. According to the documentation, a direction can only be defined at the end of an interval, but it was possible to define it in the middle of the string or define it multiple times. In spirit, this is similar to the error handling improvements done in 5b3c5953553b or bcc704b524904. Author: Joseph Koshakow Reviewed-by: Jacob Champion, Gurjeet Singh, Reid Thompson Discussion: https://postgr.es/m/CAAvxfHd-yNO+XYnUxL=GaNZ1n+eE0V-oE0+-cC1jdjdU0KS3iw@mail.gmail.com
* Remove dead code in DecodeInterval()Michael Paquier2023-08-28
| | | | | | | | | | | | This commit removes some dead code related to the unit type RESERV, whose last use has been removed from the unit lookup table used for intervals ("deltatktbl" in datetime.c) in 666cbae16da4. Before that, RESERV was used as an equivalent of "invalid", but that's now unreachable. Author: Joseph Koshakow Reviewed-by: Jacob Champion, Gurjeet Singh, Reid Thompson Discussion: https://postgr.es/m/CAAvxfHd-yNO+XYnUxL=GaNZ1n+eE0V-oE0+-cC1jdjdU0KS3iw@mail.gmail.com
* Catalog not-null constraintsAlvaro Herrera2023-08-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We now create contype='n' pg_constraint rows for not-null constraints. We propagate these constraints to other tables during operations such as adding inheritance relationships, creating and attaching partitions and creating tables LIKE other tables. We also spawn not-null constraints for inheritance child tables when their parents have primary keys. These related constraints mostly follow the well-known rules of conislocal and coninhcount that we have for CHECK constraints, with some adaptations: for example, as opposed to CHECK constraints, we don't match not-null ones by name when descending a hierarchy to alter it, instead matching by column name that they apply to. This means we don't require the constraint names to be identical across a hierarchy. For now, we omit them for system catalogs. Maybe this is worth reconsidering. We don't support NOT VALID nor DEFERRABLE clauses either; these can be added as separate features later (this patch is already large and complicated enough.) psql shows these constraints in \d+. pg_dump requires some ad-hoc hacks, particularly when dumping a primary key. We now create one "throwaway" not-null constraint for each column in the PK together with the CREATE TABLE command, and once the PK is created, all those throwaway constraints are removed. This avoids having to check each tuple for nullness when the dump restores the primary key creation. pg_upgrading from an older release requires a somewhat brittle procedure to create a constraint state that matches what would be created if the database were being created fresh in Postgres 17. I have tested all the scenarios I could think of, and it works correctly as far as I can tell, but I could have neglected weird cases. This patch has been very long in the making. The first patch was written by Bernd Helmle in 2010 to add a new pg_constraint.contype value ('n'), which I (Álvaro) then hijacked in 2011 and 2012, until that one was killed by the realization that we ought to use contype='c' instead: manufactured CHECK constraints. However, later SQL standard development, as well as nonobvious emergent properties of that design (mostly, failure to distinguish them from "normal" CHECK constraints as well as the performance implication of having to test the CHECK expression) led us to reconsider this choice, so now the current implementation uses contype='n' again. During Postgres 16 this had already been introduced by commit e056c557aef4, but there were some problems mainly with the pg_upgrade procedure that couldn't be fixed in reasonable time, so it was reverted. In 2016 Vitaly Burovoy also worked on this feature[1] but found no consensus for his proposed approach, which was claimed to be closer to the letter of the standard, requiring an additional pg_attribute column to track the OID of the not-null constraint for that column. [1] https://postgr.es/m/CAKOSWNkN6HSyatuys8xZxzRCR-KL1OkHS5-b9qd9bf1Rad3PLA@mail.gmail.com Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Bernd Helmle <mailings@oopsware.de> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
* Avoid unnecessary plancache revalidation of utility statements.Tom Lane2023-08-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Revalidation of a plancache entry (after a cache invalidation event) requires acquiring a snapshot. Normally that is harmless, but not if the cached statement is one that needs to run without acquiring a snapshot. We were already aware of that for TransactionStmts, but for some reason hadn't extrapolated to the other statements that PlannedStmtRequiresSnapshot() knows mustn't set a snapshot. This can lead to unexpected failures of commands such as SET TRANSACTION ISOLATION LEVEL. We can fix it in the same way, by excluding those command types from revalidation. However, we can do even better than that: there is no need to revalidate for any statement type for which parse analysis, rewrite, and plan steps do nothing interesting, which is nearly all utility commands. To mechanize this, invent a parser function stmt_requires_parse_analysis() that tells whether parse analysis does anything beyond wrapping a CMD_UTILITY Query around the raw parse tree. If that's what it does, then rewrite and plan will just skip the Query, so that it is not possible for the same raw parse tree to produce a different plan tree after cache invalidation. stmt_requires_parse_analysis() is basically equivalent to the existing function analyze_requires_snapshot(), except that for obscure reasons that function omits ReturnStmt and CallStmt. It is unclear whether those were oversights or intentional. I have not been able to demonstrate a bug from not acquiring a snapshot while analyzing these commands, but at best it seems mighty fragile. It seems safer to acquire a snapshot for parse analysis of these commands too, which allows making stmt_requires_parse_analysis and analyze_requires_snapshot equivalent. In passing this fixes a second bug, which is that ResetPlanCache would exclude ReturnStmts and CallStmts from revalidation. That's surely *not* safe, since they contain parsable expressions. Per bug #18059 from Pavel Kulakov. Back-patch to all supported branches. Discussion: https://postgr.es/m/18059-79c692f036b25346@postgresql.org
* Rename hook functions for debug_io_direct to match variable name.Peter Eisentraut2023-08-24
| | | | | | | | | | Commit 319bae9a renamed the GUC. Rename the check and assign functions to match, and alphabetize. Back-patch to 16. Author: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/2769341e-fa28-c2ee-3e4b-53fdcaaf2271%40eisentraut.org
* Add to_bin() and to_oct().Nathan Bossart2023-08-23
| | | | | | | | | | | This commit introduces functions for converting numbers to their equivalent binary and octal representations. Also, the base conversion code for these functions and to_hex() has been moved to a common helper function. Co-authored-by: Eric Radman Reviewed-by: Ian Barwick, Dag Lem, Vignesh C, Tom Lane, Peter Eisentraut, Kirk Wolak, Vik Fearing, John Naylor, Dean Rasheed Discussion: https://postgr.es/m/Y6IyTQQ/TsD5wnsH%40vm3.eradman.com
* Introduce macros for protocol characters.Nathan Bossart2023-08-22
| | | | | | | | | | | This commit introduces descriptively-named macros for the identifiers used in wire protocol messages. These new macros are placed in a new header file so that they can be easily used by third-party code. Author: Dave Cramer Reviewed-by: Alvaro Herrera, Tatsuo Ishii, Peter Smith, Robert Haas, Tom Lane, Peter Eisentraut, Michael Paquier Discussion: https://postgr.es/m/CADK3HHKbBmK-PKf1bPNFoMC%2BoBt%2BpD9PH8h5nvmBQskEHm-Ehw%40mail.gmail.com
* Fix pg_stat_reset_single_table_counters() for shared relationsMichael Paquier2023-08-21
| | | | | | | | | | | | This commit fixes the function of $subject for shared relations. This feature has been added by e042678. Unfortunately, this new behavior got removed by 5891c7a when moving statistics to shared memory. Reported-by: Mitsuru Hinata Author: Masahiro Ikeda Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada Discussion: https://postgr.es/m/7cc69f863d9b1bc677544e3accd0e4b4@oss.nttdata.com Backpatch-through: 15
* Add system view pg_wait_eventsMichael Paquier2023-08-20
| | | | | | | | | | | | | | | | | | | | | | This new view, wrapped around a SRF, shows some information known about wait events, as of: - Name. - Type (Activity, I/O, Extension, etc.). - Description. All the information retrieved comes from wait_event_names.txt, and the description is the same as the documentation with filters applied to remove any XML markups. This view is useful when joined with pg_stat_activity to get the description of a wait event reported. Custom wait events for extensions are included in the view. Original idea by Yves Colin. Author: Bertrand Drouvot Reviewed-by: Kyotaro Horiguchi, Masahiro Ikeda, Tom Lane, Michael Paquier Discussion: https://postgr.es/m/0e2ae164-dc89-03c3-cf7f-de86378053ac@gmail.com
* Fix format if entry in wait_event_names.txtMichael Paquier2023-08-18
| | | | | | | | | The entry LockManager had two successive whitespaces between two words. This is not an actual bug, but let's be clean. Thinko in fa88928. Reported-by: Masahiro Ikeda Author: Bertrand Drouvot Discussion: https://postgr.es/m/dd836027-2e9e-4df9-9fd9-7527cd1757e1@gmail.com
* Unify some error messagesPeter Eisentraut2023-08-16
| | | | | We had essentially the same error in several different wordings. Unify that.
* Split out tiebreaker comparisons from comparetup_* functionsJohn Naylor2023-08-16
| | | | | | | | | | | | | | | Previously, if a specialized comparator found equal datum1 keys, the "comparetup" function would repeat the comparison on the datum before proceeding with the unabbreviated first key and/or additional sort keys. Move comparing additional sort keys into "tiebreak" functions so that specialized comparators can call these directly if needed, avoiding duplicate work. Reviewed by David Rowley Discussion: https://postgr.es/m/CAFBsxsGaVfUrjTghpf%3DkDBYY%3DjWx1PN-fuusVe7Vw5s0XqGdGw%40mail.gmail.com
* Change custom wait events to use dynamic shared hash tablesMichael Paquier2023-08-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the names of the custom wait event must be registered for each backend, requiring all these to link to the shared memory area of an extension, even if these are not loaded with shared_preload_libraries. This patch relaxes the constraints related to this infrastructure by storing the wait events and their names in two dynamic hash tables in shared memory. This has the advantage to simplify the registration of custom wait events to a single routine call that returns an event ID ready for consumption: uint32 WaitEventExtensionNew(const char *wait_event_name); The caller of this routine can then cache locally the ID returned, to be used for pgstat_report_wait_start(), WaitLatch() or a similar routine. The implementation uses two hash tables: one with a key based on the event name to avoid duplicates and a second using the event ID as key for event lookups, like on pg_stat_activity. These tables can hold a minimum of 16 entries, and a maximum of 128 entries, which should be plenty enough. The code changes done in worker_spi show how things are simplified (most of the code removed in this commit comes from there): - worker_spi_init() is gone. - No more shared memory hooks required (size requested and initialization). - The custom wait event ID is cached in the process that needs to set it, with one single call to WaitEventExtensionNew() to retrieve it. Per suggestion from Andres Freund. Author: Masahiro Ikeda, with a few tweaks from me. Discussion: https://postgr.es/m/20230801032349.aaiuvhtrcvvcwzcx@awork3.anarazel.de
* Fix code indentation violations introduced by recent commitMichael Paquier2023-08-11
| | | | | | | The two culprit commits are 5765cfe and 5e0c761. Per buildfarm member koel for the first commit, while I have noticed the second one in passing.
* Transform proconfig for faster execution.Jeff Davis2023-08-10
| | | | | | | | | | | | Store function config settings in lists to avoid the need to parse and allocate for each function execution. Speedup is modest but significant. Additionally, this change also seems cleaner and supports some other performance improvements under discussion. Discussion: https://postgr.es/m/04c8592dbd694e4114a3ed87139a7a04e4363030.camel@j-davis.com Reviewed-by: Nathan Bossart
* Document RelationGetIndexAttrBitmap betterAlvaro Herrera2023-08-10
| | | | | | | | | | | | Commit 19d8e2308bc5 changed the list of set-of-columns that can be returned by RelationGetIndexAttrBitmap, but didn't update its "documentation". That was pretty hard to read already, so rewrite to make it more comprehensible, adding the missing values while at it. Backpatch to 16, like that commit. Discussion: https://postgr.es/m/20230809091155.7c7f3gttjk3dj4ze@alvherre.pgsql Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
* doc: Fix incorrect entries generated from wait_event_names.txtMichael Paquier2023-08-08
| | | | | | | | | | | | fa88928 has introduced wait_event_names.txt, and some of its entries had some documentation fields with more information than necessary. This commit brings back the description of all the wait events to be consistent with the older stable branches. Five descriptions were incorrect. Author: Bertrand Drouvot Discussion: https://postgr.es/m/e378989e-1899-643a-dec1-10f691a0a105@gmail.com
* Fix performance regression in pg_strtointNN_safe functionsDavid Rowley2023-08-02
| | | | | | | | | | | | | | | | | | | | | Between 6fcda9aba and 1b6f632a3, the pg_strtoint functions became quite a bit slower in v16, despite efforts in 6b423ec67 to speed these up. Since the majority of cases for these functions will only contain base-10 digits, perhaps prefixed by a '-', it makes sense to have a special case for this and just fall back on the more complex version which processes hex, octal, binary and underscores if the fast path version fails to parse the string. While we're here, update the header comments for these functions to mention that hex, octal and binary formats along with underscore separators are now supported. Author: Andres Freund, David Rowley Reported-by: Masahiko Sawada Reviewed-by: Dean Rasheed, John Naylor Discussion: https://postgr.es/m/CAD21AoDvDmUQeJtZrau1ovnT_smN940%3DKp6mszNGK3bq9yRN6g%40mail.gmail.com Backpatch-through: 16, where 6fcda9aba and 1b6f632a3 were added
* Fix overly strict Assert in jsonpath codeDavid Rowley2023-08-02
| | | | | | | | | | | | | This was failing for queries which try to get the .type() of a jpiLikeRegex. For example: select jsonb_path_query('["string", "string"]', '($[0] like_regex ".{7}").type()'); Reported-by: Alexander Kozhemyakin Bug: #18035 Discussion: https://postgr.es/m/18035-64af5cdcb5adf2a9@postgresql.org Backpatch-through: 12, where SQL/JSON path was added.
* Rename OverrideSearchPath to SearchPathMatcher.Noah Misch2023-07-31
| | | | | | | | | | The previous commit removed the "override" APIs. Surviving APIs facilitate plancache.c to snapshot search_path and test whether the current value equals a remembered snapshot. Aleksander Alekseev. Reported by Alexander Lakhin and Noah Misch. Discussion: https://postgr.es/m/8ffb4650-52c4-6a81-38fc-8f99be981130@gmail.com
* Support custom wait events for wait event type "Extension"Michael Paquier2023-07-31
| | | | | | | | | | | | | | | | | | | | | | | Two backend routines are added to allow extension to allocate and define custom wait events, all of these being allocated in the type "Extension": * WaitEventExtensionNew(), that allocates a wait event ID computed from a counter in shared memory. * WaitEventExtensionRegisterName(), to associate a custom string to the wait event ID allocated. Note that this includes an example of how to use this new facility in worker_spi with tests in TAP for various scenarios, and some documentation about how to use them. Any code in the tree that currently uses WAIT_EVENT_EXTENSION could switch to this new facility to define custom wait events. This is left as work for future patches. Author: Masahiro Ikeda Reviewed-by: Andres Freund, Michael Paquier, Tristan Partin, Bharath Rupireddy Discussion: https://postgr.es/m/b9f5411acda0cf15c8fbb767702ff43e@oss.nttdata.com
* Add WAIT_EVENT_{CLASS,ID}_MASK in wait_event.cMichael Paquier2023-07-31
| | | | | | | | These are useful to extract the class ID and the event ID associated to a single uint32 wait_event_info. Only two code paths use them now, but an upcoming patch will extend their use. Discussion: https://postgr.es/m/ZMcJ7F7nkGkIs8zP@paquier.xyz
* Add sanity asserts for index OID and attnums during cache initMichael Paquier2023-07-27
| | | | | | | | | | | | | | | | There was already a check on the relation OID, but not its index OID and the attributes that can be used during the syscache lookups. The two assertions added by this commit are cheap, and actually useful for developers to fasten the detection of incorrect data in a new entry added in the syscache list, as these assertions are triggered during the initial cache loading (initdb, or just backend startup), not requiring a syscache that uses the new entry. While on it, the relation OID check is switched to use OidIsValid(). Author: Aleksander Alekseev Reviewed-by: Dagfinn Ilmari Mannsåker, Zhang Mingli, Michael Paquier Discussion: https://postgr.es/m/CAJ7c6TOjUTJ0jxvWY6oJeP2-840OF8ch7qscZQsuVuotXTOS_g@mail.gmail.com
* Add more SQL/JSON constructor functionsAmit Langote2023-07-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This Patch introduces three SQL standard JSON functions: JSON() JSON_SCALAR() JSON_SERIALIZE() JSON() produces json values from text, bytea, json or jsonb values, and has facilitites for handling duplicate keys. JSON_SCALAR() produces a json value from any scalar sql value, including json and jsonb. JSON_SERIALIZE() produces text or bytea from input which containis or represents json or jsonb; For the most part these functions don't add any significant new capabilities, but they will be of use to users wanting standard compliant JSON handling. Catversion bumped as this changes ruleutils.c. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera, Peter Eisentraut Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
* Some refactoring to export json(b) conversion functionsAmit Langote2023-07-26
| | | | | | | | | | | This is to export datum_to_json(), datum_to_jsonb(), and jsonb_from_cstring(), though the last one is exported as jsonb_from_text(). A subsequent commit to add new SQL/JSON constructor functions will need them for calling from the executor. Discussion: https://postgr.es/m/20230720160252.ldk7jy6jqclxfxkq%40alvherre.pgsql
* Guard against null plan pointer in CachedPlanIsSimplyValid().Tom Lane2023-07-20
| | | | | | | | | | | | | If both the passed-in plan pointer and plansource->gplan are NULL, CachedPlanIsSimplyValid would think that the plan pointer is possibly-valid and try to dereference it. For the one extant call site in plpgsql, this situation doesn't normally happen which is why we've not noticed. However, it appears to be possible if the previous use of the cached plan failed, as per report from Justin Pryzby. Add an extra check to prevent crashing. Back-patch to v13 where this code was added. Discussion: https://postgr.es/m/ZLlV+STFz1l/WhAQ@telsasoft.com
* Revert "Add notBefore and notAfter to SSL cert info display"Daniel Gustafsson2023-07-20
| | | | | | | Due to an oversight in reviewing, this used functionality not compatible with old versions of OpenSSL. This reverts commit 75ec5e7bec700577d39d653c316e3ae6c505842c.
* Add notBefore and notAfter to SSL cert info displayDaniel Gustafsson2023-07-20
| | | | | | | | | This adds the X509 attributes notBefore and notAfter to sslinfo as well as pg_stat_ssl to allow verifying and identifying the validity period of the current client certificate. Author: Cary Huang <cary.huang@highgo.ca> Discussion: https://postgr.es/m/182b8565486.10af1a86f158715.2387262617218380588@highgo.ca
* Unify JSON categorize type API and export for external useAmit Langote2023-07-20
| | | | | | | | | | | | | | | | | | | | | | This essentially removes the JsonbTypeCategory enum and jsonb_categorize_type() and integrates any jsonb-specific logic that was in jsonb_categorize_type() into json_categorize_type(), now moved to jsonfuncs.c. The remaining JsonTypeCategory enum and json_categorize_type() cover the needs of the callers in both json.c and jsonb.c. json_categorize_type() has grown a new parameter named is_jsonb for callers to engage the jsonb-specific behavior of json_categorize_type(). One notable change in the now exported API of json_categorize_type() is that it now always returns *outfuncoid even though a caller may have no need currently to see one. This is in preparation of later commits to implement additional SQL/JSON functions. Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
* Add missing ObjectIdGetDatum() in syscache lookup calls for OidsMichael Paquier2023-07-20
| | | | | | | | | | Based on how postgres.h foes the Oid <-> Datum conversion, there is no existing bugs but let's be consistent. 17 spots have been noticed as incorrectly passing down Oids rather than Datums. Aleksander got one, Zhang two and I the rest. Author: Michael Paquier, Aleksander Alekseev, Zhang Mingli Discussion: https://postgr.es/m/ZLUhqsqQN1MOaxdw@paquier.xyz
* Remove db_user_namespace.Nathan Bossart2023-07-17
| | | | | | | | | | | This feature was intended to be a temporary measure to support per-database user names. A better one hasn't materialized in the ~21 years since it was added, and nobody claims to be using it, so let's just remove it. Reviewed-by: Michael Paquier, Magnus Hagander Discussion: https://postgr.es/m/20230630200509.GA2830328%40nathanxps13 Discussion: https://postgr.es/m/20230630215608.GD2941194%40nathanxps13
* Shrink memory contexts struct sizesDavid Rowley2023-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | Here we reduce the block size fields in AllocSetContext, GenerationContext and SlabContext from Size down to uint32. Ever since c6e0fe1f2, blocks for non-dedicated palloc chunks can no longer be larger than 1GB, so there's no need to store the various block size fields as 64-bit values. 32 bits are enough to store 2^30. Here we also further reduce the memory context struct sizes by getting rid of the 'keeper' field which stores a pointer to the context's keeper block. All the context types which have this field always allocate the keeper block in the same allocation as the memory context itself, so the keeper block always comes right at the end of the context struct. Add some macros to calculate that address rather than storing it in the context. Overall, in AllocSetContext and GenerationContext, this saves 20 bytes on 64-bit builds which for ALLOCSET_SMALL_SIZES can sometimes mean the difference between having to allocate a 2nd block and storing all the required allocations on the keeper block alone. Such contexts are used in relcache to store cache entries for indexes, of which there can be a large number in a single backend. Author: Melih Mutlu Reviewed-by: David Rowley Discussion: https://postgr.es/m/CAGPVpCSOW3uJ1QJmsMR9_oE3X7fG_z4q0AoU4R_w+2RzvroPFg@mail.gmail.com
* Improve error message for MaxAllocSize overrun in accumArrayResult.Tom Lane2023-07-14
| | | | | | | | | | | | | | | Before, if you went past about 64M array elements in array_agg() and allied functions, you got a generic "invalid memory alloc request size" error. This patch replaces that with "array size exceeds the maximum allowed", which seems more user-friendly since it points you to needing to reduce the size of your array result. (This is the same error text you'd get from construct_md_array in the event of overrunning the maximum physical size for the finished array.) Per question from Shaozhong Shi. Since this hasn't come up often, I don't feel a need to back-patch. Discussion: https://postgr.es/m/CA+i5JwYtVS9z2E71PcNKAVPbOn4R2wuj-LqbJsYr_XOz73q7dQ@mail.gmail.com
* Fix privilege check for SET SESSION AUTHORIZATION.Nathan Bossart2023-07-13
| | | | | | | | | | | | | | | | | | | Presently, the privilege check for SET SESSION AUTHORIZATION checks whether the original authenticated role was a superuser at connection start time. Even if the role loses the superuser attribute, its existing sessions are permitted to change session authorization to any role. This commit modifies this privilege check to verify the original authenticated role currently has superuser. In the event that the authenticated role loses superuser within a session authorization change, the authorization change will remain in effect, which means the user can still take advantage of the target role's privileges. However, [RE]SET SESSION AUTHORIZATION will only permit switching to the original authenticated role. Author: Joseph Koshakow Discussion: https://postgr.es/m/CAAvxfHc-HHzONQ2oXdvhFF9ayRnidPwK%2BfVBhRzaBWYYLVQL-g%40mail.gmail.com
* Move privilege check for SET SESSION AUTHORIZATION.Nathan Bossart2023-07-13
| | | | | | | | | | | | | | | | Presently, the privilege check for SET SESSION AUTHORIZATION is performed in session_authorization's assign_hook. A relevant comment states, "It's OK because the check does not require catalog access and can't fail during an end-of-transaction GUC reversion..." However, we plan to add a catalog lookup to this privilege check in a follow-up commit. This commit moves this privilege check to the check_hook for session_authorization. Like check_role(), we do not throw a hard error for insufficient privileges when the source is PGC_S_TEST. Author: Joseph Koshakow Discussion: https://postgr.es/m/CAAvxfHc-HHzONQ2oXdvhFF9ayRnidPwK%2BfVBhRzaBWYYLVQL-g%40mail.gmail.com
* Allow the use of a hash index on the subscriber during replication.Amit Kapila2023-07-14
| | | | | | | | | | | | | | | Commit 89e46da5e5 allowed using BTREE indexes that are neither PRIMARY KEY nor REPLICA IDENTITY on the subscriber during apply of update/delete. This patch extends that functionality to also allow HASH indexes. We explored supporting other index access methods as well but they don't have a fixed strategy for equality operation which is required by the current infrastructure in logical replication to scan the indexes. Author: Kuroda Hayato Reviewed-by: Peter Smith, Onder Kalaci, Amit Kapila Discussion: https://postgr.es/m/TYAPR01MB58669D7414E59664E17A5827F522A@TYAPR01MB5866.jpnprd01.prod.outlook.com
* Add indisreplident to fields refreshed by RelationReloadIndexInfo()Michael Paquier2023-07-14
| | | | | | | | | | | | | | | | | | | RelationReloadIndexInfo() is a fast-path used for index reloads in the relation cache, and it has always forgotten about updating indisreplident, which is something that would happen after an index is selected for a replica identity. This can lead to incorrect cache information provided when executing a command in a transaction context that updates indisreplident. None of the code paths currently on HEAD that need to check upon pg_index.indisreplident fetch its value from the relation cache, always relying on a fresh copy on the syscache. Unfortunately, this may not be the case of out-of-core code, that could see out-of-date value. Author: Shruthi Gowda Reviewed-by: Robert Haas, Dilip Kumar, Michael Paquier Discussion: https://postgr.es/m/CAASxf_PBcxax0wW-3gErUyftZ0XrCs3Lrpuhq4-Z3Fak1DoW7Q@mail.gmail.com Backpatch-through: 11
* Add information about line contents on parsing failure of wait_event_names.txtMichael Paquier2023-07-14
| | | | | | | | | The contents of the line whose parsing failed was not reported in the error message produced by generate-wait_event_types.pl, making harder than necessary the debugging of incorrectly-shaped entries in the file. Reported-by: Andres Freund Discussion: https://postgr.es/m/ZK9S3jFEV1X797Ll@paquier.xyz