aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* Fix typo in comment.Etsuro Fujita2019-11-27
|
* Avoid assertion failure with LISTEN in a serializable transaction.Tom Lane2019-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If LISTEN is the only action in a serializable-mode transaction, and the session was not previously listening, and the notify queue is not empty, predicate.c reported an assertion failure. That happened because we'd acquire the transaction's initial snapshot during PreCommit_Notify, which was called *after* predicate.c expects any such snapshot to have been established. To fix, just swap the order of the PreCommit_Notify and PreCommit_CheckForSerializationFailure calls during CommitTransaction. This will imply holding the notify-insertion lock slightly longer, but the difference could only be meaningful in serializable mode, which is an expensive option anyway. It appears that this is just an assertion failure, with no consequences in non-assert builds. A snapshot used only to scan the notify queue could not have been involved in any serialization conflicts, so there would be nothing for PreCommit_CheckForSerializationFailure to do except assign it a prepareSeqNo and set the SXACT_FLAG_PREPARED flag. And given no conflicts, neither of those omissions affect the behavior of ReleasePredicateLocks. This admittedly once-over-lightly analysis is backed up by the lack of field reports of trouble. Per report from Mark Dilger. The bug is old, so back-patch to all supported branches; but the new test case only goes back to 9.6, for lack of adequate isolationtester infrastructure before that. Discussion: https://postgr.es/m/3ac7f397-4d5f-be8e-f354-440020675694@gmail.com Discussion: https://postgr.es/m/13881.1574557302@sss.pgh.pa.us
* Defend against self-referential views in relation_is_updatable().Tom Lane2019-11-21
| | | | | | | | | | | | | | | | | | | | | While a self-referential view doesn't actually work, it's possible to create one, and it turns out that this breaks some of the information_schema views. Those views call relation_is_updatable(), which neglected to consider the hazards of being recursive. In older PG versions you get a "stack depth limit exceeded" error, but since v10 it'd recurse to the point of stack overrun and crash, because commit a4c35ea1c took out the expression_returns_set() call that was incidentally checking the stack depth. Since this function is only used by information_schema views, it seems like it'd be better to return "not updatable" than suffer an error. Hence, add tracking of what views we're examining, in just the same way that the nearby fireRIRrules() code detects self-referential views. I added a check_stack_depth() call too, just to be defensive. Per private report from Manuel Rigger. Back-patch to all supported versions.
* Revise GIN READMEAlexander Korotkov2019-11-20
| | | | | | | | | | | | | | | | | We find GIN concurrency bugs from time to time. One of the problems here is that concurrency of GIN isn't well-documented in README. So, it might be even hard to distinguish design bugs from implementation bugs. This commit revised concurrency section in GIN README providing more details. Some examples are illustrated in ASCII art. Also, this commit add the explanation of how is tuple layout in internal GIN B-tree page different in comparison with nbtree. Discussion: https://postgr.es/m/CAPpHfduXR_ywyaVN4%2BOYEGaw%3DcPLzWX6RxYLBncKw8de9vOkqw%40mail.gmail.com Author: Alexander Korotkov Reviewed-by: Peter Geoghegan Backpatch-through: 9.4
* Fix traversing to the deleted GIN page via downlinkAlexander Korotkov2019-11-20
| | | | | | | | | | | | | | | Current GIN code appears to don't handle traversing to the deleted page via downlink. This commit fixes that by stepping right from the delete page like we do in nbtree. This commit also fixes setting 'deleted' flag to the GIN pages. Now other page flags are not erased once page is deleted. That helps to keep our assertions true if we arrive deleted page via downlink. Discussion: https://postgr.es/m/CAPpHfdvMvsw-NcE5bRS7R1BbvA4BxoDnVVjkXC5W0Czvy9LVrg%40mail.gmail.com Author: Alexander Korotkov Reviewed-by: Peter Geoghegan Backpatch-through: 9.4
* Further fix dumping of views that contain just VALUES(...).Tom Lane2019-11-16
| | | | | | | | | | | | | | | | | | | | | | | It turns out that commit e9f1c01b7 missed a case: we must print a VALUES clause in long format if get_query_def is given a resultDesc that would require the query's output column name(s) to be different from what the bare VALUES clause would produce. This applies in case an ALTER ... RENAME COLUMN has been done to a view that formerly could be printed in simple format, as shown in the added regression test case. It also explains bug #16119 from Dmitry Telpt, because it turns out that (unlike CREATE VIEW) CREATE MATERIALIZED VIEW fails to apply any column aliases it's given to the stored ON SELECT rule. So to get them to be printed, we have to account for the resultDesc renaming. It might be worth changing the matview code so that it creates the ON SELECT rule with the correct aliases; but we'd still need these messy checks in get_simple_values_rte to handle the case of a subsequent column rename, so any such change would be just neatnik-ism not a bug fix. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/16119-e64823f30a45a754@postgresql.org
* Handle arrays and ranges in pg_upgrade's test for non-upgradable types.Tom Lane2019-11-13
| | | | | | | | | | | | | | | | | | | | | | | | pg_upgrade needs to check whether certain non-upgradable data types appear anywhere on-disk in the source cluster. It knew that it has to check for these types being contained inside domains and composite types; but it somehow overlooked that they could be contained in arrays and ranges, too. Extend the existing recursive-containment query to handle those cases. We probably should have noticed this oversight while working on commit 0ccfc2822 and follow-ups, but we failed to :-(. The whole thing's possibly a bit overdesigned, since we don't really expect that any of these types will appear on disk; but if we're going to the effort of doing a recursive search then it's silly not to cover all the possibilities. While at it, refactor so that we have only one copy of the search logic, not three-and-counting. Also, to keep the branches looking more alike, back-patch the output wording change of commit 1634d3615. Back-patch to all supported branches. Discussion: https://postgr.es/m/31473.1573412838@sss.pgh.pa.us
* Stamp 9.5.20.REL9_5_20Tom Lane2019-11-11
|
* Translation updatesPeter Eisentraut2019-11-11
| | | | | Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: ccaca304fd66160be942ccf657be787511659ac1
* Remove test case that doesn't work with --disable-integer-timestamps.Tom Lane2019-11-09
| | | | | | | | | | Commits a7145f6bc et al. added a test to verify integer overflow detection in interval_mul. That only applies with integer timestamps, of course, so it's problematic in pre-v10 branches where we supported float timestamps. The test was only marginally worth the trouble to begin with, so just remove it in those branches. Per buildfarm. Discussion: https://postgr.es/m/6437.1573319193@sss.pgh.pa.us
* Fix gratuitous error message variationPeter Eisentraut2019-11-08
|
* Move declaration of ecpg_gettext() to a saner place.Tom Lane2019-11-07
| | | | | | | | | | | | | | | | | | | | | Declaring this in the client-visible header ecpglib.h was a pretty poor decision. It's not meant to be application-callable (and if it was, putting it outside the extern "C" { ... } wrapper means that C++ clients would fail to call it). And the declaration would not even compile for a client, anyway, since it would not have the macro pg_attribute_format_arg(). Fortunately, it seems that no clients have tried to include this header with ENABLE_NLS defined, or we'd have gotten complaints about that. But we have no business putting such a restriction on client code. Move the declaration to ecpglib_extern.h, since in fact nothing outside src/interfaces/ecpg/ecpglib/ needs to call it. The practical effect of this is just that clients can now safely #include ecpglib.h while having ENABLE_NLS defined, but that seems like enough of a reason to back-patch it. Discussion: https://postgr.es/m/20590.1573069709@sss.pgh.pa.us
* Fix integer-overflow edge case detection in interval_mul and pgbench.Tom Lane2019-11-07
| | | | | | | | | | | | | | | | | | This patch adopts the overflow check logic introduced by commit cbdb8b4c0 into two more places. interval_mul() failed to notice if it computed a new microseconds value that was one more than INT64_MAX, and pgbench's double-to-int64 logic had the same sorts of edge-case problems that cbdb8b4c0 fixed in the core code. To make this easier to get right in future, put the guts of the checks into new macros in c.h, and add commentary about how to use the macros correctly. Back-patch to all supported branches, as we did with the previous fix. Yuya Watari Discussion: https://postgr.es/m/CAJ2pMkbkkFw2hb9Qb1Zj8d06EhWAQXFLy73St4qWv6aX=vqnjw@mail.gmail.com
* Fix assertion failure when running pgbench -s.Fujii Masao2019-11-07
| | | | | | | | | | | | | | | | | | If there is the WAL page that the continuation WAL record just fits within (i.e., the continuation record ends just at the end of the page) and the LSN in such page is specified with -s option, previously pg_waldump caused an assertion failure. The cause of this assertion failure was that XLogFindNextRecord() that pg_waldump -s calls mistakenly handled such special WAL page. This commit changes XLogFindNextRecord() so that it can handle such WAL page correctly. Back-patch to all supported versions. Author: Andrey Lepikhov Reviewed-by: Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/99303554-5dd5-06e6-f943-b3005ccd6edd@postgrespro.ru
* Fix timestamp of sent message for write context in logical decodingMichael Paquier2019-11-06
| | | | | | | | | | | | | | | | | | | When sending data for logical decoding using the streaming replication protocol via a WAL sender, the timestamp of the sent write message is allocated at the beginning of the message when preparing for the write, and actually computed when the write message is ready to be sent. The timestamp was getting computed after sending the message. This impacts anything using logical decoding, causing for example logical replication to report mostly NULL for last_msg_send_time in pg_stat_subscription. This commit makes sure that the timestamp is computed before sending the message. This is wrong since 5a991ef, so backpatch down to 9.4. Author: Jeff Janes Discussion: https://postgr.es/m/CAMkU=1z=WMn8jt7iEdC5sYNaPgAgOASb_OW5JYv-vMdYaJSL-w@mail.gmail.com Backpatch-through: 9.4
* Avoid logging complaints about abandoned connections when using PAM.Tom Lane2019-11-05
| | | | | | | | | | | | | | | | | | | For a long time (since commit aed378e8d) we have had a policy to log nothing about a connection if the client disconnects when challenged for a password. This is because libpq-using clients will typically do that, and then come back for a new connection attempt once they've collected a password from their user, so that logging the abandoned connection attempt will just result in log spam. However, this did not work well for PAM authentication: the bottom-level function pam_passwd_conv_proc() was on board with it, but we logged messages at higher levels anyway, for lack of any reporting mechanism. Add a flag and tweak the logic so that the case is silent, as it is for other password-using auth mechanisms. Per complaint from Yoann La Cancellera. It's been like this for awhile, so back-patch to all supported branches. Discussion: https://postgr.es/m/CACP=ajbrFFYUrLyJBLV8=q+eNCapa1xDEyvXhMoYrNphs-xqPw@mail.gmail.com
* Change pg_restore -f- to dump to stdout instead of to ./-Alvaro Herrera2019-11-05
| | | | | | | | | | | | Starting with PostgreSQL 12, pg_restore refuses to run when neither -d nor -f are specified (c.f. commit 413ccaa74d9a), and it also makes "-f -" mean the old implicit behavior of dumping to stdout. However, older branches write to a file called ./- when invoked like that, making it impossible to write pg_restore scripts that work across versions. This is a partial backpatch of the aforementioned commit to all older supported branches, providing an upgrade path. Discussion: https://postgr.es/m/20191006190839.GE18030@telsasoft.com
* Stabilize pg_dump output order for similarly-named triggers and policies.Tom Lane2019-11-04
| | | | | | | | | | | | | | | | | The code only compared two triggers' names and namespaces (the latter being the owning table's schema). This could result in falling back to an OID-based sort of similarly-named triggers on different tables. We prefer to avoid that, so add a comparison of the table names too. (The sort order is thus table namespace, trigger name, table name, which is a bit odd, but it doesn't seem worth contorting the code to work around that.) Likewise for policy objects, in 9.5 and up. Complaint and fix by Benjie Gillam. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAMThMzEEt2mvBbPgCaZ1Ap1N-moGn=Edxmadddjq89WG4NpPtQ@mail.gmail.com
* Catch invalid typlens in a couple of placesPeter Eisentraut2019-11-04
| | | | | | | | | | Rearrange the logic in record_image_cmp() and record_image_eq() to error out on unexpected typlens (either not supported there or completely invalid due to corruption). Barring corruption, this is not possible today but it seems more future-proof and robust to fix this. Reported-by: Peter Geoghegan <pg@bowt.ie>
* Fix race condition at backend exit when deleting element in syncrep queueMichael Paquier2019-11-01
| | | | | | | | | | | | | | | When a backend exits, it gets deleted from the syncrep queue if present. The queue was checked without SyncRepLock taken in exclusive mode, so it would have been possible for a backend to remove itself after a WAL sender already did the job. Fix this issue based on a suggestion from Fujii Masao, by first checking the queue without the lock. Then, if the backend is present in the queue, take the lock and perform an additional lookup check before doing the element deletion. Author: Dongming Liu Reviewed-by: Kyotaro Horiguchi, Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/a0806273-8bbb-43b3-bbe1-c45a58f6ae21.lingce.ldm@alibaba-inc.com Backpatch-through: 9.4
* pg_waldump: Fix --bkp-details to not issue spurious newlines for FPWs.Andres Freund2019-10-29
| | | | | | | | | | | | | | | | The additional newline seems to have accidentally been introduced in 2c03216d831, in 9.5. The newline is only issued when an FPW is present for the block reference. While there could be an argument that removing the newlines in the back branches could cause a problem for somebody parsing the pg_waldump output, the likelihood of that seems small enough. It seems at least equally likely that the randomness of when newlines are issued causes problems. Author: Andres Freund Discussion: https://postgr.es/m/20191029233341.4gnyau7e5v2lh5sc@alap3.anarazel.de Backpatch: 9.5, like 2c03216d831.
* pg_waldump: Fix small memory leak when rmgr->rm_identify returns NULL.Andres Freund2019-10-29
| | | | | | | | | This got broken in 604f7956b94, shortly after rm_identify's introduction. Author: Andres Freund Discussion: https://postgr.es/m/20191029233341.4gnyau7e5v2lh5sc@alap3.anarazel.de Backpatch: 9.5, where rm_identify was introduced
* Fix pg_dump's handling of circular dependencies in views.Tom Lane2019-10-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a back-patch of the v10 commit d8c05aff5. The motivation for doing this now is that we received a complaint that a view with a circular dependency is dumped with an extra bogus command "ALTER TABLE ONLY myview REPLICA IDENTITY NOTHING", because pg_dump forgets that it's a view not a table, and the relreplident value stored for a view is that. So you'll get an error message during restore even if not using --clean; this would break pg_upgrade for example. While that could be handled with a one-line patch, it seems better to back-patch d8c05aff5, since that produces cleaner more future-proof output and fixes an additional bug. Per gripe from Alex Williams. Back-patch to 9.4-9.6 (even if 9.3 were still in support, it hasn't got REPLICA IDENTITY so no bug). Discussion: https://postgr.es/m/NFqxoEi7-8Rw9OW0f-GwHcjvS2I4YQXov4g9OoWv3i7lVOZdLWkAWl9jQQqwEaUq6WV0vdobromhW82e8y5I0_59yZTXcZnXsrmFuldlmZc=@protonmail.com Original commit message follows: pg_dump's traditional solution for breaking a circular dependency involving a view was to create the view with CREATE TABLE and then later issue CREATE RULE "_RETURN" ... to convert the table to a view, relying on the backend's very very ancient code that supports making views that way. We've wanted to get rid of that kluge for a long time, but the thing that finally motivates doing something about it is the recognition that this method fails with the --clean option, because it leads to issuing DROP RULE "_RETURN" followed by DROP TABLE --- and the backend won't let you drop a view's _RETURN rule. Instead, let's break circular dependencies by initially creating the view using CREATE VIEW AS SELECT NULL::columntype AS columnname, ... (so that it has the right column names and types to support external references, but no dependencies beyond the column data types), and then later dumping the ON SELECT rule using the spelling CREATE OR REPLACE VIEW. This method wasn't available when this code was originally written, but it's been possible since PG 7.3, so it seems fine to start relying on it now. To solve the --clean problem, make the dropStmt for an ON SELECT rule be CREATE OR REPLACE VIEW with the same dummy target list as above. In this way, during the DROP phase, we first reduce the view to have no extra dependencies, and then we can drop it entirely when we've gotten rid of whatever had a circular dependency on it. (Note: this should work adequately well with the --if-exists option, since the CREATE OR REPLACE VIEW will go through whether the view exists or not. It could fail if the view exists with a conflicting column set, but we don't really support --clean against a non-matching database anyway.) This allows cleaning up some other kluges inside pg_dump, notably that we don't need a notion of reloptions attached to a rule anymore. Although this is a bug fix, commit to HEAD only for now. The problem's existed for a long time and we've had relatively few complaints, so it doesn't really seem worth taking risks to fix it in the back branches. We might revisit that choice if no problems emerge. Discussion: <19092.1479325184@sss.pgh.pa.us>
* Clean up properly error_context_stack in autovacuum worker on exceptionMichael Paquier2019-10-23
| | | | | | | | | | | | | | Any callback set would have no meaning in the context of an exception. As an autovacuum worker exits quickly in this context, this could be only an issue within EmitErrorReport(), where the elog hook is for example called. That's unlikely to going to be a problem, but let's be clean and consistent with other code paths handling exceptions. This is present since 2909419, which introduced autovacuum. Author: Ashwin Agrawal Reviewed-by: Tom Lane, Michael Paquier Discussion: https://postgr.es/m/CALfoeisM+_+dgmAdAOHAu0k-ZpEHHqSSG=GRf3pKJGm8OqWX0w@mail.gmail.com Backpatch-through: 9.4
* Deal with yet another issue related to "Norwegian (Bokmål)" locale.Tom Lane2019-10-21
| | | | | | | | | | | | | It emerges that recent versions of Windows (at least 2016 Standard) spell this locale name as "Norwegian Bokmål_Norway.1252", defeating our mapping code that translates "Norwegian (Bokmål)_Norway" to something that's all-ASCII (cf commits db29620d4 and aa1d2fc5e). Add another mapping entry to handle this spelling. Per bug #16068 from Robert Ford. Like the previous patches, back-patch to all supported branches. Discussion: https://postgr.es/m/16068-4cb6eeaa7eb46d93@postgresql.org
* Select CFLAGS_SL at configure time, not in platform-specific Makefiles.Tom Lane2019-10-21
| | | | | | | | | | | | | | | | | | | | | | | Move the platform-dependent logic that sets CFLAGS_SL from src/makefiles/Makefile.foo to src/template/foo, so that the value is determined at configure time and thus is available while running configure's tests. On a couple of platforms this might save a few microseconds of build time by eliminating a test that make otherwise has to do over and over. Otherwise it's pretty much a wash for build purposes; in particular, this makes no difference to anyone who might be overriding CFLAGS_SL via a make option. This patch in itself does nothing with the value and thus should not change any behavior, though you'll probably have to re-run configure to get a correctly updated Makefile.global. We'll use the new configure variable in a follow-on patch. Per gripe from Kyotaro Horiguchi. Back-patch to all supported branches, because the follow-on patch is a portability bug fix. Discussion: https://postgr.es/m/20191010.144533.263180400.horikyota.ntt@gmail.com
* For PowerPC instruction "addi", use constraint "b".Noah Misch2019-10-18
| | | | | | | | | | | | Without "b", a variant of the tas() code miscompiles on macOS 10.4. This may also fix a compilation failure involving macOS 10.1. Today's compilers have been allocating acceptable registers with or without this change, but this future-proofs the code by precisely conveying the acceptable registers. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20191009063900.GA4066266@rfd.leadboat.com
* Fix failure of archive recovery with recovery_min_apply_delay enabled.Fujii Masao2019-10-18
| | | | | | | | | | | | | | | | | | | | | | | | recovery_min_apply_delay parameter is intended for use with streaming replication deployments. However, the document clearly explains that the parameter will be honored in all cases if it's specified. So it should take effect even if in archive recovery. But, previously, archive recovery with recovery_min_apply_delay enabled always failed, and caused assertion failure if --enable-caasert is enabled. The cause of this problem is that; the ownership of recoveryWakeupLatch that recovery_min_apply_delay uses was taken only when standby mode is requested. So unowned latch could be used in archive recovery, and which caused the failure. This commit changes recovery code so that the ownership of recoveryWakeupLatch is taken even in archive recovery. Which prevents archive recovery with recovery_min_apply_delay from failing. Back-patch to v9.4 where recovery_min_apply_delay was added. Author: Fujii Masao Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAHGQGwEyD6HdZLfdWc+95g=VQFPR4zQL4n+yHxQgGEGjaSVheQ@mail.gmail.com
* Fix minor bug in logical-replication walsender shutdownAlvaro Herrera2019-10-17
| | | | | | | | | | | | | | | | | Logical walsender should exit when it catches up with sending WAL during shutdown; but there was a rare corner case when it failed to because of a race condition that puts it back to wait for more WAL instead -- but since there wasn't any, it'd not shut down immediately. It would only continue the shutdown when wal_sender_timeout terminates the sleep, which causes annoying waits during shutdown procedure. Restructure the code so that we no longer forget to set WalSndCaughtUp in that case. This was an oversight in commit c6c333436. Backpatch all the way down to 9.4. Author: Craig Ringer, Álvaro Herrera Discussion: https://postgr.es/m/CAMsr+YEuz4XwZX_QmnX_-2530XhyAmnK=zCmicEnq1vLr0aZ-g@mail.gmail.com
* When restoring GUCs in parallel workers, show an error context.Thomas Munro2019-10-17
| | | | | | | | | | | Otherwise it can be hard to see where an error is coming from, when the parallel worker sets all the GUCs that it received from the leader. Bug #15726. Back-patch to 9.5, where RestoreGUCState() appeared. Reported-by: Tiago Anastacio Reviewed-by: Daniel Gustafsson, Tom Lane Discussion: https://postgr.es/m/15726-6d67e4fa14f027b3%40postgresql.org
* Fix bug that could try to freeze running multixacts.Thomas Munro2019-10-17
| | | | | | | | | | | | | Commits 801c2dc7 and 801c2dc7 made it possible for vacuum to try to freeze a multixact that is still running. That was prevented by a check, but raised an error. Repair. Back-patch all the way. Author: Nathan Bossart, Jeremy Schneider Reported-by: Jeremy Schneider Reviewed-by: Jim Nasby, Thomas Munro Discussion: https://postgr.es/m/DAFB8AFF-2F05-4E33-AD7F-FF8B0F760C17%40amazon.com
* Add missing include to pg_upgrade/version.cTomas Vondra2019-10-16
| | | | | | | | | | | | | | Commit 8d48e6a724 uses RELKIND_ constants when building the query, but did not include the header defining them. On 10+ this header is already included, but on 9.6 and earlier it was missing. It compiles just fine, but then fails during execution ERROR: column "relkind_relation" does not exist Fix by adding the necessary header file, and backpatch to 9.4-. Backpatch-to: 9.4- Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
* Improve the check for pg_catalog.line data type in pg_upgradeTomas Vondra2019-10-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pg_upgrade check for pg_catalog.line data type when upgrading from 9.3 had a couple of issues with domains and composite types. Firstly, it triggered false positives for composite types unused in objects with storage. This was enough to trigger an unnecessary pg_upgrade failure: CREATE TYPE line_composite AS (l pg_catalog.line) On the other hand, this only happened with composite types directly on the pg_catalog.line data type, but not with a domain. So this was not detected CREATE DOMAIN line_domain AS pg_catalog.line; CREATE TYPE line_composite_2 AS (l line_domain); unlike the first example. These false positives and inconsistencies are unfortunate, but what's worse we've failed to detected objects using the pg_catalog.line data type through a domain. So we missed cases like this CREATE TABLE t (l line_composite_2); The consequence is clusters broken after a pg_upgrade. This fixes these false positives and false negatives by using the same recursive CTE introduced by eaf900e842 for sql_identifier. 9.3 did not support domains on composite types, but we can still have multi-level composite types. Backpatch all the way to 9.4, where the format for pg_catalog.line data type changed. Author: Tomas Vondra Backpatch-to: 9.4- Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
* AIX: Stop adding option -qsrcmsg.Noah Misch2019-10-12
| | | | | | | | With xlc v16.1.0, it causes internal compiler errors. With xlc versions not exhibiting that bug, removing -qsrcmsg merely changes the compiler error reporting format. Back-patch to 9.4 (all supported versions). Discussion: https://postgr.es/m/20191003064105.GA3955242@rfd.leadboat.com
* Flush logical mapping files with fd opened for read/write at checkpointMichael Paquier2019-10-09
| | | | | | | | | | | | | | | The file descriptor was opened with read-only to fsync a regular file, which would cause EBADFD errors on some platforms. This is similar to the recent fix done by a586cc4b (which was broken by me with 82a5649), except that I noticed this issue while monitoring the backend code for similar mistakes. Backpatch to 9.4, as this has been introduced since logical decoding exists as of b89e151. Author: Michael Paquier Reviewed-by: Andres Freund Discussion: https://postgr.es/m/20191006045548.GA14532@paquier.xyz Backpatch-through: 9.4
* Check for too many postmaster children before spawning a bgworker.Tom Lane2019-10-07
| | | | | | | | | | | | | | | | | | | | The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
* Report test_atomic_ops() failures consistently, via macros.Noah Misch2019-10-05
| | | | | | | | | | This prints the unexpected value in more failure cases, and it removes forty-eight hand-maintained error messages. Back-patch to 9.5, which introduced these tests. Reviewed (in an earlier version) by Andres Freund. Discussion: https://postgr.es/m/20190915160021.GA24376@alvherre.pgsql
* Handle spaces in OpenSSL install location for MSVCAndrew Dunstan2019-10-04
| | | | | | | | | First, make sure that the .exe name is quoted when trying to get the version number. Also, don't quote the lib name for using in the project files if it's already been quoted. This second change applies to all libraries, not just OpenSSL. This has clearly been broken forever, so backpatch to all live branches.
* Fix bitshiftright()'s zero-padding some more.Tom Lane2019-10-04
| | | | | | | | | | | | | | | Commit 5ac0d9360 failed to entirely fix bitshiftright's habit of leaving one-bits in the pad space that should be all zeroes, because in a moment of sheer brain fade I'd concluded that only the code path used for not-a-multiple-of-8 shift distances needed to be fixed. Of course, a multiple-of-8 shift distance can also cause the problem, so we need to forcibly zero the extra bits in both cases. Per bug #16037 from Alexander Lakhin. As before, back-patch to all supported branches. Discussion: https://postgr.es/m/16037-1d1ebca564db54f4@postgresql.org
* Avoid unnecessary out-of-memory errors during encoding conversion.Tom Lane2019-10-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Encoding conversion uses the very simplistic rule that the output can't be more than 4X longer than the input, and palloc's a buffer of that size. This results in failure to convert any string longer than 1/4 GB, which is becoming an annoying limitation. As a band-aid to improve matters, allow the allocated output buffer size to exceed 1GB. We still insist that the final result fit into MaxAllocSize (1GB), though. Perhaps it'd be safe to relax that restriction, but it'd require close analysis of all callers, which is daunting (not least because external modules might call these functions). For the moment, this should allow a 2X to 4X improvement in the longest string we can convert, which is a useful gain in return for quite a simple patch. Also, once we have successfully converted a long string, repalloc the output down to the actual string length, returning the excess to the malloc pool. This seems worth doing since we can usually expect to give back several MB if we take this path at all. This still leaves much to be desired, most notably that the assumption that MAX_CONVERSION_GROWTH == 4 is very fragile, and yet we have no guard code verifying that the output buffer isn't overrun. Fixing that would require significant changes in the encoding conversion APIs, so it'll have to wait for some other day. The present patch seems safely back-patchable, so patch all supported branches. Alvaro Herrera and Tom Lane Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
* Allow repalloc() to give back space when a large chunk is downsized.Tom Lane2019-10-03
| | | | | | | | | | | | | | | | | | Up to now, if you resized a large (>8K) palloc chunk down to a smaller size, aset.c made no attempt to return any space to the malloc pool. That's unpleasant if a really large allocation is resized to a significantly smaller size. I think no such cases existed when this code was designed, and I'm not sure whether they're common even yet, but an upcoming fix to encoding conversion will certainly create such cases. Therefore, fix AllocSetRealloc so that it gives realloc() a chance to do something with the block. This doesn't noticeably increase complexity, we mostly just have to change the order in which the cases are considered. Back-patch to all supported branches. Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
* Selectively include window frames in expression walks/mutates.Andrew Gierth2019-10-03
| | | | | | | | | | | | | | | | | | | | query_tree_walker and query_tree_mutator were skipping the windowClause of the query, without regard for the fact that the startOffset and endOffset in a WindowClause node are expression trees that need to be processed. This was an oversight in commit ec4be2ee6 from 2010 which added the expression fields; the main symptom is that function parameters in window frame clauses don't work in inlined functions. Fix (as conservatively as possible since this needs to not break existing out-of-tree callers) and add tests. Backpatch all the way, since this has been broken since 9.0. Per report from Alastair McKinley; fix by me with kibitzing and review from Tom Lane. Discussion: https://postgr.es/m/DB6PR0202MB2904E7FDDA9D81504D1E8C68E3800@DB6PR0202MB2904.eurprd02.prod.outlook.com
* Remove temporary WAL and history files at the end of archive recoveryMichael Paquier2019-10-02
| | | | | | | | | | | | | | | cbc55da has reworked the order of some actions at the end of archive recovery. Unfortunately this overlooked the fact that the startup process needs to remove RECOVERYXLOG (for temporary WAL segment newly recovered from archives) and RECOVERYHISTORY (for temporary history file) at this step, leaving the files around even after recovery ended. Backpatch to 9.5, like the previous commit. Author: Sawada Masahiko Reviewed-by: Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/CAD21AoBO_eDQub6zojFnWtnmutRBWvYf7=cW4Hsqj+U_R26w3Q@mail.gmail.com Backpatch-through: 9.5
* Fix oversight in commit 4429f6a9e3e12bb4af6e3677fbc78cd80f160252.Amit Kapila2019-09-27
| | | | | | | | | | | | | | The test name and the following test cases suggest the index created should be hash index, but it forgot to add 'using hash' in the test case. This in itself won't improve code coverage as there were some other tests which were covering the corresponding code. However, it is better if the added tests serve their actual purpose. Reported-by: Paul A Jungwirth Author: Paul A Jungwirth Reviewed-by: Mahendra Singh Backpatch-through: 9.4 Discussion: https://postgr.es/m/CA+renyV=Us-5XfMC25bNp-uWSj39XgHHmGE9Rh2cQKMegSj52g@mail.gmail.com
* Fix failure to zero-pad the result of bitshiftright().Tom Lane2019-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the bitstring length is not a multiple of 8, we'd shift the rightmost bits into the pad space, which must be zeroes --- bit_cmp, for one, depends on that. This'd lead to the result failing to compare equal to what it should compare equal to, as reported in bug #16013 from Daryl Waycott. This is, if memory serves, not the first such bug in the bitstring functions. In hopes of making it the last one, do a bit more work than minimally necessary to fix the bug: * Add assertion checks to bit_out() and varbit_out() to complain if they are given incorrectly-padded input. This will improve the odds that manual testing of any new patch finds problems. * Encapsulate the padding-related logic in macros to make it easier to use. Also, remove unnecessary padding logic from bit_or() and bitxor(). Somebody had already noted that we need not re-pad the result of bit_and() since the inputs are required to be the same length, but failed to extrapolate that to the other two. Also, move a comment block that once was near the head of varbit.c (but people kept putting other stuff in front of it), to put it in the header block. Note for the release notes: if anyone has inconsistent data as a result of saving the output of bitshiftright() in a table, it's possible to fix it with something like UPDATE mytab SET bitcol = ~(~bitcol) WHERE bitcol != ~(~bitcol); This has been broken since day one, so back-patch to all supported branches. Discussion: https://postgr.es/m/16013-c2765b6996aacae9@postgresql.org
* Update time zone data files to tzdata release 2019c.Tom Lane2019-09-20
| | | | | | | DST law changes in Fiji and Norfolk Island. Historical corrections for Alberta, Austria, Belgium, British Columbia, Cambodia, Hong Kong, Indiana (Perry County), Kaliningrad, Kentucky, Michigan, Norfolk Island, South Korea, and Turkey.
* Fix oversight in backpatch of 6cae9d2c10Alexander Korotkov2019-09-19
| | | | | | | | | | During backpatch of 6cae9d2c10 Float8GetDatum() was accidentally removed. This commit turns it back. Reported-by: Erik Rijkers Discussion: https://postgr.es/m/6d51305e1159241cabee132f7efc7eff%40xs4all.nl Author: Tom Lane Backpatch-through: from 11 to 9.5
* Improve handling of NULLs in KNN-GiST and KNN-SP-GiSTAlexander Korotkov2019-09-19
| | | | | | | | | | | | | | | | | | | | | This commit improves subject in two ways: * It removes ugliness of 02f90879e7, which stores distance values and null flags in two separate arrays after GISTSearchItem struct. Instead we pack both distance value and null flag in IndexOrderByDistance struct. Alignment overhead should be negligible, because we typically deal with at most few "col op const" expressions in ORDER BY clause. * It fixes handling of "col op NULL" expression in KNN-SP-GiST. Now, these expression are not passed to support functions, which can't deal with them. Instead, NULL result is implicitly assumed. It future we may decide to teach support functions to deal with NULL arguments, but current solution is bugfix suitable for backpatch. Reported-by: Nikita Glukhov Discussion: https://postgr.es/m/826f57ee-afc7-8977-c44c-6111d18b02ec%40postgrespro.ru Author: Nikita Glukhov Reviewed-by: Alexander Korotkov Backpatch-through: 9.4
* pg_upgrade/test.sh: Quote sed(1) argumentAlvaro Herrera2019-09-18
| | | | | | | Lack of quotes results in failure to run the test under older Solaris. Author: Marina Polyakova, Victor Wagner Discussion: https://postgr.es/m/feba89f89e8925b3535cb7d72b9e05e1@postgrespro.ru
* Replace xlc __fetch_and_add() with inline asm.Noah Misch2019-09-13
| | | | | | | | | | PostgreSQL has been unusable when built with xlc 13 and newer, which are incompatible with our use of __fetch_and_add(). Back-patch to 9.5, which introduced pg_atomic_fetch_add_u32(). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20190831071157.GA3251746@rfd.leadboat.com