aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* Change delimiter used for display of NextXIDJoe Conway2016-02-12
| | | | | | | | | | | | | NextXID has been rendered in the form of a pg_lsn even though it really is not. This can cause confusion, so change the format from %u/%u to %u:%u, per discussion on hackers. Complaint by me, patch by me and Bruce, reviewed by Michael Paquier and Alvaro. Applied to HEAD only. Author: Joe Conway, Bruce Momjian Reviewed-by: Michael Paquier, Alvaro Herrera Backpatch-through: master
* Increase deadlock_timeout some more in the deadlock-hard isolation test.Tom Lane2016-02-12
| | | | | | | | The previous value of 5s is inadequate for the buildfarm's CLOBBER_CACHE_ALWAYS animals: they take long enough to do the is-it-waiting queries that the timeout expires, allowing the database state to change, before isolationtester is done looking. Perhaps 10s will be enough. (If it isn't, I'm inclined to reduce the number of sessions involved.)
* Revert "isolationtester: don't repeat the is-it-waiting query when retrying ↵Tom Lane2016-02-12
| | | | | | | | a step." This mostly reverts commit 9c9782f066e0ce5424b8706df2cce147cb78170f. I left in the parts that rearranged removal of completed waiting steps; but the idea of not rechecking a step's blocked-ness isn't working.
* Revert "Still further tweaking of deadlock isolation tests."Tom Lane2016-02-12
| | | | | | This reverts commit d03130d378b5fb071d231a7822784ad87268583a. That was dependent on an isolationtester.c change that now proves to be broken; we will need to find another solution.
* pgbench: cleanup use of a "logfile" parameterAlvaro Herrera2016-02-12
| | | | | | | | | | | | | There is no reason to have the per-thread logfile file pointer as a separate parameter in various functions: it's much simpler to put it in the per-thread state struct instead, which is already being passed to all functions that need the log file anyway. Change the callsites in which it was used as a boolean to test whether logging is active, so that they use the use_log global variable instead. No backpatch, even though this exists since commit a887c486d5df of March 2010, because this is just for cleanliness' sake and the surrounding code has been modified a lot recently anyway.
* pgbench: fix segfault with empty sql fileAlvaro Herrera2016-02-12
| | | | | | | | | Commit 1d0c3b3f8a introduced a bug that causes pgbench to crash if an empty script file is specified. Fix it by rejecting such files at startup, which is the historical and intended behavior. Reported-By: Jeff Janes Discussion: https://www.postgresql.org/message-id/CAMkU=1zxKUbLPOt9hQWFp14pTc=V0cGo2GQBbn2GsK2Pu+8ZfA@mail.gmail.com
* Still further tweaking of deadlock isolation tests.Tom Lane2016-02-12
| | | | | | | | | | | | It turns out that there is a second race condition in the new deadlock-hard test: once the deadlock detector fires, it's uncertain whether step s7a8 or step s8a1 will report first, because killing s8's transaction unblocks s7. So far, s7 has only been seen to report first in CLOBBER_CACHE_ALWAYS builds, but it's pretty reproducible there, and in theory it should sometimes occur in normal builds too. If s7 were a bit slower than usual, that could also break the test, since the existing expected-file assumes that we'll see s7a8 report the first time we check it after s8a1 completes. To fix, add a post-lock delay to s7a8.
* isolationtester: don't repeat the is-it-waiting query when retrying a step.Tom Lane2016-02-12
| | | | | | | | | | | | | | | | | | If we're retrying a step, then we already decided it was blocked on a lock, and there's no need to recheck that. The original coding of commit 38f8bdcac4982215beb9f65a19debecaf22fd470 resulted in a large number of is-it-waiting queries when dealing with multiple concurrently-blocked sessions, which is fairly pointless and also results in test failures in CLOBBER_CACHE_ALWAYS builds, where the is-it-waiting query is quite slow. This definition also permits appending pg_sleep() calls to steps where it's needed to control the order of finish of concurrent steps. Before, that did not work nicely because we'd decide that a step performing a sleep was not blocked and hang up waiting for it to finish, rather than noticing the completion of the concurrent step we're supposed to notice first. In passing, revise handling of removal of completed waiting steps to make it a bit less messy.
* Re-pgindent isolationtester.c.Tom Lane2016-02-12
| | | | | Need to do some more hacking on this, and got annoyed that it's not indent clean.
* Fix whitespacePeter Eisentraut2016-02-12
|
* Introduce extensible node types.Robert Haas2016-02-12
| | | | | | | | | | | | | | | | | | An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
* Make builtin lwlock tranche names consistent.Robert Haas2016-02-12
| | | | | | Previously, we had a mix of styles. Amit Kapila
* Further tweaking of deadlock isolation tests.Tom Lane2016-02-11
| | | | | | | | The new deadlock-soft-2 test has a timing dependency too: it supposes that isolationtester will detect step s1b as waiting before the deadlock detector runs and grants it the lock. Adjust deadlock_timeout to ensure that that's true even in CLOBBER_CACHE_ALWAYS builds, where the wait detection query is quite slow. Per buildfarm member jaguarundi.
* Refactor check_functional_grouping() to use get_primary_key_attnos().Tom Lane2016-02-11
| | | | | | | | | If we ever get around to allowing functional dependency to be proven from other things besides simple primary keys, this code will need to be rethought, but that was true anyway. In the meantime, we might as well not have two very-similar routines for scanning pg_constraint. David Rowley, reviewed by Julien Rouhaud
* Remove GROUP BY columns that are functionally dependent on other columns.Tom Lane2016-02-11
| | | | | | | | | | | | | | | | | | | | If a GROUP BY clause includes all columns of a non-deferred primary key, as well as other columns of the same relation, those other columns are redundant and can be dropped from the grouping; the pkey is enough to ensure that each row of the table corresponds to a separate group. Getting rid of the excess columns will reduce the cost of the sorting or hashing needed to implement GROUP BY, and can indeed remove the need for a sort step altogether. This seems worth testing for since many query authors are not aware of the GROUP-BY-primary-key exception to the rule about queries not being allowed to reference non-grouped-by columns in their targetlists or HAVING clauses. Thus, redundant GROUP BY items are not uncommon. Also, we can make the test pretty cheap in most queries where it won't help by not looking up a rel's primary key until we've found that at least two of its columns are in GROUP BY. David Rowley, reviewed by Julien Rouhaud
* Move pg_constraint.h function declarations to new file pg_constraint_fn.h.Tom Lane2016-02-11
| | | | | | | | | | | A pending patch requires exporting a function returning Bitmapset from catalog/pg_constraint.c. As things stand, that would mean including nodes/bitmapset.h in pg_constraint.h, which might be hazardous for the client-side includability of that header. It's not entirely clear whether any client-side code needs to include pg_constraint.h, but it seems prudent to assume that there is some such code somewhere. Therefore, split off the function definitions into a new file pg_constraint_fn.h, similarly to what we've done for some other catalog header files.
* Fix typo in comment.Tom Lane2016-02-11
|
* Shift the responsibility for emitting "database system is shut down".Tom Lane2016-02-11
| | | | | | | | | | | | | | | | | | Historically this message has been emitted at the end of ShutdownXLOG(). That's not an insane place for it in a standalone backend, but in the postmaster environment we've grown a fair amount of stuff that happens later, including archiver/walsender shutdown, stats collector shutdown, etc. Recent buildfarm experimentation showed that on slower machines there could be many seconds' delay between finishing ShutdownXLOG() and actual postmaster exit. That's fairly confusing, both for testing purposes and for DBAs. Hence, move the code that prints this message into UnlinkLockFiles(), so that it comes out just after we remove the postmaster's pidfile. That is a more appropriate definition of "is shut down" from the point of view of "pg_ctl stop", for example. In general, removing the pidfile should be the last externally-visible action of either a postmaster or a standalone backend; compare commit d73d14c271653dff10c349738df79ea03b85236c for instance. So this seems like a reasonably future-proof approach.
* Use separate lwlock tranches for buffer, lock, and predicate lock managers.Robert Haas2016-02-11
| | | | | | | | This finishes the work - spread across many commits over the last several months - of putting each type of lock other than the named individual locks into a separate tranche. Amit Kapila
* Make new deadlock isolation test more reproducible.Tom Lane2016-02-11
| | | | | | | | | | | | | | | The original formulation of 4c9864b9b4d87d02f07f40bb27976da737afdcab was extremely timing-sensitive, because it arranged for the deadlock detector to be running (and possibly unblocking the current query) at almost exactly the same time as isolationtester would be probing to see if the query is blocked. The committed expected-file assumed that the deadlock detection would finish first, but we see the opposite on both fast and slow buildfarm animals. Adjust the deadlock timeout settings to make it predictable that isolationtester *will* see the query as waiting before deadlock detection unblocks it. I used a 5s timeout for the same reasons mentioned in a7921f71a3c747141344d8604f6a6d7b4cddb2a9.
* Code review for isolationtester changes.Tom Lane2016-02-11
| | | | | | | Fix a few oversights in 38f8bdcac4982215beb9f65a19debecaf22fd470: don't leak memory in run_permutation(), remember when we've issued a cancel rather than issuing another one every 10ms, fix some typos in comments.
* Improve error reporting in format()Teodor Sigaev2016-02-11
| | | | | | Clarify invalid format conversion type error message and add hint. Author: Jim Nasby
* Rename PGPROC fields related to group XID clearing again.Robert Haas2016-02-11
| | | | | | | | | | | | | | Commit 0e141c0fbb211bdd23783afa731e3eef95c9ad7a introduced a new facility to reduce ProcArrayLock contention by clearing several XIDs from the ProcArray under a single lock acquisition. The names initially chosen were deemed not to be very good choices, so commit 4aec49899e5782247e134f94ce1c6ee926f88e1c renamed them. But now it seems like we still didn't get it right. A pending patch wants to add similar infrastructure for batching CLOG updates, so the names need to be clear enough to allow a new set of structure members with a related purpose. Amit Kapila
* Add some isolation tests for deadlock detection and resolution.Robert Haas2016-02-11
| | | | Previously, we had no test coverage for the deadlock detector.
* Modify the isolation tester so that multiple sessions can wait.Robert Haas2016-02-11
| | | | | | This allows testing of deadlock scenarios. Scenarios that would previously have been considered invalid are now simply taken as a scenario in which more than one backend will wait.
* Specify permutations for isolation tests with "invalid" permutations.Robert Haas2016-02-11
| | | | | | | This is a necessary prerequisite for forthcoming changes to allow deadlock scenarios to be tested by the isolation tester. It is also a good idea on general principle, since these scenarios add no useful test coverage not provided by other scenarios, but do to take time to execute.
* In pg_rewind test suite, triple promote timeout to 90s.Noah Misch2016-02-10
| | | | | | Thirty seconds was not consistently enough for promotion to complete on buildfarm members sungazer and tern. Experiments suggest 43s would have been enough. Back-patch to 9.5, where pg_rewind was introduced.
* Accept pg_ctl timeout from the PGCTLTIMEOUT environment variable.Noah Misch2016-02-10
| | | | | | | | | | | Many automated test suites call pg_ctl. Buildfarm members axolotl, hornet, mandrill, shearwater, sungazer and tern have failed when server shutdown took longer than the pg_ctl default 60s timeout. This addition permits slow hosts to easily raise the timeout without us editing a --timeout argument into every test suite pg_ctl call. Back-patch to 9.1 (all supported versions) for the sake of automated testing. Reviewed by Tom Lane.
* Avoid use of sscanf() to parse ispell dictionary files.Tom Lane2016-02-10
| | | | | | | | | | | | | | | | | | | | | | | | It turns out that on FreeBSD-derived platforms (including OS X), the *scanf() family of functions is pretty much brain-dead about multibyte characters. In particular it will apply isspace() to individual bytes of input even when those bytes are part of a multibyte character, thus allowing false recognition of a field-terminating space. We appear to have little alternative other than instituting a coding rule that *scanf() is not to be used if the input string might contain multibyte characters. (There was some discussion of relying on "%ls", but that probably just moves the portability problem somewhere else, and besides it doesn't fully prevent BSD *scanf() from using isspace().) This patch is a down payment on that: it gets rid of use of sscanf() to parse ispell dictionary files, which are certainly at great risk of having a problem. The code is cleaner this way anyway, though a bit longer. In passing, improve a few comments. Report and patch by Artur Zakirov, reviewed and somewhat tweaked by me. Back-patch to all supported branches.
* Revert "Temporarily make pg_ctl and server shutdown a whole lot chattier."Tom Lane2016-02-10
| | | | | | This reverts commit 3971f64843b02e4a55d854156bd53e46a0588e45 and a couple of followon debugging commits; I think we've learned what we can from them.
* Code cleanup in the wake of recent LWLock refactoring.Robert Haas2016-02-10
| | | | | | | | | | As of commit c1772ad9225641c921545b35c84ee478c326b95e, there's no longer any way of requesting additional LWLocks in the main tranche, so we don't need NumLWLocks() or LWLockAssign() any more. Also, some of the allocation counters that we had previously aren't needed any more either. Amit Kapila
* Add still more chattiness in server shutdown.Tom Lane2016-02-09
| | | | | | Further investigation says that there may be some slow operations after we've finished ShutdownXLOG(), so add some more log messages to try to isolate that. This is all temporary code too.
* Add more chattiness in server shutdown.Tom Lane2016-02-09
| | | | | | | Early returns from the buildfarm show that there's a bit of a gap in the logging I added in 3971f64843b02e4a: the portion of CreateCheckPoint() after CheckPointGuts() can take a fair amount of time. Add a few more log messages in that section of code. This too shall be reverted later.
* Temporarily make pg_ctl and server shutdown a whole lot chattier.Tom Lane2016-02-08
| | | | | | | | | | | | | This is a quick hack, due to be reverted when its purpose has been served, to try to gather information about why some of the buildfarm critters regularly fail with "postmaster does not shut down" complaints. Maybe they are just really overloaded, but maybe something else is going on. Hence, instrument pg_ctl to print the current time when it starts waiting for postmaster shutdown and when it gives up, and add a lot of logging of the current time in the server's checkpoint and shutdown code paths. No attempt has been made to make this pretty. I'm not even totally sure if it will build on Windows, but we'll soon find out.
* Re-pgindent varlena.c.Tom Lane2016-02-08
| | | | Just to make sure previous commit worked ...
* Rename typedef "string" to "VarString".Tom Lane2016-02-08
| | | | | | | | | | Since pgindent treats typedef names as global, the original coding of b47b4dbf683f13e6 would have had rather nasty effects on the formatting of other files in which "string" is used as a variable or field name. Use a less generic name for this typedef, and rename some other identifiers to match. Peter Geoghegan, per gripe from me
* Fix some regex issues with out-of-range characters and large char ranges.Tom Lane2016-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, our regex code defined CHR_MAX as 0xfffffffe, which is a bad choice because it is outside the range of type "celt" (int32). Characters approaching that limit could lead to infinite loops in logic such as "for (c = a; c <= b; c++)" where c is of type celt but the range bounds are chr. Such loops will work safely only if CHR_MAX+1 is representable in celt, since c must advance to beyond b before the loop will exit. Fortunately, there seems no reason not to restrict CHR_MAX to 0x7ffffffe. It's highly unlikely that Unicode will ever assign codes that high, and none of our other backend encodings need characters beyond that either. In addition to modifying the macro, we have to explicitly enforce character range restrictions on the values of \u, \U, and \x escape sequences, else the limit is trivially bypassed. Also, the code for expanding case-independent character ranges in bracket expressions had a potential integer overflow in its calculation of the number of characters it could generate, which could lead to allocating too small a character vector and then overwriting memory. An attacker with the ability to supply arbitrary regex patterns could easily cause transient DOS via server crashes, and the possibility for privilege escalation has not been ruled out. Quite aside from the integer-overflow problem, the range expansion code was unnecessarily inefficient in that it always produced a result consisting of individual characters, abandoning the knowledge that we had a range to start with. If the input range is large, this requires excessive memory. Change it so that the original range is reported as-is, and then we add on any case-equivalent characters that are outside that range. With this approach, we can bound the number of individual characters allowed without sacrificing much. This patch allows at most 100000 individual characters, which I believe to be more than the number of case pairs existing in Unicode, so that the restriction will never be hit in practice. It's still possible for range() to take awhile given a large character code range, so also add statement-cancel detection to its loop. The downstream function dovec() also lacked cancel detection, and could take a long time given a large output from range(). Per fuzz testing by Greg Stark. Back-patch to all supported branches. Security: CVE-2016-0773
* Make GIN regression test stable.Fujii Masao2016-02-08
| | | | | | | | | | | | | | | | | | | | | | | Commit 7f46eaf added the regression test which checks that gin_clean_pending_list() cleans up the GIN pending list and returns >0. This usually works fine. But if autovacuum comes along and cleans the list before gin_clean_pending_list() starts, the function may return 0, and then the regression test may fail. To fix the problem, this commit disables autovacuum on the target index of gin_clean_pending_list() by setting autovacuum_enabled reloption to off when creating the table. Also this commit sets gin_pending_list_limit reloption to 4MB on the target index. Otherwise when running "make installcheck" with small gin_pending_list_limit GUC, insertions of data may trigger the cleanup of pending list before gin_clean_pending_list() starts and the function may return 0. This could cause the regression test to fail. Per buildfarm member spoonbill. Reported-By: Tom Lane
* Fix overeager pushdown of HAVING clauses when grouping sets are used.Andres Freund2016-02-08
| | | | | | | | | | | | | | | | | | | | | | | In 61444bfb we started to allow HAVING clauses to be fully pushed down into WHERE, even when grouping sets are in use. That turns out not to work correctly, because grouping sets can "produce" NULLs, meaning that filtering in WHERE and HAVING can have different results, even when no aggregates or volatile functions are involved. Instead only allow pushdown of empty grouping sets. It'd be nice to do better, but the exact mechanics of deciding which cases are safe are still being debated. It's important to give correct results till we find a good solution, and such a solution might not be appropriate for backpatching anyway. Bug: #13863 Reported-By: 'wrb' Diagnosed-By: Dean Rasheed Author: Andrew Gierth Reviewed-By: Dean Rasheed and Andres Freund Discussion: 20160113183558.12989.56904@wrigleys.postgresql.org Backpatch: 9.5, where grouping sets were introduced
* Fix deparsing of ON CONFLICT arbiter WHERE clauses.Tom Lane2016-02-07
| | | | | | | | | | | The parser doesn't allow qualification of column names appearing in these clauses, but ruleutils.c would sometimes qualify them, leading to dump/reload failures. Per bug #13891 from Onder Kalaci. (In passing, make stanzas in ruleutils.c that save/restore varprefix more consistent.) Peter Geoghegan
* ExecHashRemoveNextSkewBucket must physically copy tuples to main hashtable.Tom Lane2016-02-07
| | | | | | | | | | | | | | | | | | | | | | Commit 45f6240a8fa9d355 added an assumption in ExecHashIncreaseNumBatches and ExecHashIncreaseNumBuckets that they could find all tuples in the main hash table by iterating over the "dense storage" introduced by that patch. However, ExecHashRemoveNextSkewBucket continued its old practice of simply re-linking deleted skew tuples into the main table's hashchains. Hence, such tuples got lost during any subsequent increase in nbatch or nbuckets, and would never get joined, as reported in bug #13908 from Seth P. I (tgl) think that the aforesaid commit has got multiple design issues and should be reworked rather completely; but there is no time for that right now, so band-aid the problem by making ExecHashRemoveNextSkewBucket physically copy deleted skew tuples into the "dense storage" arena. The added test case is able to exhibit the problem by means of fooling the planner with a WHERE condition that it will underestimate the selectivity of, causing the initial nbatch estimate to be too small. Tomas Vondra and Tom Lane. Thanks to David Johnston for initial investigation into the bug report.
* Fix parallel-safety markings for pg_upgrade functions.Robert Haas2016-02-07
| | | | | | These establish backend-local state which will not be copied to parallel workers, so they must be marked parallel-restricted, not parallel-safe.
* Introduce a new GUC force_parallel_mode for testing purposes.Robert Haas2016-02-07
| | | | | | | | | | | | | | | When force_parallel_mode = true, we enable the parallel mode restrictions for all queries for which this is believed to be safe. For the subset of those queries believed to be safe to run entirely within a worker, we spin up a worker and run the query there instead of running it in the original process. When force_parallel_mode = regress, make additional changes to allow the regression tests to run cleanly even though parallel workers have been injected under the hood. Taken together, this facilitates both better user testing and better regression testing of the parallelism code. Robert Haas, with help from Amit Kapila and Rushabh Lathia.
* Introduce group locking to prevent parallel processes from deadlocking.Robert Haas2016-02-07
| | | | | | | | | | | | | For locking purposes, we now regard heavyweight locks as mutually non-conflicting between cooperating parallel processes. There are some possible pitfalls to this approach that are not to be taken lightly, but it works OK for now and can be changed later if we find a better approach. Without this, it's very easy for parallel queries to silently self-deadlock if the user backend holds strong relation locks. Robert Haas, with help from Amit Kapila. Thanks to Noah Misch and Andres Freund for extensive discussion of possible issues with this approach.
* Improve speed of timestamp/time/date output functions.Tom Lane2016-02-06
| | | | | | | | | | | It seems that sprintf(), at least in glibc's version, is unreasonably slow compared to hand-rolled code for printing integers. Replacing most uses of sprintf() in the datetime.c output functions with special-purpose code turns out to give more than a 2X speedup in COPY of a table with a single timestamp column; which is pretty impressive considering all the other logic in that code path. David Rowley and Andres Freund, reviewed by Peter Geoghegan and myself
* Fix comment block trashed by pgindent.Tom Lane2016-02-06
| | | | Looks like I put the protective dashes in the wrong place in f4e4b32743.
* Improve HJDEBUG code a bit.Tom Lane2016-02-06
| | | | | | | | | | | | | | | Commit 30d7ae3c76d2de144232ae6ab328ca86b70e72c3 introduced an HJDEBUG stanza that probably didn't compile at the time, and definitely doesn't compile now, because it refers to a nonexistent variable. It doesn't seem terribly useful anyway, so just get rid of it. While I'm fooling with it, use %z modifier instead of the obsolete hack of casting size_t to unsigned long, and include the HashJoinTable's address in each printout so that it's possible to distinguish the activities of multiple hashjoins occurring in one query. Noted while trying to use HJDEBUG to investigate bug #13908. Back-patch to 9.5, because code that doesn't compile is certainly not very helpful.
* Comment on dead code in AtAbort_Portals() and AtSubAbort_Portals().Noah Misch2016-02-05
| | | | Reviewed by Tom Lane and Robert Haas.
* Force certain "pljava" custom GUCs to be PGC_SUSET.Noah Misch2016-02-05
| | | | | | | Future PL/Java versions will close CVE-2016-0766 by making these GUCs PGC_SUSET. This PostgreSQL change independently mitigates that PL/Java vulnerability, helping sites that update PostgreSQL more frequently than PL/Java. Back-patch to 9.1 (all supported versions).
* Update time zone data files to tzdata release 2016a.Tom Lane2016-02-05
| | | | | DST law changes in Cayman Islands, Metlakatla, Trans-Baikal Territory (Zabaykalsky Krai). Historical corrections for Pakistan.
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-22 10:35:59 +0000