2016-02-11 This release contains a variety of fixes from 9.4.5. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you are upgrading an installation that contains any GIN indexes that use the (non-default) jsonb_path_ops operator class, see the first changelog entry below. Also, if you are upgrading from a version earlier than 9.4.4, see . Fix inconsistent hash calculations in jsonb_path_ops GIN indexes (Tom Lane) When processing jsonb values that contain both scalars and sub-objects at the same nesting level, for example an array containing both scalars and sub-arrays, key hash values could be calculated differently than they would be for the same key in a different context. This could result in queries not finding entries that they should find. Fixing this means that existing indexes may now be inconsistent with the new hash calculation code. Users should REINDEX jsonb_path_ops GIN indexes after installing this update to make sure that all searches work as expected. Fix infinite loops and buffer-overrun problems in regular expressions (Tom Lane) Very large character ranges in bracket expressions could cause infinite loops in some cases, and memory overwrites in other cases. (CVE-2016-0773) Perform an immediate shutdown if the postmaster.pid file is removed (Tom Lane) The postmaster now checks every minute or so that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The main motivation for this change is to ensure that failed buildfarm runs will get cleaned up without manual intervention; but it also serves to limit the bad effects if a DBA forcibly removes postmaster.pid and then starts a new postmaster. In SERIALIZABLE transaction isolation mode, serialization anomalies could be missed due to race conditions during insertions (Kevin Grittner, Thomas Munro) Fix failure to emit appropriate WAL records when doing ALTER TABLE ... SET TABLESPACE for unlogged relations (Michael Paquier, Andres Freund) Even though the relation's data is unlogged, the move must be logged or the relation will be inaccessible after a standby is promoted to master. Fix possible misinitialization of unlogged relations at the end of crash recovery (Andres Freund, Michael Paquier) Ensure walsender slots are fully re-initialized when being re-used (Magnus Hagander) Fix ALTER COLUMN TYPE to reconstruct inherited check constraints properly (Tom Lane) Fix REASSIGN OWNED to change ownership of composite types properly (Álvaro Herrera) Fix REASSIGN OWNED and ALTER OWNER to correctly update granted-permissions lists when changing owners of data types, foreign data wrappers, or foreign servers (Bruce Momjian, Álvaro Herrera) Fix REASSIGN OWNED to ignore foreign user mappings, rather than fail (Álvaro Herrera) Fix possible crash after doing query rewrite for an updatable view (Stephen Frost) Fix planner's handling of LATERAL references (Tom Lane) This fixes some corner cases that led to failed to build any N-way joins or could not devise a query plan planner failures. Add more defenses against bad planner cost estimates for GIN index scans when the index's internal statistics are very out-of-date (Tom Lane) Make planner cope with hypothetical GIN indexes suggested by an index advisor plug-in (Julien Rouhaud) Speed up generation of unique table aliases in EXPLAIN and rule dumping, and ensure that generated aliases do not exceed NAMEDATALEN (Tom Lane) Fix dumping of whole-row Vars in ROW() and VALUES() lists (Tom Lane) Translation of minus-infinity dates and timestamps to json or jsonb incorrectly rendered them as plus-infinity (Tom Lane) Fix possible internal overflow in numeric division (Dean Rasheed) Fix enforcement of restrictions inside parentheses within regular expression lookahead constraints (Tom Lane) Lookahead constraints aren't allowed to contain backrefs, and parentheses within them are always considered non-capturing, according to the manual. However, the code failed to handle these cases properly inside a parenthesized subexpression, and would give unexpected results. Conversion of regular expressions to indexscan bounds could produce incorrect bounds from regexps containing lookahead constraints (Tom Lane) Fix regular-expression compiler to handle loops of constraint arcs (Tom Lane) The code added for CVE-2007-4772 was both incomplete, in that it didn't handle loops involving more than one state, and incorrect, in that it could cause assertion failures (though there seem to be no bad consequences of that in a non-assert build). Multi-state loops would cause the compiler to run until the query was canceled or it reached the too-many-states error condition. Improve memory-usage accounting in regular-expression compiler (Tom Lane) This causes the code to emit regular expression is too complex errors in some cases that previously used unreasonable amounts of time and memory. Improve performance of regular-expression compiler (Tom Lane) Make %h and %r escapes in log_line_prefix work for messages emitted due to log_connections (Tom Lane) Previously, %h/%r started to work just after a new session had emitted the connection received log message; now they work for that message too. On Windows, ensure the shared-memory mapping handle gets closed in child processes that don't need it (Tom Lane, Amit Kapila) This oversight resulted in failure to recover from crashes whenever logging_collector is turned on. Fix possible failure to detect socket EOF in non-blocking mode on Windows (Tom Lane) It's not entirely clear whether this problem can happen in pre-9.5 branches, but if it did, the symptom would be that a walsender process would wait indefinitely rather than noticing a loss of connection. Avoid leaking a token handle during SSPI authentication (Christian Ullrich) In psql, ensure that libreadline's idea of the screen size is updated when the terminal window size changes (Merlin Moncure) Previously, libreadline did not notice if the window was resized during query output, leading to strange behavior during later input of multiline queries. Fix psql's \det command to interpret its pattern argument the same way as other \d commands with potentially schema-qualified patterns do (Reece Hart) Avoid possible crash in psql's \c command when previous connection was via Unix socket and command specifies a new hostname and same username (Tom Lane) In pg_ctl start -w, test child process status directly rather than relying on heuristics (Tom Lane, Michael Paquier) Previously, pg_ctl relied on an assumption that the new postmaster would always create postmaster.pid within five seconds. But that can fail on heavily-loaded systems, causing pg_ctl to report incorrectly that the postmaster failed to start. Except on Windows, this change also means that a pg_ctl start -w done immediately after another such command will now reliably fail, whereas previously it would report success if done within two seconds of the first command. In pg_ctl start -w, don't attempt to use a wildcard listen address to connect to the postmaster (Kondo Yuta) On Windows, pg_ctl would fail to detect postmaster startup if listen_addresses is set to 0.0.0.0 or ::, because it would try to use that value verbatim as the address to connect to, which doesn't work. Instead assume that 127.0.0.1 or ::1, respectively, is the right thing to use. In pg_ctl on Windows, check service status to decide where to send output, rather than checking if standard output is a terminal (Michael Paquier) In pg_dump and pg_basebackup, adopt the GNU convention for handling tar-archive members exceeding 8GB (Tom Lane) The POSIX standard for tar file format does not allow archive member files to exceed 8GB, but most modern implementations of tar support an extension that fixes that. Adopt this extension so that pg_dump with -Ft no longer fails on tables with more than 8GB of data, and so that pg_basebackup can handle files larger than 8GB. In addition, fix some portability issues that could cause failures for members between 4GB and 8GB on some platforms. Potentially these problems could cause unrecoverable data loss due to unreadable backup files. Fix assorted corner-case bugs in pg_dump's processing of extension member objects (Tom Lane) Make pg_dump mark a view's triggers as needing to be processed after its rule, to prevent possible failure during parallel pg_restore (Tom Lane) Ensure that relation option values are properly quoted in pg_dump (Kouhei Sutou, Tom Lane) A reloption value that isn't a simple identifier or number could lead to dump/reload failures due to syntax errors in CREATE statements issued by pg_dump. This is not an issue with any reloption currently supported by core PostgreSQL, but extensions could allow reloptions that cause the problem. Avoid repeated password prompts during parallel pg_dump (Zeus Kronion) Fix pg_upgrade's file-copying code to handle errors properly on Windows (Bruce Momjian) Install guards in pgbench against corner-case overflow conditions during evaluation of script-specified division or modulo operators (Fabien Coelho, Michael Paquier) Fix failure to localize messages emitted by pg_receivexlog and pg_recvlogical (Ioseph Kim) Avoid dump/reload problems when using both plpython2 and plpython3 (Tom Lane) In principle, both versions of PL/Python can be used in the same database, though not in the same session (because the two versions of libpython cannot safely be used concurrently). However, pg_restore and pg_upgrade both do things that can fall foul of the same-session restriction. Work around that by changing the timing of the check. Fix PL/Python regression tests to pass with Python 3.5 (Peter Eisentraut) Fix premature clearing of libpq's input buffer when socket EOF is seen (Tom Lane) This mistake caused libpq to sometimes not report the backend's final error message before reporting server closed the connection unexpectedly. Prevent certain PL/Java parameters from being set by non-superusers (Noah Misch) This change mitigates a PL/Java security bug (CVE-2016-0766), which was fixed in PL/Java by marking these parameters as superuser-only. To fix the security hazard for sites that update PostgreSQL more frequently than PL/Java, make the core code aware of them also. Improve libpq's handling of out-of-memory situations (Michael Paquier, Amit Kapila, Heikki Linnakangas) Fix order of arguments in ecpg-generated typedef statements (Michael Meskes) Use %g not %f format in ecpg's PGTYPESnumeric_from_double() (Tom Lane) Fix ecpg-supplied header files to not contain comments continued from a preprocessor directive line onto the next line (Michael Meskes) Such a comment is rejected by ecpg. It's not yet clear whether ecpg itself should be changed. Fix hstore_to_json_loose()'s test for whether an hstore value can be converted to a JSON number (Tom Lane) Previously this function could be fooled by non-alphanumeric trailing characters, leading to emitting syntactically-invalid JSON. Ensure that contrib/pgcrypto's crypt() function can be interrupted by query cancel (Andreas Karlsson) In contrib/postgres_fdw, fix bugs triggered by use of tableoid in data-modifying commands (Etsuro Fujita, Robert Haas) Accept flex versions later than 2.5.x (Tom Lane, Michael Paquier) Now that flex 2.6.0 has been released, the version checks in our build scripts needed to be adjusted. Improve reproducibility of build output by ensuring filenames are given to the linker in a fixed order (Christoph Berg) This avoids possible bitwise differences in the produced executable files from one build to the next. Install our missing script where PGXS builds can find it (Jim Nasby) This allows sane behavior in a PGXS build done on a machine where build tools such as bison are missing. Ensure that dynloader.h is included in the installed header files in MSVC builds (Bruce Momjian, Michael Paquier) Add variant regression test expected-output file to match behavior of current libxml2 (Tom Lane) The fix for libxml2's CVE-2015-7499 causes it not to output error context reports in some cases where it used to do so. This seems to be a bug, but we'll probably have to live with it for some time, so work around it. Update time zone data files to tzdata release 2016a for DST law changes in Cayman Islands, Metlakatla, and Trans-Baikal Territory (Zabaykalsky Krai), plus historical corrections for Pakistan. 2015-10-08 This release contains a variety of fixes from 9.4.4. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you are upgrading from a version earlier than 9.4.4, see . Guard against stack overflows in json parsing (Oskari Saarenmaa) If an application constructs PostgreSQL json or jsonb values from arbitrary user input, the application's users can reliably crash the PostgreSQL server, causing momentary denial of service. (CVE-2015-5289) Fix contrib/pgcrypto to detect and report too-short crypt() salts (Josh Kupershmidt) Certain invalid salt arguments crashed the server or disclosed a few bytes of server memory. We have not ruled out the viability of attacks that arrange for presence of confidential information in the disclosed bytes, but they seem unlikely. (CVE-2015-5288) Fix subtransaction cleanup after a portal (cursor) belonging to an outer subtransaction fails (Tom Lane, Michael Paquier) A function executed in an outer-subtransaction cursor could cause an assertion failure or crash by referencing a relation created within an inner subtransaction. Fix possible deadlock during WAL insertion when commit_delay is set (Heikki Linnakangas) Ensure all relations referred to by an updatable view are properly locked during an update statement (Dean Rasheed) Fix insertion of relations into the relation cache init file (Tom Lane) An oversight in a patch in the most recent minor releases caused pg_trigger_tgrelid_tgname_index to be omitted from the init file. Subsequent sessions detected this, then deemed the init file to be broken and silently ignored it, resulting in a significant degradation in session startup time. In addition to fixing the bug, install some guards so that any similar future mistake will be more obvious. Avoid O(N^2) behavior when inserting many tuples into a SPI query result (Neil Conway) Improve LISTEN startup time when there are many unread notifications (Matt Newell) Fix performance problem when a session alters large numbers of foreign key constraints (Jan Wieck, Tom Lane) This was seen primarily when restoring pg_dump output for databases with many thousands of tables. Disable SSL renegotiation by default (Michael Paquier, Andres Freund) While use of SSL renegotiation is a good idea in theory, we have seen too many bugs in practice, both in the underlying OpenSSL library and in our usage of it. Renegotiation will be removed entirely in 9.5 and later. In the older branches, just change the default value of ssl_renegotiation_limit to zero (disabled). Lower the minimum values of the *_freeze_max_age parameters (Andres Freund) This is mainly to make tests of related behavior less time-consuming, but it may also be of value for installations with limited disk space. Limit the maximum value of wal_buffers to 2GB to avoid server crashes (Josh Berkus) Avoid logging complaints when a parameter that can only be set at server start appears multiple times in postgresql.conf, and fix counting of line numbers after an include_dir directive (Tom Lane) Fix rare internal overflow in multiplication of numeric values (Dean Rasheed) Guard against hard-to-reach stack overflows involving record types, range types, json, jsonb, tsquery, ltxtquery and query_int (Noah Misch) Fix handling of DOW and DOY in datetime input (Greg Stark) These tokens aren't meant to be used in datetime values, but previously they resulted in opaque internal error messages rather than invalid input syntax. Add more query-cancel checks to regular expression matching (Tom Lane) Add recursion depth protections to regular expression, SIMILAR TO, and LIKE matching (Tom Lane) Suitable search patterns and a low stack depth limit could lead to stack-overrun crashes. Fix potential infinite loop in regular expression execution (Tom Lane) A search pattern that can apparently match a zero-length string, but actually doesn't match because of a back reference, could lead to an infinite loop. In regular expression execution, correctly record match data for capturing parentheses within a quantifier even when the match is zero-length (Tom Lane) Fix low-memory failures in regular expression compilation (Andreas Seltenreich) Fix low-probability memory leak during regular expression execution (Tom Lane) Fix rare low-memory failure in lock cleanup during transaction abort (Tom Lane) Fix unexpected out-of-memory situation during sort errors when using tuplestores with small work_mem settings (Tom Lane) Fix very-low-probability stack overrun in qsort (Tom Lane) Fix invalid memory alloc request size failure in hash joins with large work_mem settings (Tomas Vondra, Tom Lane) Fix assorted planner bugs (Tom Lane) These mistakes could lead to incorrect query plans that would give wrong answers, or to assertion failures in assert-enabled builds, or to odd planner errors such as could not devise a query plan for the given query, could not find pathkey item to sort, plan should not reference subplan's variable, or failed to assign all NestLoopParams to plan nodes. Thanks are due to Andreas Seltenreich and Piotr Stefaniak for fuzz testing that exposed these problems. Improve planner's performance for UPDATE/DELETE on large inheritance sets (Tom Lane, Dean Rasheed) Ensure standby promotion trigger files are removed at postmaster startup (Michael Paquier, Fujii Masao) This prevents unwanted promotion from occurring if these files appear in a database backup that is used to initialize a new standby server. During postmaster shutdown, ensure that per-socket lock files are removed and listen sockets are closed before we remove the postmaster.pid file (Tom Lane) This avoids race-condition failures if an external script attempts to start a new postmaster as soon as pg_ctl stop returns. Ensure that the postmaster does not exit until all its child processes are gone, even in an immediate shutdown (Tom Lane) Like the previous item, this avoids possible race conditions against a subsequently-started postmaster. Fix postmaster's handling of a startup-process crash during crash recovery (Tom Lane) If, during a crash recovery cycle, the startup process crashes without having restored database consistency, we'd try to launch a new startup process, which typically would just crash again, leading to an infinite loop. Make emergency autovacuuming for multixact wraparound more robust (Andres Freund) Do not print a WARNING when an autovacuum worker is already gone when we attempt to signal it, and reduce log verbosity for such signals (Tom Lane) Prevent autovacuum launcher from sleeping unduly long if the server clock is moved backwards a large amount (Álvaro Herrera) Ensure that cleanup of a GIN index's pending-insertions list is interruptable by cancel requests (Jeff Janes) Allow all-zeroes pages in GIN indexes to be reused (Heikki Linnakangas) Such a page might be left behind after a crash. Fix handling of all-zeroes pages in SP-GiST indexes (Heikki Linnakangas) VACUUM attempted to recycle such pages, but did so in a way that wasn't crash-safe. Fix off-by-one error that led to otherwise-harmless warnings about apparent wraparound in subtrans/multixact truncation (Thomas Munro) Fix misreporting of CONTINUE and MOVE statement types in PL/pgSQL's error context messages (Pavel Stehule, Tom Lane) Fix PL/Perl to handle non-ASCII error message texts correctly (Alex Hunsaker) Fix PL/Python crash when returning the string representation of a record result (Tom Lane) Fix some places in PL/Tcl that neglected to check for failure of malloc() calls (Michael Paquier, Álvaro Herrera) In contrib/isn, fix output of ISBN-13 numbers that begin with 979 (Fabien Coelho) EANs beginning with 979 (but not 9790) are considered ISBNs, but they must be printed in the new 13-digit format, not the 10-digit format. Improve contrib/pg_stat_statements' handling of query-text garbage collection (Peter Geoghegan) The external file containing query texts could bloat to very large sizes; once it got past 1GB attempts to trim it would fail, soon leading to situations where the file could not be read at all. Improve contrib/postgres_fdw's handling of collation-related decisions (Tom Lane) The main user-visible effect is expected to be that comparisons involving varchar columns will be sent to the remote server for execution in more cases than before. Improve libpq's handling of out-of-memory conditions (Michael Paquier, Heikki Linnakangas) Fix memory leaks and missing out-of-memory checks in ecpg (Michael Paquier) Fix psql's code for locale-aware formatting of numeric output (Tom Lane) The formatting code invoked by \pset numericlocale on did the wrong thing for some uncommon cases such as numbers with an exponent but no decimal point. It could also mangle already-localized output from the money data type. Prevent crash in psql's \c command when there is no current connection (Noah Misch) Make pg_dump handle inherited NOT VALID check constraints correctly (Tom Lane) Fix selection of default zlib compression level in pg_dump's directory output format (Andrew Dunstan) Ensure that temporary files created during a pg_dump run with tar-format output are not world-readable (Michael Paquier) Fix pg_dump and pg_upgrade to support cases where the postgres or template1 database is in a non-default tablespace (Marti Raudsepp, Bruce Momjian) Fix pg_dump to handle object privileges sanely when dumping from a server too old to have a particular privilege type (Tom Lane) When dumping data types from pre-9.2 servers, and when dumping functions or procedural languages from pre-7.3 servers, pg_dump would produce GRANT/REVOKE commands that revoked the owner's grantable privileges and instead granted all privileges to PUBLIC. Since the privileges involved are just USAGE and EXECUTE, this isn't a security problem, but it's certainly a surprising representation of the older systems' behavior. Fix it to leave the default privilege state alone in these cases. Fix pg_dump to dump shell types (Tom Lane) Shell types (that is, not-yet-fully-defined types) aren't useful for much, but nonetheless pg_dump should dump them. Fix assorted minor memory leaks in pg_dump and other client-side programs (Michael Paquier) Fix pgbench's progress-report behavior when a query, or pgbench itself, gets stuck (Fabien Coelho) Fix spinlock assembly code for Alpha hardware (Tom Lane) Fix spinlock assembly code for PPC hardware to be compatible with AIX's native assembler (Tom Lane) Building with gcc didn't work if gcc had been configured to use the native assembler, which is becoming more common. On AIX, test the -qlonglong compiler option rather than just assuming it's safe to use (Noah Misch) On AIX, use -Wl,-brtllib link option to allow symbols to be resolved at runtime (Noah Misch) Perl relies on this ability in 5.8.0 and later. Avoid use of inline functions when compiling with 32-bit xlc, due to compiler bugs (Noah Misch) Use librt for sched_yield() when necessary, which it is on some Solaris versions (Oskari Saarenmaa) Translate encoding UHC as Windows code page 949 (Noah Misch) This fixes presentation of non-ASCII log messages from processes that are not attached to any particular database, such as the postmaster. On Windows, avoid failure when doing encoding conversion to UTF16 outside a transaction, such as for log messages (Noah Misch) Fix postmaster startup failure due to not copying setlocale()'s return value (Noah Misch) This has been reported on Windows systems with the ANSI code page set to CP936 (Chinese (Simplified, PRC)), and may occur with other multibyte code pages. Fix Windows install.bat script to handle target directory names that contain spaces (Heikki Linnakangas) Make the numeric form of the PostgreSQL version number (e.g., 90405) readily available to extension Makefiles, as a variable named VERSION_NUM (Michael Paquier) Update time zone data files to tzdata release 2015g for DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, and Uruguay. There is a new zone name America/Fort_Nelson for the Canadian Northern Rockies. 2015-06-12 This release contains a small number of fixes from 9.4.3. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you are upgrading an installation that was previously upgraded using a pg_upgrade version between 9.3.0 and 9.3.4 inclusive, see the first changelog entry below. Also, if you are upgrading from a version earlier than 9.4.2, see . Fix possible failure to recover from an inconsistent database state (Robert Haas) Recent PostgreSQL releases introduced mechanisms to protect against multixact wraparound, but some of that code did not account for the possibility that it would need to run during crash recovery, when the database may not be in a consistent state. This could result in failure to restart after a crash, or failure to start up a secondary server. The lingering effects of a previously-fixed bug in pg_upgrade could also cause such a failure, in installations that had used pg_upgrade versions between 9.3.0 and 9.3.4. The pg_upgrade bug in question was that it would set oldestMultiXid to 1 in pg_control even if the true value should be higher. With the fixes introduced in this release, such a situation will result in immediate emergency autovacuuming until a correct oldestMultiXid value can be determined. If that would pose a hardship, users can avoid it by doing manual vacuuming before upgrading to this release. In detail: Check whether pg_controldata reports Latest checkpoint's oldestMultiXid to be 1. If not, there's nothing to do. Look in PGDATA/pg_multixact/offsets to see if there's a file named 0000. If there is, there's nothing to do. Otherwise, for each table that has pg_class.relminmxid equal to 1, VACUUM that table with both and set to zero. (You can use the vacuum cost delay parameters described in to reduce the performance consequences for concurrent sessions.) Fix rare failure to invalidate relation cache init file (Tom Lane) With just the wrong timing of concurrent activity, a VACUUM FULL on a system catalog might fail to update the init file that's used to avoid cache-loading work for new sessions. This would result in later sessions being unable to access that catalog at all. This is a very ancient bug, but it's so hard to trigger that no reproducible case had been seen until recently. Avoid deadlock between incoming sessions and CREATE/DROP DATABASE (Tom Lane) A new session starting in a database that is the target of a DROP DATABASE command, or is the template for a CREATE DATABASE command, could cause the command to wait for five seconds and then fail, even if the new session would have exited before that. Improve planner's cost estimates for semi-joins and anti-joins with inner indexscans (Tom Lane, Tomas Vondra) This type of plan is quite cheap when all the join clauses are used as index scan conditions, even if the inner scan would nominally fetch many rows, because the executor will stop after obtaining one row. The planner only partially accounted for that effect, and would therefore overestimate the cost, leading it to possibly choose some other much less efficient plan type. 2015-06-04 This release contains a small number of fixes from 9.4.2. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you are upgrading from a version earlier than 9.4.2, see . Avoid failures while fsync'ing data directory during crash restart (Abhijit Menon-Sen, Tom Lane) In the previous minor releases we added a patch to fsync everything in the data directory after a crash. Unfortunately its response to any error condition was to fail, thereby preventing the server from starting up, even when the problem was quite harmless. An example is that an unwritable file in the data directory would prevent restart on some platforms; but it is common to make SSL certificate files unwritable by the server. Revise this behavior so that permissions failures are ignored altogether, and other types of failures are logged but do not prevent continuing. Also apply the same rules in initdb --sync-only. This case is less critical but it should act similarly. Fix pg_get_functiondef() to show functions' LEAKPROOF property, if set (Jeevan Chalke) Fix pushJsonbValue() to unpack jbvBinary objects (Andrew Dunstan) This change does not affect any behavior in the core code as of 9.4, but it avoids a corner case for possible third-party callers. Remove configure's check prohibiting linking to a threaded libpython on OpenBSD (Tom Lane) The failure this restriction was meant to prevent seems to not be a problem anymore on current OpenBSD versions. 2015-05-22 This release contains a variety of fixes from 9.4.1. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you use contrib/citext's regexp_matches() functions, see the changelog entry below about that. Also, if you are upgrading from a version earlier than 9.4.1, see . Avoid possible crash when client disconnects just before the authentication timeout expires (Benkocs Norbert Attila) If the timeout interrupt fired partway through the session shutdown sequence, SSL-related state would be freed twice, typically causing a crash and hence denial of service to other sessions. Experimentation shows that an unauthenticated remote attacker could trigger the bug somewhat consistently, hence treat as security issue. (CVE-2015-3165) Improve detection of system-call failures (Noah Misch) Our replacement implementation of snprintf() failed to check for errors reported by the underlying system library calls; the main case that might be missed is out-of-memory situations. In the worst case this might lead to information exposure, due to our code assuming that a buffer had been overwritten when it hadn't been. Also, there were a few places in which security-relevant calls of other system library functions did not check for failure. It remains possible that some calls of the *printf() family of functions are vulnerable to information disclosure if an out-of-memory error occurs at just the wrong time. We judge the risk to not be large, but will continue analysis in this area. (CVE-2015-3166) In contrib/pgcrypto, uniformly report decryption failures as Wrong key or corrupt data (Noah Misch) Previously, some cases of decryption with an incorrect key could report other error message texts. It has been shown that such variance in error reports can aid attackers in recovering keys from other systems. While it's unknown whether pgcrypto's specific behaviors are likewise exploitable, it seems better to avoid the risk by using a one-size-fits-all message. (CVE-2015-3167) Protect against wraparound of multixact member IDs (Álvaro Herrera, Robert Haas, Thomas Munro) Under certain usage patterns, the existing defenses against this might be insufficient, allowing pg_multixact/members files to be removed too early, resulting in data loss. The fix for this includes modifying the server to fail transactions that would result in overwriting old multixact member ID data, and improving autovacuum to ensure it will act proactively to prevent multixact member ID wraparound, as it does for transaction ID wraparound. Fix incorrect declaration of contrib/citext's regexp_matches() functions (Tom Lane) These functions should return setof text[], like the core functions they are wrappers for; but they were incorrectly declared as returning just text[]. This mistake had two results: first, if there was no match you got a scalar null result, whereas what you should get is an empty set (zero rows). Second, the g flag was effectively ignored, since you would get only one result array even if there were multiple matches. While the latter behavior is clearly a bug, there might be applications depending on the former behavior; therefore the function declarations will not be changed by default until PostgreSQL 9.5. In pre-9.5 branches, the old behavior exists in version 1.0 of the citext extension, while we have provided corrected declarations in version 1.1 (which is not installed by default). To adopt the fix in pre-9.5 branches, execute ALTER EXTENSION citext UPDATE TO '1.1' in each database in which citext is installed. (You can also update back to 1.0 if you need to undo that.) Be aware that either update direction will require dropping and recreating any views or rules that use citext's regexp_matches() functions. Render infinite dates and timestamps as infinity when converting to json, rather than throwing an error (Andrew Dunstan) Fix json/jsonb's populate_record() and to_record() functions to handle empty input properly (Andrew Dunstan) Fix incorrect checking of deferred exclusion constraints after a HOT update (Tom Lane) If a new row that potentially violates a deferred exclusion constraint is HOT-updated (that is, no indexed columns change and the row can be stored back onto the same table page) later in the same transaction, the exclusion constraint would be reported as violated when the check finally occurred, even if the row(s) the new row originally conflicted with had been deleted. Fix behavior when changing foreign key constraint deferrability status with ALTER TABLE ... ALTER CONSTRAINT (Tom Lane) Operations later in the same session or concurrent sessions might not honor the status change promptly. Fix planning of star-schema-style queries (Tom Lane) Sometimes, efficient scanning of a large table requires that index parameters be provided from more than one other table (commonly, dimension tables whose keys are needed to index a large fact table). The planner should be able to find such plans, but an overly restrictive search heuristic prevented it. Prevent improper reordering of antijoins (NOT EXISTS joins) versus other outer joins (Tom Lane) This oversight in the planner has been observed to cause could not find RelOptInfo for given relids errors, but it seems possible that sometimes an incorrect query plan might get past that consistency check and result in silently-wrong query output. Fix incorrect matching of subexpressions in outer-join plan nodes (Tom Lane) Previously, if textually identical non-strict subexpressions were used both above and below an outer join, the planner might try to re-use the value computed below the join, which would be incorrect because the executor would force the value to NULL in case of an unmatched outer row. Fix GEQO planner to cope with failure of its join order heuristic (Tom Lane) This oversight has been seen to lead to failed to join all relations together errors in queries involving LATERAL, and that might happen in other cases as well. Ensure that row locking occurs properly when the target of an UPDATE or DELETE is a security-barrier view (Stephen Frost) Use a file opened for read/write when syncing replication slot data during database startup (Andres Freund) On some platforms, the previous coding could result in errors like could not fsync file "pg_replslot/...": Bad file descriptor. Fix possible deadlock at startup when max_prepared_transactions is too small (Heikki Linnakangas) Don't archive useless preallocated WAL files after a timeline switch (Heikki Linnakangas) Recursively fsync() the data directory after a crash (Abhijit Menon-Sen, Robert Haas) This ensures consistency if another crash occurs shortly later. (The second crash would have to be a system-level crash, not just a database crash, for there to be a problem.) Fix autovacuum launcher's possible failure to shut down, if an error occurs after it receives SIGTERM (Álvaro Herrera) Fix failure to handle invalidation messages for system catalogs early in session startup (Tom Lane) This oversight could result in failures in sessions that start concurrently with a VACUUM FULL on a system catalog. Fix crash in BackendIdGetTransactionIds() when trying to get status for a backend process that just exited (Tom Lane) Cope with unexpected signals in LockBufferForCleanup() (Andres Freund) This oversight could result in spurious errors about multiple backends attempting to wait for pincount 1. Fix crash when doing COPY IN to a table with check constraints that contain whole-row references (Tom Lane) The known failure case only crashes in 9.4 and up, but there is very similar code in 9.3 and 9.2, so back-patch those branches as well. Avoid waiting for WAL flush or synchronous replication during commit of a transaction that was read-only so far as the user is concerned (Andres Freund) Previously, a delay could occur at commit in transactions that had written WAL due to HOT page pruning, leading to undesirable effects such as sessions getting stuck at startup if all synchronous replicas are down. Sessions have also been observed to get stuck in catchup interrupt processing when using synchronous replication; this will fix that problem as well. Avoid busy-waiting with short recovery_min_apply_delay values (Andres Freund) Fix crash when manipulating hash indexes on temporary tables (Heikki Linnakangas) Fix possible failure during hash index bucket split, if other processes are modifying the index concurrently (Tom Lane) Fix memory leaks in GIN index vacuum (Heikki Linnakangas) Check for interrupts while analyzing index expressions (Jeff Janes) ANALYZE executes index expressions many times; if there are slow functions in such an expression, it's desirable to be able to cancel the ANALYZE before that loop finishes. Ensure tableoid of a foreign table is reported correctly when a READ COMMITTED recheck occurs after locking rows in SELECT FOR UPDATE, UPDATE, or DELETE (Etsuro Fujita) Add the name of the target server to object description strings for foreign-server user mappings (Álvaro Herrera) Include the schema name in object identity strings for conversions (Álvaro Herrera) Recommend setting include_realm to 1 when using Kerberos/GSSAPI/SSPI authentication (Stephen Frost) Without this, identically-named users from different realms cannot be distinguished. For the moment this is only a documentation change, but it will become the default setting in PostgreSQL 9.5. Remove code for matching IPv4 pg_hba.conf entries to IPv4-in-IPv6 addresses (Tom Lane) This hack was added in 2003 in response to a report that some Linux kernels of the time would report IPv4 connections as having IPv4-in-IPv6 addresses. However, the logic was accidentally broken in 9.0. The lack of any field complaints since then shows that it's not needed anymore. Now we have reports that the broken code causes crashes on some systems, so let's just remove it rather than fix it. (Had we chosen to fix it, that would make for a subtle and potentially security-sensitive change in the effective meaning of IPv4 pg_hba.conf entries, which does not seem like a good thing to do in minor releases.) Fix status reporting for terminated background workers that were never actually started (Robert Haas) After a database crash, don't restart background workers that are marked BGW_NEVER_RESTART (Amit Khandekar) Report WAL flush, not insert, position in IDENTIFY_SYSTEM replication command (Heikki Linnakangas) This avoids a possible startup failure in pg_receivexlog. While shutting down service on Windows, periodically send status updates to the Service Control Manager to prevent it from killing the service too soon; and ensure that pg_ctl will wait for shutdown (Krystian Bigaj) Reduce risk of network deadlock when using libpq's non-blocking mode (Heikki Linnakangas) When sending large volumes of data, it's important to drain the input buffer every so often, in case the server has sent enough response data to cause it to block on output. (A typical scenario is that the server is sending a stream of NOTICE messages during COPY FROM STDIN.) This worked properly in the normal blocking mode, but not so much in non-blocking mode. We've modified libpq to opportunistically drain input when it can, but a full defense against this problem requires application cooperation: the application should watch for socket read-ready as well as write-ready conditions, and be sure to call PQconsumeInput() upon read-ready. In libpq, fix misparsing of empty values in URI connection strings (Thomas Fanghaenel) Fix array handling in ecpg (Michael Meskes) Fix psql to sanely handle URIs and conninfo strings as the first parameter to \connect (David Fetter, Andrew Dunstan, Álvaro Herrera) This syntax has been accepted (but undocumented) for a long time, but previously some parameters might be taken from the old connection instead of the given string, which was agreed to be undesirable. Suppress incorrect complaints from psql on some platforms that it failed to write ~/.psql_history at exit (Tom Lane) This misbehavior was caused by a workaround for a bug in very old (pre-2006) versions of libedit. We fixed it by removing the workaround, which will cause a similar failure to appear for anyone still using such versions of libedit. Recommendation: upgrade that library, or use libreadline. Fix pg_dump's rule for deciding which casts are system-provided casts that should not be dumped (Tom Lane) In pg_dump, fix failure to honor -Z compression level option together with -Fd (Michael Paquier) Make pg_dump consider foreign key relationships between extension configuration tables while choosing dump order (Gilles Darold, Michael Paquier, Stephen Frost) This oversight could result in producing dumps that fail to reload because foreign key constraints are transiently violated. Avoid possible pg_dump failure when concurrent sessions are creating and dropping temporary functions (Tom Lane) Fix dumping of views that are just VALUES(...) but have column aliases (Tom Lane) Ensure that a view's replication identity is correctly set to nothing during dump/restore (Marko Tiikkaja) Previously, if the view was involved in a circular dependency, it might wind up with an incorrect replication identity property. In pg_upgrade, force timeline 1 in the new cluster (Bruce Momjian) This change prevents upgrade failures caused by bogus complaints about missing WAL history files. In pg_upgrade, check for improperly non-connectable databases before proceeding (Bruce Momjian) In pg_upgrade, quote directory paths properly in the generated delete_old_cluster script (Bruce Momjian) In pg_upgrade, preserve database-level freezing info properly (Bruce Momjian) This oversight could cause missing-clog-file errors for tables within the postgres and template1 databases. Run pg_upgrade and pg_resetxlog with restricted privileges on Windows, so that they don't fail when run by an administrator (Muhammad Asif Naeem) Improve handling of readdir() failures when scanning directories in initdb and pg_basebackup (Marco Nenciarini) Fix slow sorting algorithm in contrib/intarray (Tom Lane) Fix compile failure on Sparc V8 machines (Rob Rowan) Silence some build warnings on OS X (Tom Lane) Update time zone data files to tzdata release 2015d for DST law changes in Egypt, Mongolia, and Palestine, plus historical changes in Canada and Chile. Also adopt revised zone abbreviations for the America/Adak zone (HST/HDT not HAST/HADT). 2015-02-05 This release contains a variety of fixes from 9.4.0. For information about new features in the 9.4 major release, see . A dump/restore is not required for those running 9.4.X. However, if you are a Windows user and are using the Norwegian (Bokmål) locale, manual action is needed after the upgrade to replace any Norwegian (Bokmål)_Norway or norwegian-bokmal locale names stored in PostgreSQL system catalogs with the plain-ASCII alias Norwegian_Norway. For details see Fix buffer overruns in to_char() (Bruce Momjian) When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241) Fix buffer overrun in replacement *printf() functions (Tom Lane) PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well. This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242) Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch) Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243) Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas) If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message's data as a new protocol message. An attacker able to submit crafted binary data within a command parameter might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244) Fix information leak via constraint-violation error messages (Stephen Frost) Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161) Lock down regression testing's temporary installations on Windows (Noah Misch) Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067) Cope with the Windows locale named Norwegian (Bokmål) (Heikki Linnakangas) Non-ASCII locale names are problematic since it's not clear what encoding they should be represented in. Map the troublesome locale name to a plain-ASCII alias, Norwegian_Norway. 9.4.0 mapped the troublesome name to norwegian-bokmal, but that turns out not to work on all Windows configurations. Norwegian_Norway is now recommended instead. Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane) In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug. Avoid possible deadlock while trying to acquire tuple locks in EvalPlanQual processing (Álvaro Herrera, Mark Kirkwood) Fix failure to wait when a transaction tries to acquire a FOR NO KEY EXCLUSIVE tuple lock, while multiple other transactions currently hold FOR SHARE locks (Álvaro Herrera) Improve performance of EXPLAIN with large range tables (Tom Lane) Fix jsonb Unicode escape processing, and in consequence disallow \u0000 (Tom Lane) Previously, the JSON Unicode escape \u0000 was accepted and was stored as those six characters; but that is indistinguishable from what is stored for the input \\u0000, resulting in ambiguity. Moreover, in cases where de-escaped textual output is expected, such as the ->> operator, the sequence was printed as \u0000, which does not meet the expectation that JSON escaping would be removed. (Consistent behavior would require emitting a zero byte, but PostgreSQL does not support zero bytes embedded in text strings.) 9.4.0 included an ill-advised attempt to improve this situation by adjusting JSON output conversion rules; but of course that could not fix the fundamental ambiguity, and it turned out to break other usages of Unicode escape sequences. Revert that, and to avoid the core problem, reject \u0000 in jsonb input. If a jsonb column contains a \u0000 value stored with 9.4.0, it will henceforth read out as though it were \\u0000, which is the other valid interpretation of the data stored by 9.4.0 for this case. The json type did not have the storage-ambiguity problem, but it did have the problem of inconsistent de-escaped textual output. Therefore \u0000 will now also be rejected in json values when conversion to de-escaped form is required. This change does not break the ability to store \u0000 in json columns so long as no processing is done on the values. This is exactly parallel to the cases in which non-ASCII Unicode escapes are allowed when the database encoding is not UTF8. Fix namespace handling in xpath() (Ali Akbar) Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation. Fix assorted oversights in range-operator selectivity estimation (Emre Hasegeli) This patch fixes corner-case unexpected operator NNNN planner errors, and improves the selectivity estimates for some other cases. Revert unintended reduction in maximum size of a GIN index item (Heikki Linnakangas) 9.4.0 could fail with index row size exceeds maximum errors for data that previous versions would accept. Fix query-duration memory leak during repeated GIN index rescans (Heikki Linnakangas) Fix possible crash when using nonzero gin_fuzzy_search_limit (Heikki Linnakangas) Assorted fixes for logical decoding (Andres Freund) Fix incorrect replay of WAL parameter change records that report changes in the wal_log_hints setting (Petr Jelinek) Change pgstat wait timeout warning message to be LOG level, and rephrase it to be more understandable (Tom Lane) This message was originally thought to be essentially a can't-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads using stale statistics instead of current ones because stats collector is not responding. Warn if OS X's setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch) Fix libpq's behavior when /etc/passwd isn't readable (Tom Lane) While doing PQsetdbLogin(), libpq attempts to ascertain the user's operating system name, which on most Unix platforms involves reading /etc/passwd. As of 9.4, failure to do that was treated as a hard error. Restore the previous behavior, which was to fail only if the application does not provide a database role name to connect as. This supports operation in chroot environments that lack an /etc/passwd file. Improve consistency of parsing of psql's special variables (Tom Lane) Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors. Fix pg_dump to handle comments on event triggers without failing (Tom Lane) Allow parallel pg_dump to use --serializable-deferrable (Kevin Grittner) Prevent WAL files created by pg_basebackup -x/-X from being archived again when the standby is promoted (Andres Freund) Handle unexpected query results, especially NULLs, safely in contrib/tablefunc's connectby() (Michael Paquier) connectby() previously crashed if it encountered a NULL key value. It now prints that row but doesn't recurse further. Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier) These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues. Allow CFLAGS from configure's environment to override automatically-supplied CFLAGS (Tom Lane) Previously, configure would add any switches that it chose of its own accord to the end of the user-specified CFLAGS string. Since most compilers process switches left-to-right, this meant that configure's choices would override the user-specified flags in case of conflicts. That should work the other way around, so adjust the logic to put the user's string at the end not the beginning. Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane) This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations. Add CST (China Standard Time) to our lists of timezone abbreviations (Tom Lane) Update time zone data files to tzdata release 2015a for DST law changes in Chile and Mexico, plus historical changes in Iceland. 2014-12-18 Major enhancements in PostgreSQL 9.4 include: Add jsonb, a more capable and efficient data type for storing JSON data Add new SQL command for changing postgresql.conf configuration file entries Reduce lock strength for some commands Allow materialized views to be refreshed without blocking concurrent reads Add support for logical decoding of WAL data, to allow database changes to be streamed out in a customizable format Allow background worker processes to be dynamically registered, started and terminated The above items are explained in more detail in the sections below. A dump/restore using , or use of , is required for those wishing to migrate data from any previous release. Version 9.4 contains a number of changes that may affect compatibility with previous releases. Observe the following incompatibilities: Tighten checks for multidimensional array input (Bruce Momjian) Previously, an input array string that started with a single-element sub-array could later contain multi-element sub-arrays, e.g. '{{1}, {2,3}}'::int[] would be accepted. When converting values of type date, timestamp or timestamptz to JSON, render the values in a format compliant with ISO 8601 (Andrew Dunstan) Previously such values were rendered according to the current setting; but many JSON processors require timestamps to be in ISO 8601 format. If necessary, the previous behavior can be obtained by explicitly casting the datetime value to text before passing it to the JSON conversion function. The json #> text[] path extraction operator now returns its lefthand input, not NULL, if the array is empty (Tom Lane) This is consistent with the notion that this represents zero applications of the simple field/element extraction operator ->. Similarly, json #>> text[] with an empty array merely coerces its lefthand input to text. Corner cases in the JSON field/element/path extraction operators now return NULL rather than raising an error (Tom Lane) For example, applying field extraction to a JSON array now yields NULL not an error. This is more consistent (since some comparable cases such as no-such-field already returned NULL), and it makes it safe to create expression indexes that use these operators, since they will now not throw errors for any valid JSON input. Cause consecutive whitespace in to_timestamp() and to_date() format strings to consume a corresponding number of characters in the input string (whitespace or not), then conditionally consume adjacent whitespace, if not in FX mode (Jeevan Chalke) Previously, consecutive whitespace characters in a non-FX format string behaved like a single whitespace character and consumed all adjacent whitespace in the input string. For example, previously a format string of three spaces would consume only the first space in ' 12', but it will now consume all three characters. Fix ts_rank_cd() to ignore stripped lexemes (Alex Hill) Previously, stripped lexemes were treated as if they had a default location, producing a rank of dubious usefulness. For functions declared to take VARIADIC "any", an actual parameter marked as VARIADIC must be of a determinable array type (Pavel Stehule) Such parameters can no longer be written as an undecorated string literal or NULL; a cast to an appropriate array data type will now be required. Note that this does not affect parameters not marked VARIADIC. Ensure that whole-row variables expose the expected column names to functions that pay attention to column names within composite arguments (Tom Lane) Constructs like row_to_json(tab.*) now always emit column names that match the column aliases visible for table tab at the point of the call. In previous releases the emitted column names would sometimes be the table's actual column names regardless of any aliases assigned in the query. now also discards sequence-related state (Fabrízio de Royes Mello, Robert Haas) Rename EXPLAIN ANALYZE's total runtime output to execution time (Tom Lane) Now that planning time is also reported, the previous name was confusing. SHOW TIME ZONE now outputs simple numeric UTC offsets in POSIX timezone format (Tom Lane) Previously, such timezone settings were displayed as interval values. The new output is properly interpreted by SET TIME ZONE when passed as a simple string, whereas the old output required special treatment to be re-parsed correctly. Foreign data wrappers that support updating foreign tables must consider the possible presence of AFTER ROW triggers (Noah Misch) When an AFTER ROW trigger is present, all columns of the table must be returned by updating actions, since the trigger might inspect any or all of them. Previously, foreign tables never had triggers, so the FDW might optimize away fetching columns not mentioned in the RETURNING clause (if any). Prevent CHECK constraints from referencing system columns, except tableoid (Amit Kapila) Previously such check constraints were allowed, but they would often cause errors during restores. Use the last specified recovery target parameter if multiple target parameters are specified (Heikki Linnakangas) Previously, there was an undocumented precedence order among the recovery_target_xxx parameters. On Windows, automatically preserve quotes in command strings supplied by the user (Heikki Linnakangas) User commands that did their own quote preservation might need adjustment. This is likely to be an issue for commands used in , , and COPY TO/FROM PROGRAM. Remove catalog column pg_class.reltoastidxid (Michael Paquier) Remove catalog column pg_rewrite.ev_attr (Kevin Grittner) Per-column rules have not been supported since PostgreSQL 7.3. Remove native support for Kerberos authentication (--with-krb5, etc) (Magnus Hagander) The supported way to use Kerberos authentication is with GSSAPI. The native code has been deprecated since PostgreSQL 8.3. In PL/Python, handle domains over arrays like the underlying array type (Rodolfo Campero) Previously such values were treated as strings. Make libpq's PQconnectdbParams() and PQpingParams() functions process zero-length strings as defaults (Adrian Vondendriesch) Previously, these functions treated zero-length string values as selecting the default in only some cases. Change empty arrays returned by the module to be zero-dimensional arrays (Bruce Momjian) Previously, empty arrays were returned as zero-length one-dimensional arrays, whose text representation looked the same as zero-dimensional arrays ({}), but they acted differently in array operations. intarray's behavior in this area now matches the built-in array operators. now uses -U or --username to specify the user name (Bruce Momjian) Previously this option was spelled -u or --user, but that was inconsistent with other tools. Below you will find a detailed account of the changes between PostgreSQL 9.4 and the previous major release. Allow background worker processes to be dynamically registered, started and terminated (Robert Haas) The new worker_spi module shows an example of use of this feature. Allow dynamic allocation of shared memory segments (Robert Haas, Amit Kapila) This feature is illustrated in the test_shm_mq module. During crash recovery or immediate shutdown, send uncatchable termination signals (SIGKILL) to child processes that do not shut down promptly (MauMau, Álvaro Herrera) This reduces the likelihood of leaving orphaned child processes behind after shutdown, as well as ensuring that crash recovery can proceed if some child processes have become stuck. Improve randomness of the database system identifier (Tom Lane) Make properly report dead but not-yet-removable rows to the statistics collector (Hari Babu) Previously these were reported as live rows. Reduce GIN index size (Alexander Korotkov, Heikki Linnakangas) Indexes upgraded via will work fine but will still be in the old, larger GIN format. Use to recreate old GIN indexes in the new format. Improve speed of multi-key GIN lookups (Alexander Korotkov, Heikki Linnakangas) Add GiST index support for inet and cidr data types (Emre Hasegeli) Such indexes improve subnet and supernet lookups and ordering comparisons. Fix rare race condition in B-tree page deletion (Heikki Linnakangas) Make the handling of interrupted B-tree page splits more robust (Heikki Linnakangas) Allow multiple backends to insert into WAL buffers concurrently (Heikki Linnakangas) This improves parallel write performance. Conditionally write only the modified portion of updated rows to WAL (Amit Kapila) Improve performance of aggregate functions used as window functions (David Rowley, Florian Pflug, Tom Lane) Improve speed of aggregates that use numeric state values (Hadi Moshayedi) Attempt to freeze tuples when tables are rewritten with or VACUUM FULL (Robert Haas, Andres Freund) This can avoid the need to freeze the tuples in the future. Improve speed of with default nextval() columns (Simon Riggs) Improve speed of accessing many different sequences in the same session (David Rowley) Raise hard limit on the number of tuples held in memory during sorting and B-tree index builds (Noah Misch) Reduce memory allocated by PL/pgSQL blocks (Tom Lane) Make the planner more aggressive about extracting restriction clauses from mixed AND/OR clauses (Tom Lane) Disallow pushing volatile WHERE clauses down into DISTINCT subqueries (Tom Lane) Pushing down a WHERE clause can produce a more efficient plan overall, but at the cost of evaluating the clause more often than is implied by the text of the query; so don't do it if the clause contains any volatile functions. Auto-resize the catalog caches (Heikki Linnakangas) This reduces memory consumption for sessions accessing only a few tables, and improves performance for sessions accessing many tables. Add system view to report WAL archiver activity (Gabriele Bartolini) Add n_mod_since_analyze columns to and related system views (Mark Kirkwood) These columns expose the system's estimate of the number of changed tuples since the table's last . This estimate drives decisions about when to auto-analyze. Add backend_xid and backend_xmin columns to the system view , and a backend_xmin column to (Christian Kruse) Add support for SSL ECDH key exchange (Marko Kreen) This allows use of Elliptic Curve keys for server authentication. Such keys are faster and have better security than RSA keys. The new configuration parameter controls which curve is used for ECDH. Improve the default setting (Marko Kreen) By default, the server not the client now controls the preference order of SSL ciphers (Marko Kreen) Previously, the order specified by was usually ignored in favor of client-side defaults, which are not configurable in most PostgreSQL clients. If desired, the old behavior can be restored via the new configuration parameter . Make show SSL encryption information (Andreas Kunert) Improve SSL renegotiation handling (Álvaro Herrera) Add new SQL command for changing postgresql.conf configuration file entries (Amit Kapila) Previously such settings could only be changed by manually editing postgresql.conf. Add configuration parameter to control the amount of memory used by autovacuum workers (Peter Geoghegan) Add parameter to allow using huge memory pages on Linux (Christian Kruse, Richard Poole, Abhijit Menon-Sen) This can improve performance on large-memory systems. Add parameter to limit the number of background workers (Robert Haas) This is helpful in configuring a standby server to have the required number of worker processes (the same as the primary). Add superuser-only parameter to load libraries at session start (Peter Eisentraut) In contrast to , this parameter can load any shared library, not just those in the $libdir/plugins directory. Add parameter to enable WAL logging of hint-bit changes (Sawada Masahiko) Hint bit changes are not normally logged, except when checksums are enabled. This is useful for external tools like pg_rewind. Increase the default settings of and by four times (Bruce Momjian) The new defaults are 4MB and 64MB respectively. Increase the default setting of to 4GB (Bruce Momjian, Tom Lane) Allow printf-style space padding to be specified in (David Rowley) Allow terabyte units (TB) to be used when specifying configuration variable values (Simon Riggs) Show PIDs of lock holders and waiters and improve information about relations in log messages (Christian Kruse) Reduce server logging level when loading shared libraries (Peter Geoghegan) The previous level was LOG, which was too verbose for libraries loaded per-session. On Windows, make SQL_ASCII-encoded databases and server processes (e.g., ) emit messages in the character encoding of the server's Windows user locale (Alexander Law, Noah Misch) Previously these messages were output in the Windows ANSI code page. Add replication slots to coordinate activity on streaming standbys with the node they are streaming from (Andres Freund, Robert Haas) Replication slots allow preservation of resources like WAL files on the primary until they are no longer needed by standby servers. Add recovery parameter to delay replication (Robert Haas, Fabrízio de Royes Mello, Simon Riggs) Delaying replay on standby servers can be useful for recovering from user errors. Add option immediate to stop WAL recovery as soon as a consistent state is reached (MauMau, Heikki Linnakangas) Improve recovery target processing (Heikki Linnakangas) The timestamp reported by pg_last_xact_replay_timestamp() now reflects already-committed records, not transactions about to be committed. Recovering to a restore point now replays the restore point, rather than stopping just before the restore point. pg_switch_xlog() now clears any unused trailing space in the old WAL file (Heikki Linnakangas) This improves the compression ratio for WAL files. Report failure return codes from external recovery commands (Peter Eisentraut) Reduce spinlock contention during WAL replay (Heikki Linnakangas) Write WAL records of running transactions more frequently (Andres Freund) This allows standby servers to start faster and clean up resources more aggressively. Logical decoding allows database changes to be streamed in a configurable format. The data is read from the WAL and transformed into the desired target format. To implement this feature, the following changes were made: Add support for logical decoding of WAL data, to allow database changes to be streamed out in a customizable format (Andres Freund) Add new setting logical to enable logical change-set encoding in WAL (Andres Freund) Add table-level parameter REPLICA IDENTITY to control logical replication (Andres Freund) Add relation option user_catalog_table to identify user-created tables involved in logical change-set encoding (Andres Freund) Add application to receive logical-decoding data (Andres Freund) Add module to illustrate logical decoding at the SQL level (Andres Freund) Add WITH ORDINALITY syntax to number the rows returned from a set-returning function in the FROM clause (Andrew Gierth, David Fetter) This is particularly useful for functions like unnest(). Add ROWS FROM() syntax to allow horizontal concatenation of set-returning functions in the FROM clause (Andrew Gierth) Allow to have an empty target list (Tom Lane) This was added so that views that select from a table with zero columns can be dumped and restored correctly. Ensure that SELECT ... FOR UPDATE NOWAIT does not wait in corner cases involving already-concurrently-updated tuples (Craig Ringer and Thomas Munro) Add DISCARD SEQUENCES command to discard cached sequence-related state (Fabrízio de Royes Mello, Robert Haas) DISCARD ALL will now also discard such information. Add FORCE NULL option to COPY FROM, which causes quoted strings matching the specified null string to be converted to NULLs in CSV mode (Ian Barwick, Michael Paquier) Without this option, only unquoted matching strings will be imported as null values. Issue warnings for commands used outside of transaction blocks when they can have no effect (Bruce Momjian) New warnings are issued for SET LOCAL, SET CONSTRAINTS, SET TRANSACTION and ABORT when used outside a transaction block. Make EXPLAIN ANALYZE show planning time (Andreas Karlsson) Make EXPLAIN show the grouping columns in Agg and Group nodes (Tom Lane) Make EXPLAIN ANALYZE show exact and lossy block counts in bitmap heap scans (Etsuro Fujita) Allow a materialized view to be refreshed without blocking other sessions from reading the view meanwhile (Kevin Grittner) This is done with REFRESH MATERIALIZED VIEW CONCURRENTLY. Allow views to be automatically updated even if they contain some non-updatable columns (Dean Rasheed) Previously the presence of non-updatable output columns such as expressions, literals, and function calls prevented automatic updates. Now INSERTs, UPDATEs and DELETEs are supported, provided that they do not attempt to assign new values to any of the non-updatable columns. Allow control over whether INSERTs and UPDATEs can add rows to an auto-updatable view that would not appear in the view (Dean Rasheed) This is controlled with the new clause WITH CHECK OPTION. Allow security barrier views to be automatically updatable (Dean Rasheed) Support triggers on foreign tables (Ronan Dunklau) Allow moving groups of objects from one tablespace to another using the ALL IN TABLESPACE ... SET TABLESPACE form of , , or (Stephen Frost) Allow changing foreign key constraint deferrability via ... ALTER CONSTRAINT (Simon Riggs) Reduce lock strength for some commands (Simon Riggs, Noah Misch, Robert Haas) Specifically, VALIDATE CONSTRAINT, CLUSTER ON, SET WITHOUT CLUSTER, ALTER COLUMN SET STATISTICS, ALTER COLUMN SET (attribute_option), ALTER COLUMN RESET (attribute_option) no longer require ACCESS EXCLUSIVE locks. Allow tablespace options to be set in (Vik Fearing) Formerly these options could only be set via . Allow to define the estimated size of the aggregate's transition state data (Hadi Moshayedi) Proper use of this feature allows the planner to better estimate how much memory will be used by aggregates. Fix DROP IF EXISTS to avoid errors for non-existent objects in more cases (Pavel Stehule, Dean Rasheed) Improve how system relations are identified (Andres Freund, Robert Haas) Previously, relations once moved into the pg_catalog schema could no longer be modified or dropped. Fully implement the line data type (Peter Eisentraut) The line segment data type (lseg) has always been fully supported. The previous line data type (which was enabled only via a compile-time option) is not binary or dump-compatible with the new implementation. Add pg_lsn data type to represent a WAL log sequence number (LSN) (Robert Haas, Michael Paquier) Allow single-point polygons to be converted to circles (Bruce Momjian) Support time zone abbreviations that change UTC offset from time to time (Tom Lane) Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date. Allow 5+ digit years for non-ISO timestamp and date strings, where appropriate (Bruce Momjian) Add checks for overflow/underflow of interval values (Bruce Momjian) Add jsonb, a more capable and efficient data type for storing JSON data (Oleg Bartunov, Teodor Sigaev, Alexander Korotkov, Peter Geoghegan, Andrew Dunstan) This new type allows faster access to values within a JSON document, and faster and more useful indexing of JSON columns. Scalar values in jsonb documents are stored as appropriate scalar SQL types, and the JSON document structure is pre-parsed rather than being stored as text as in the original json data type. Add new JSON functions to allow for the construction of arbitrarily complex JSON trees (Andrew Dunstan, Laurence Rowe) New functions include json_array_elements_text(), json_build_array(), json_object(), json_object_agg(), json_to_record(), and json_to_recordset(). Add json_typeof() to return the data type of a json value (Andrew Tipton) Add pg_sleep_for(interval) and pg_sleep_until(timestamp) to specify delays more flexibly (Vik Fearing, Julien Rouhaud) The existing pg_sleep() function only supports delays specified in seconds. Add cardinality() function for arrays (Marko Tiikkaja) This returns the total number of elements in the array, or zero for an array with no elements. Add SQL functions to allow large object reads/writes at arbitrary offsets (Pavel Stehule) Allow unnest() to take multiple arguments, which are individually unnested then horizontally concatenated (Andrew Gierth) Add functions to construct times, dates, timestamps, timestamptzs, and intervals from individual values, rather than strings (Pavel Stehule) These functions' names are prefixed with make_, e.g. make_date(). Make to_char()'s TZ format specifier return a useful value for simple numeric time zone offsets (Tom Lane) Previously, to_char(CURRENT_TIMESTAMP, 'TZ') returned an empty string if the timezone was set to a constant like -4. Add timezone offset format specifier OF to to_char() (Bruce Momjian) Improve the random seed used for random() (Honza Horak) Tighten validity checking for Unicode code points in chr(int) (Tom Lane) This function now only accepts values that are valid UTF8 characters according to RFC 3629. Add functions for looking up objects in pg_class, pg_proc, pg_type, and pg_operator that do not generate errors for non-existent objects (Yugo Nagata, Nozomi Anzai, Robert Haas) For example, to_regclass() does a lookup in pg_class similarly to the regclass input function, but it returns NULL for a non-existent object instead of failing. Add function pg_filenode_relation() to allow for more efficient lookup of relation names from filenodes (Andres Freund) Add parameter_default column to information_schema.parameters view (Peter Eisentraut) Make information_schema.schemata show all accessible schemas (Peter Eisentraut) Previously it only showed schemas owned by the current user. Add control over which rows are passed into aggregate functions via the FILTER clause (David Fetter) Support ordered-set (WITHIN GROUP) aggregates (Atri Sharma, Andrew Gierth, Tom Lane) Add standard ordered-set aggregates percentile_cont(), percentile_disc(), mode(), rank(), dense_rank(), percent_rank(), and cume_dist() (Atri Sharma, Andrew Gierth) Support VARIADIC aggregate functions (Tom Lane) Allow polymorphic aggregates to have non-polymorphic state data types (Tom Lane) This allows proper declaration in SQL of aggregates like the built-in aggregate array_agg(). Add event trigger support to PL/Perl and PL/Tcl (Dimitri Fontaine) Convert numeric values to decimal in PL/Python (Szymon Guz, Ronan Dunklau) Previously such values were converted to Python float values, risking loss of precision. Add ability to retrieve the current PL/PgSQL call stack using GET DIAGNOSTICS (Pavel Stehule, Stephen Frost) Add option print_strict_params to display the parameters passed to a query that violated a STRICT constraint (Marko Tiikkaja) Add variables plpgsql.extra_warnings and plpgsql.extra_errors to enable additional PL/pgSQL warnings and errors (Marko Tiikkaja, Petr Jelinek) Currently only warnings/errors about shadowed variables are available. Make libpq's PQconndefaults() function ignore invalid service files (Steve Singer, Bruce Momjian) Previously it returned NULL if an incorrect service file was encountered. Accept TLS protocol versions beyond TLSv1 in libpq (Marko Kreen) Add option -g to specify role membership (Chistopher Browne) Add option --analyze-in-stages to analyze in stages of increasing granularity (Peter Eisentraut) This allows minimal statistics to be created quickly. Make with option -n output current and potentially changed values (Rajeev Rastogi) Make throw error for incorrect locale settings, rather than silently falling back to a default choice (Tom Lane) Make return exit code 4 for an inaccessible data directory (Amit Kapila, Bruce Momjian) This behavior more closely matches the Linux Standard Base (LSB) Core Specification. On Windows, ensure that a non-absolute -D path specification is interpreted relative to 's current directory (Kumar Rajeev Rastogi) Previously it would be interpreted relative to whichever directory the underlying Windows service was started in. Allow sizeof() in ECPG C array definitions (Michael Meskes) Make ECPG properly handle nesting of C-style comments in both C and SQL text (Michael Meskes) Suppress No rows output in psql expanded mode when the footer is disabled (Bruce Momjian) Allow Control-C to abort psql when it's hung at connection startup (Peter Eisentraut) Make psql's \db+ show tablespace options (Magnus Hagander) Make \do+ display the functions that implement the operators (Marko Tiikkaja) Make \d+ output an OID line only if an oid column exists in the table (Bruce Momjian) Previously, the presence or absence of an oid column was always reported. Make \d show disabled system triggers (Bruce Momjian) Previously, if you disabled all triggers, only user triggers would show as disabled. Fix \copy to no longer require a space between stdin and a semicolon (Etsuro Fujita) Output the row count at the end of \copy, just like COPY already did (Kumar Rajeev Rastogi) Fix \conninfo to display the server's IP address for connections using hostaddr (Fujii Masao) Previously \conninfo could not display the server's IP address in such cases. Show the SSL protocol version in \conninfo (Marko Kreen) Add tab completion for \pset (Pavel Stehule) Allow \pset with no arguments to show all settings (Gilles Darold) Make \s display the name of the history file it wrote without converting it to an absolute path (Tom Lane) The code previously attempted to convert a relative file name to an absolute path for display, but frequently got it wrong. Allow options -I, -P, -T and -n to be specified multiple times (Heikki Linnakangas) This allows multiple objects to be restored in one operation. Optionally add IF EXISTS clauses to the DROP commands emitted when removing old objects during a restore (Pavel Stehule) This change prevents unnecessary errors when removing old objects. The new --if-exists option for , , and is only available when --clean is also specified. Add pg_basebackup option --xlogdir to specify the pg_xlog directory location (Haribabu Kommi) Allow pg_basebackup to relocate tablespaces in the backup copy (Steeve Lennmark) This is particularly useful for using pg_basebackup on the same machine as the primary. Allow network-stream base backups to be throttled (Antonin Houska) This can be controlled with the pg_basebackup --max-rate parameter. Improve the way tuples are frozen to preserve forensic information (Robert Haas, Andres Freund) This change removes the main objection to freezing tuples as soon as possible. Code that inspects tuple flag bits will need to be modified. No longer require function prototypes for functions marked with the PG_FUNCTION_INFO_V1 macro (Peter Eisentraut) This change eliminates the need to write boilerplate prototypes. Note that the PG_FUNCTION_INFO_V1 macro must appear before the corresponding function definition to avoid compiler warnings. Remove SnapshotNow and HeapTupleSatisfiesNow() (Robert Haas) All existing uses have been switched to more appropriate snapshot types. Catalog scans now use MVCC snapshots. Add an API to allow memory allocations over one gigabyte (Noah Misch) Add psprintf() to simplify memory allocation during string composition (Peter Eisentraut, Tom Lane) Support printf() size modifier z to print size_t values (Andres Freund) Change API of appendStringInfoVA() to better use vsnprintf() (David Rowley, Tom Lane) Allow new types of external toast datums to be created (Andres Freund) Add single-reader, single-writer, lightweight shared message queue (Robert Haas) Improve spinlock speed on x86_64 CPUs (Heikki Linnakangas) Remove spinlock support for unsupported platforms SINIX, Sun3, and NS32K (Robert Haas) Remove IRIX port (Robert Haas) Reduce the number of semaphores required by --disable-spinlocks builds (Robert Haas) Rewrite duplicate_oids Unix shell script in Perl (Andrew Dunstan) Add Test Anything Protocol (TAP) tests for client programs (Peter Eisentraut) Currently, these tests are run by make check-world only if the --enable-tap-tests option was given to configure. This might become the default behavior in some future release. Add make targets check-tests and installcheck-tests, which allow selection of individual tests to be run (Andrew Dunstan) Remove maintainer-check makefile rule (Peter Eisentraut) The default build rules now include all the formerly-optional tests. Improve support for VPATH builds of PGXS modules (Cédric Villemain, Andrew Dunstan, Peter Eisentraut) Upgrade to Autoconf 2.69 (Peter Eisentraut) Add a configure flag that appends custom text to the PG_VERSION string (Oskari Saarenmaa) This is useful for packagers building custom binaries. Improve DocBook XML validity (Peter Eisentraut) Fix various minor security and sanity issues reported by the Coverity scanner (Stephen Frost) Improve detection of invalid memory usage when testing PostgreSQL with Valgrind (Noah Misch) Improve sample Emacs configuration file emacs.samples (Peter Eisentraut) Also add .dir-locals.el to the top of the source tree. Allow pgindent to accept a command-line list of typedefs (Bruce Momjian) Make pgindent smarter about blank lines around preprocessor conditionals (Bruce Momjian) Avoid most uses of dlltool in Cygwin and Mingw builds (Marco Atzeri, Hiroshi Inoue) Support client-only installs in MSVC (Windows) builds (MauMau) Add extension to preload relation data into the shared buffer cache at server start (Robert Haas) This allows reaching full operating performance more quickly. Add UUID random number generator gen_random_uuid() to (Oskari Saarenmaa) This allows creation of version 4 UUIDs without requiring installation of . Allow to work with the BSD or e2fsprogs UUID libraries, not only the OSSP UUID library (Matteo Beccati) This improves the uuid-ossp module's portability since it no longer has to have the increasingly-obsolete OSSP library. The module's name is now rather a misnomer, but we won't change it. Add option to to include trigger execution time (Horiguchi Kyotaro) Fix to not report rows from uncommitted transactions as dead (Robert Haas) Make functions use regclass-type arguments (Satoshi Nagayasu) While text-type arguments are still supported, they may be removed in a future major release. Improve consistency of output to honor snapshot rules more consistently (Robert Haas) Improve 's choice of trigrams for indexed regular expression searches (Alexander Korotkov) This change discourages use of trigrams containing whitespace, which are usually less selective. Allow to report a live log stream with --follow (Heikki Linnakangas) Store data more compactly (Stas Kelvich) Existing data must be dumped/restored to use the new format. The old format can still be read. Reduce client-side memory usage by using a cursor (Andrew Dunstan) Dramatically reduce memory consumption in (Bruce Momjian) Pass 's user name (-U) option to generated analyze scripts (Bruce Momjian) Remove line length limit for pgbench scripts (Sawada Masahiko) The previous line limit was BUFSIZ. Add long option names to pgbench (Fabien Coelho) Add pgbench option --rate to control the transaction rate (Fabien Coelho) Add pgbench option --progress to print periodic progress reports (Fabien Coelho) Make pg_stat_statements use a file, rather than shared memory, for query text storage (Peter Geoghegan) This removes the previous limitation on query text length, and allows a higher number of unique statements to be tracked by default. Allow reporting of pg_stat_statements's internal query hash identifier (Daniel Farina, Sameer Thakur, Peter Geoghegan) Add the ability to retrieve all pg_stat_statements information except the query text (Peter Geoghegan) This allows monitoring tools to fetch query text only for just-created entries, improving performance during repeated querying of the statistics. Make pg_stat_statements ignore DEALLOCATE commands (Fabien Coelho) It already ignored PREPARE, as well as planning time in general, so this seems more consistent. Save the statistics file into $PGDATA/pg_stat at server shutdown, rather than $PGDATA/global (Fujii Masao)