aboutsummaryrefslogtreecommitdiff
path: root/src/backend/access
Commit message (Collapse)AuthorAge
...
* Improve scalability of WAL insertions.Heikki Linnakangas2013-07-08
| | | | | | | | | | | | | | | | | | | | | | This patch replaces WALInsertLock with a number of WAL insertion slots, allowing multiple backends to insert WAL records to the WAL buffers concurrently. This is particularly useful for parallel loading large amounts of data on a system with many CPUs. This has one user-visible change: switching to a new WAL segment with pg_switch_xlog() now fills the remaining unused portion of the segment with zeros. This potentially adds some overhead, but it has been a very common practice by DBA's to clear the "tail" of the segment with an external pg_clearxlogtail utility anyway, to make the WAL files compress better. With this patch, it's no longer necessary to do that. This patch adds a new GUC, xloginsert_slots, to tune the number of WAL insertion slots. Performance testing suggests that the default, 8, works pretty well for all kinds of worklods, but I left the GUC in place to allow others with different hardware to test that easily. We might want to remove that before release. Reviewed by Andres Freund.
* Handle posix_fallocate() errors.Jeff Davis2013-07-06
| | | | | | | | | On some platforms, posix_fallocate() is available but may still return EINVAL if the underlying filesystem does not support it. So, in case of an error, fall through to the alternate implementation that just writes zeros. Per buildfarm failure and analysis by Tom Lane.
* Update messages, comments and documentation for materialized views.Noah Misch2013-07-05
| | | | | All instances of the verbiage lagging the code. Back-patch to 9.3, where materialized views were introduced.
* Use posix_fallocate() for new WAL files, where available.Jeff Davis2013-07-05
| | | | | | | | This function is more efficient than actually writing out zeroes to the new file, per microbenchmarks by Jon Nelson. Also, it may reduce the likelihood of WAL file fragmentation. Jon Nelson, with review by Andres Freund, Greg Smith and me.
* Fix typo in comment.Fujii Masao2013-07-05
| | | | Michael Paquier
* Add new GUC, max_worker_processes, limiting number of bgworkers.Robert Haas2013-07-04
| | | | | | | | | | | | | | | | | | | | | | | | In 9.3, there's no particular limit on the number of bgworkers; instead, we just count up the number that are actually registered, and use that to set MaxBackends. However, that approach causes problems for Hot Standby, which needs both MaxBackends and the size of the lock table to be the same on the standby as on the master, yet it may not be desirable to run the same bgworkers in both places. 9.3 handles that by failing to notice the problem, which will probably work fine in nearly all cases anyway, but is not theoretically sound. A further problem with simply counting the number of registered workers is that new workers can't be registered without a postmaster restart. This is inconvenient for administrators, since bouncing the postmaster causes an interruption of service. Moreover, there are a number of applications for background processes where, by necessity, the background process must be started on the fly (e.g. parallel query). While this patch doesn't actually make it possible to register new background workers after startup time, it's a necessary prerequisite. Patch by me. Review by Michael Paquier.
* Get rid of pg_class.reltoastidxid.Fujii Masao2013-07-04
| | | | | | | | | | Treat TOAST index just the same as normal one and get the OID of TOAST index from pg_index but not pg_class.reltoastidxid. This change allows us to handle multiple TOAST indexes, and which is required infrastructure for upcoming REINDEX CONCURRENTLY feature. Patch by Michael Paquier, reviewed by Andres Freund and me.
* Add support for multiple kinds of external toast datums.Robert Haas2013-07-02
| | | | | | | | | | | To that end, support tags rather than lengths for external datums. As an example of how this can be used, add support or "indirect" tuples which point to some externally allocated memory containing a toast tuple. Similar infrastructure could be used for other purposes, including, perhaps, support for alternative compression algorithms. Andres Freund, reviewed by Hitoshi Harada and myself
* Use an MVCC snapshot, rather than SnapshotNow, for catalog scans.Robert Haas2013-07-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | SnapshotNow scans have the undesirable property that, in the face of concurrent updates, the scan can fail to see either the old or the new versions of the row. In many cases, we work around this by requiring DDL operations to hold AccessExclusiveLock on the object being modified; in some cases, the existing locking is inadequate and random failures occur as a result. This commit doesn't change anything related to locking, but will hopefully pave the way to allowing lock strength reductions in the future. The major issue has held us back from making this change in the past is that taking an MVCC snapshot is significantly more expensive than using a static special snapshot such as SnapshotNow. However, testing of various worst-case scenarios reveals that this problem is not severe except under fairly extreme workloads. To mitigate those problems, we avoid retaking the MVCC snapshot for each new scan; instead, we take a new snapshot only when invalidation messages have been processed. The catcache machinery already requires that invalidation messages be sent before releasing the related heavyweight lock; else other backends might rely on locally-cached data rather than scanning the catalog at all. Thus, making snapshot reuse dependent on the same guarantees shouldn't break anything that wasn't already subtly broken. Patch by me. Review by Michael Paquier and Andres Freund.
* Retry short writes when flushing WAL.Heikki Linnakangas2013-07-01
| | | | | | | | | | | | | We don't normally bother retrying when the number of bytes written by write() is short of what was requested. It is generally assumed that a write() to disk doesn't return short, unless you run out of disk space. While writing the WAL, however, it seems prudent to try a bit harder, because a failure leads to PANIC. The write() is also much larger than most write()s in the backend (up to wal_buffers), so there's more room for surprises. Also retry on EINTR. All signals used in the backend are flagged SA_RESTART nowadays, so it shouldn't happen, but better to be defensive.
* Inline ginCompareItemPointers function for speed.Heikki Linnakangas2013-06-29
| | | | | ginCompareItemPointers function is called heavily in gin index scans - inlining it speeds up some kind of queries a lot.
* Cooperate with the Valgrind instrumentation framework.Noah Misch2013-06-26
| | | | | | | | | | | | | | | | Valgrind "client requests" in aset.c and mcxt.c teach Valgrind and its Memcheck tool about the PostgreSQL allocator. This makes Valgrind roughly as sensitive to memory errors involving palloc chunks as it is to memory errors involving malloc chunks. Further client requests in PageAddItem() and printtup() verify that all bits being added to a buffer page or furnished to an output function are predictably-defined. Those tests catch failures of C-language functions to fully initialize the bits of a Datum, which in turn stymie optimizations that rely on _equalConst(). Define the USE_VALGRIND symbol in pg_config_manual.h to enable these additions. An included "suppression file" silences nominal errors we don't plan to fix. Reviewed in earlier versions by Peter Geoghegan and Korry Douglas.
* Initialize pad bytes in GinFormTuple().Noah Misch2013-06-26
| | | | | | | Every other core buffer page consumer initializes the bytes it furnishes to PageAddItem(). For consistency, do the same here. No back-patch; regardless, we couldn't count on the fix so long as binary upgrade can carry forward affected index builds.
* Avoid inconsistent type declarationAlvaro Herrera2013-06-25
| | | | | | | | | Clang 3.3 correctly complains that a variable of type enum MultiXactStatus cannot hold a value of -1, which makes sense. Change the declared type of the variable to int instead, and apply casting as necessary to avoid the warning. Per notice from Andres Freund
* Ensure no xid gaps during Hot Standby startupSimon Riggs2013-06-23
| | | | | | | | | In some cases with higher numbers of subtransactions it was possible for us to incorrectly initialize subtrans leading to complaints of missing pages. Bug report by Sergey Konoplev Analysis and fix by Andres Freund
* Clarify terminology standalone backend vs. single-user modePeter Eisentraut2013-06-20
| | | | | | | | Most of the documentation uses "single-user mode", so use that in the code as well. Adjust the documentation to match the new error message wording. Also add a documentation index entry for "single-user mode". Based-on-patch-by: Jeff Janes <jeff.janes@gmail.com>
* Add buffer_std flag to MarkBufferDirtyHint().Jeff Davis2013-06-17
| | | | | | | | | | MarkBufferDirtyHint() writes WAL, and should know if it's got a standard buffer or not. Currently, the only callers where buffer_std is false are related to the FSM. In passing, rename XLOG_HINT to XLOG_FPI, which is more descriptive. Back-patch to 9.3.
* Avoid deadlocks during insertion into SP-GiST indexes.Tom Lane2013-06-14
| | | | | | | | | | | | | SP-GiST's original scheme for avoiding deadlocks during concurrent index insertions doesn't work, as per report from Hailong Li, and there isn't any evident way to make it work completely. We could possibly lock individual inner tuples instead of their whole pages, but preliminary experimentation suggests that the performance penalty would be huge. Instead, if we fail to get a buffer lock while descending the tree, just restart the tree descent altogether. We keep the old tuple positioning rules, though, in hopes of reducing the number of cases where this can happen. Teodor Sigaev, somewhat edited by Tom Lane
* Remove special-case treatment of LOG severity level in standalone mode.Tom Lane2013-06-13
| | | | | | | | | | | | | elog.c has historically treated LOG messages as low-priority during bootstrap and standalone operation. This has led to confusion and even masked a bug, because the normal expectation of code authors is that elog(LOG) will put something into the postmaster log, and that wasn't happening during initdb. So get rid of the special-case rule and make the priority order the same as it is in normal operation. To keep from cluttering initdb's output and the behavior of a standalone backend, tweak the severity level of three messages routinely issued by xlog.c during startup and shutdown so that they won't appear in these cases. Per my proposal back in December.
* Observe array length in HaveVirtualXIDsDelayingChkpt().Noah Misch2013-06-12
| | | | | | | | Since commit f21bb9cfb5646e1793dcc9c0ea697bab99afa523, this function ignores the caller-provided length and loops until it finds a terminator, which GetVirtualXIDsDelayingChkpt() never adds. Restore the previous loop control logic. In passing, revert the addition of an unused variable by the same commit, presumably a debugging relic.
* Fix typo in comment.Heikki Linnakangas2013-06-06
|
* Additional spelling correctionsStephen Frost2013-06-03
| | | | | | A few more minor spelling corrections, no functional changes. Thom Brown
* Code review of recycling WAL segments in a restartpoint.Heikki Linnakangas2013-06-03
| | | | | | | | Seems cleaner to get the currently-replayed TLI in the same call to GetXLogReplayRecPtr that we get the WAL position. Make it more clear in the comment what the code does when recovery has already ended (RecoveryInProgress() will set ThisTimeLineID in that case). Finally, make resetting ThisTimeLineID afterwards more explicit.
* Minor spelling fixesStephen Frost2013-06-01
| | | | | | Fix a few spelling mistakes. Per bug report #8193 from Lajos Veres.
* Post-pgindent cleanupStephen Frost2013-06-01
| | | | | | | | | | Make slightly better decisions about indentation than what pgindent is capable of. Mostly breaking out long function calls into one line per argument, with a few other minor adjustments. No functional changes- all whitespace. pgindent ran cleanly (didn't change anything) after. Passes all regressions.
* pgindent run for release 9.3Bruce Momjian2013-05-29
| | | | | This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
* After fast promotion use CHECKPOINT_FORCESimon Riggs2013-05-21
| | | | | | | Not necessary for correctness, just to make log_checkpoints output look less singular. Requested by Fujii Masao
* Maintain ThisTimeLineID correctly in checkpointerSimon Riggs2013-05-21
| | | | | | | | | | | | checkpointer needs to reset ThisTimeLineID after a restartpoint to allow installing/recycling new WAL files. If recovery has already ended this would leave ThisTimeLineID set incorrectly and so we must reset it otherwise later checkpoints do not have the correct timeline. Bug report by Heikki Linnakangas. Further investigation by Heikki and myself.
* Init crash recovery using the latest available TLISimon Riggs2013-05-19
| | | | | | | | This simplifies the handling of crashes after fast promotion and various minor cases that can exist in short timing windows around that case. Broad fix to bug reported by Michael Paquier on -hackers, approach prompted by Heikki Linnakangas
* Emit msg correctly for timeline-crossing crashSimon Riggs2013-05-19
|
* Remove single space on end of a line in xlog.cSimon Riggs2013-05-19
| | | | Michael Paquier
* Fix handling of OID wraparound while in standalone mode.Tom Lane2013-05-13
| | | | | | | | | | | If OID wraparound should occur while in standalone mode (unlikely but possible), we want to advance the counter to FirstNormalObjectId not FirstBootstrapObjectId. Otherwise, user objects might be created with OIDs in the system-reserved range. That isn't immediately harmful but it poses a risk of conflicts during future pg_upgrade operations. Noted by Andres Freund. Back-patch to all supported branches, since all of them are supported sources for pg_upgrade operations.
* Fix management of fn_extra caching during repeated GiST index scans.Tom Lane2013-05-09
| | | | | | | | | | | | | | Commit d22a09dc70f9830fa78c1cd1a3a453e4e473d354 introduced official support for GiST consistentFns that want to cache data using the FmgrInfo fn_extra pointer: the idea was to preserve the cached values across gistrescan(), whereas formerly they'd been leaked. However, there was an oversight in that, namely that multiple scan keys might reference the same column's consistentFn; the code would result in propagating the same cache value into multiple scan keys, resulting in crashes or wrong answers. Use a separate array instead to ensure that each scan key keeps its own state. Per bug #8143 from Joel Roller. Back-patch to 9.2 where the bug was introduced.
* Fix walsender failure at promotion.Heikki Linnakangas2013-05-08
| | | | | | | | | | | | | | | | | | | | | | | | If a standby server has a cascading standby server connected to it, it's possible that WAL has already been sent up to the next WAL page boundary, splitting a WAL record in the middle, when the first standby server is promoted. Don't throw an assertion failure or error in walsender if that happens. Also, fix a variant of the same bug in pg_receivexlog: if it had already received WAL on previous timeline up to a segment boundary, when the upstream standby server is promoted so that the timeline switch record falls on the previous segment, pg_receivexlog would miss the segment containing the timeline switch. To fix that, have walsender send the position of the timeline switch at end-of-streaming, in addition to the next timeline's ID. It was previously assumed that the switch happened exactly where the streaming stopped. Note: this is an incompatible change in the streaming protocol. You might get an error if you try to stream over timeline switches, if the client is running 9.3beta1 and the server is more recent. It should be fine after a reconnect, however. Reported by Fujii Masao.
* Use the term "radix tree" instead of "suffix tree" for SP-GiST text opclass.Heikki Linnakangas2013-05-08
| | | | | | | What we have implemented is a radix tree (or a radix trie or a patricia trie), but the docs and code comments incorrectly called it a "suffix tree". Alexander Korotkov
* Record data_checksum_version in control file.Simon Riggs2013-04-30
| | | | | | The value is not used anywhere in code, but will allow future changes to the checksum version should that become necessary in the future.
* Make fast promotion the default promotion mode.Simon Riggs2013-04-24
| | | | | Continue to allow a request for synchronous checkpoints as a mechanism in case of problems.
* Remove some unused and seldom used fields from RelationAmInfo.Heikki Linnakangas2013-04-16
| | | | | | | This saves some memory from each index relcache entry. At least on a 64-bit machine, it saves just enough to shrink a typical relcache entry's memory usage from 2k to 1k. That's nice if you have a lot of backends and a lot of indexes.
* Remove duplicate initialization in XLogReadRecord.Robert Haas2013-04-09
| | | | Per a note from Dickson S. Guedes.
* Fix calculation of how many segments to retain for wal_keep_segments.Heikki Linnakangas2013-04-08
| | | | | | | KeepLogSeg function was broken when we switched to use a 64-bit int for the segment number. Per report from Jeff Janes.
* Skip extraneous locking in XLogCheckBuffer().Simon Riggs2013-04-08
| | | | | | | Heikki reported comment was wrong, so fixed code to match the comment: we only need to take additional locking precautions when we have a shared lock on the buffer.
* Avoid tricky race condition recording XLOG_HINTSimon Riggs2013-04-08
| | | | | | | | | | | | | | | | | | We copy the buffer before inserting an XLOG_HINT to avoid WAL CRC errors caused by concurrent hint writes to buffer while share locked. To make this work we refactor RestoreBackupBlock() to allow an XLOG_HINT to avoid the normal path for backup blocks, which assumes the underlying buffer is exclusive locked. Resulting code completely changes layout of XLOG_HINT WAL records, but this isn't even beta code, so this is a low impact change. In passing, avoid taking WALInsertLock for full page writes on checksummed hints, remove related cruft from XLogInsert() and improve xlog_desc record for XLOG_HINT. Andres Freund Bug report by Fujii Masao, testing by Jeff Janes and Jaime Casanova, review by Jeff Davis and Simon Riggs. Applied with changes from review and some comment editing.
* Fix checksums for CLUSTER, VACUUM FULL etc.Simon Riggs2013-04-07
| | | | | | | | | In CLUSTER, VACUUM FULL and ALTER TABLE SET TABLESPACE I erroneously set checksum before log_newpage, which sets the LSN and invalidates the checksum. So set checksum immediately *after* log_newpage. Bug report Fujii Masao, Fix and patch by Jeff Davis
* Make REPLICATION privilege checks test current user not authenticated user.Tom Lane2013-04-01
| | | | | | | | | | | The pg_start_backup() and pg_stop_backup() functions checked the privileges of the initially-authenticated user rather than the current user, which is wrong. For example, a user-defined index function could successfully call these functions when executed by ANALYZE within autovacuum. This could allow an attacker with valid but low-privilege database access to interfere with creation of routine backups. Reported and fixed by Noah Misch. Security: CVE-2013-1901
* Revoke bc5334d8679c428a709d150666b288171795bd76Simon Riggs2013-03-28
|
* Fix buffer pin leak in heap update redo routine.Heikki Linnakangas2013-03-27
| | | | | | | | | | | | | | In a heap update, if the old and new tuple were on different pages, and the new page no longer existed (because it was subsequently truncated away by vacuum), heap_xlog_update forgot to release the pin on the old buffer. This bug was introduced by the "Fix multiple problems in WAL replay" patch, commit 3bbf668de9f1bc172371681e80a4e769b6d014c8 (on master branch). With full_page_writes=off, this triggered an "incorrect local pin count" error later in replay, if the old page was vacuumed. This fixes bug #7969, reported by Yunong Xiao. Backpatch to 9.0, like the commit that introduced this bug.
* Allow external recovery_config_directorySimon Riggs2013-03-27
| | | | | If required, recovery.conf can now be located outside of the data directory. Server needs read/write permissions on this directory.
* Fix grammatical errors in some new message strings.Tom Lane2013-03-26
| | | | Daniele Varrazzo
* Allow I/O reliability checks using 16-bit checksumsSimon Riggs2013-03-22
| | | | | | | | | | | | | | | | | | | Checksums are set immediately prior to flush out of shared buffers and checked when pages are read in again. Hint bit setting will require full page write when block is dirtied, which causes various infrastructure changes. Extensive comments, docs and README. WARNING message thrown if checksum fails on non-all zeroes page; ERROR thrown but can be disabled with ignore_checksum_failure = on. Feature enabled by an initdb option, since transition from option off to option on is long and complex and has not yet been implemented. Default is not to use checksums. Checksum used is WAL CRC-32 truncated to 16-bits. Simon Riggs, Jeff Davis, Greg Smith Wide input and assistance from many community members. Thank you.
* Remove PageSetTLI and rename pd_tli to pd_checksumSimon Riggs2013-03-18
| | | | | | | | | | | | | | Remove use of PageSetTLI() from all page manipulation functions and adjust README to indicate change in the way we make changes to pages. Repurpose those bytes into the pd_checksum field and explain how that works in comments about page header. Refactoring ahead of actual feature patch which would make use of the checksum field, arriving later. Jeff Davis, with comments and doc changes by Simon Riggs Direction suggested by Robert Haas; many others providing review comments.
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-07 10:02:46 +0000