aboutsummaryrefslogtreecommitdiff
path: root/src/bin/pg_rewind
Commit message (Collapse)AuthorAge
* Add some const decorations to prototypesPeter Eisentraut2017-11-10
| | | | Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
* Remove wal_keep_segments from default configuration in PostgresNode.pmPeter Eisentraut2017-11-02
| | | | | | | This is only used in the pg_rewind tests, so only set it there. It's better if other tests run closer to a default configuration. Author: Michael Paquier <michael.paquier@gmail.com>
* Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h.Andres Freund2017-10-01
| | | | | | | | | | | | | | | | | All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
* Make WAL segment size configurable at initdb time.Andres Freund2017-09-19
| | | | | | | | | | | | | | | | | | | | | | | For performance reasons a larger segment size than the default 16MB can be useful. A larger segment size has two main benefits: Firstly, in setups using archiving, it makes it easier to write scripts that can keep up with higher amounts of WAL, secondly, the WAL has to be written and synced to disk less frequently. But at the same time large segment size are disadvantageous for smaller databases. So far the segment size had to be configured at compile time, often making it unrealistic to choose one fitting to a particularly load. Therefore change it to a initdb time setting. This includes a breaking changes to the xlogreader.h API, which now requires the current segment size to be configured. For that and similar reasons a number of binaries had to be taught how to recognize the current segment size. Author: Beena Emerson, editorialized by Andres Freund Reviewed-By: Andres Freund, David Steele, Kuntal Ghosh, Michael Paquier, Peter Eisentraut, Robert Hass, Tushar Ahuja Discussion: https://postgr.es/m/CAOG9ApEAcQ--1ieKbhFzXSQPw_YLmepaa4hNdnY5+ZULpt81Mw@mail.gmail.com
* Support retaining data dirs on successful TAP testsPeter Eisentraut2017-09-05
| | | | | | | | | | | | | This moves the data directories from using temporary directories with randomness in the directory name to a static name, to make it easier to debug. The data directory will be retained if tests fail or the test code dies/exits with failure, and is automatically removed on the next make check. If the environment variable PG_TEST_NOCLEAN is defined, the data directories will be retained regardless of test or exit status. Author: Daniel Gustafsson <daniel@yesql.se>
* Fix translatable stringAlvaro Herrera2017-09-04
| | | | Discussion: https://postgr.es/m/20170828130545.sdajqlpr37hmmd6a@alvherre.pgsql
* Final pgindent + perltidy run for v10.Tom Lane2017-08-14
|
* Remove uses of "slave" in replication contextsPeter Eisentraut2017-08-10
| | | | | This affects mostly code comments, some documentation, and tests. Official APIs already used "standby".
* Translation updatesPeter Eisentraut2017-08-07
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 1a0b5e655d7871506c2b1c7ba562c2de6b6a55de
* pg_rewind: Fix some problems when copying files >2GB.Robert Haas2017-07-21
| | | | | | | | | | | | | | | | | | | | | | When incrementally updating a file larger than 2GB, the old code could either fail outright (if the client asked the server for bytes beyond the 2GB boundary) or fail to copy all the blocks that had actually been modified (if the server reported a file size to the client in excess of 2GB), resulting in data corruption. Generally, such files won't occur anyway, but they might if using a non-default segment size or if there the directory contains stray files unrelated to PostgreSQL. Fix by a more prudent choice of data types. Even with these improvements, this code still uses a mix of different types (off_t, size_t, uint64, int64) to represent file sizes and offsets, not all of which necessarily have the same width or signedness, so further cleanup might be in order here. However, at least now they all have the potential to be 64 bits wide on 64-bit platforms. Kuntal Ghosh and Michael Paquier, with a tweak by me. Discussion: http://postgr.es/m/CAGz5QC+8gbkz=Brp0TgoKNqHWTzonbPtPex80U0O6Uh_bevbaA@mail.gmail.com
* pg_rewind: Fix busted sanity check.Robert Haas2017-07-21
| | | | | | | | As written, the code would only fail the sanity check if none of the columns returned by the server were of the expected type, but we want it to fail if even one column is not of the expected type. Discussion: http://postgr.es/m/CA+TgmoYuY5zW7JEs+1hSS1D=V5K8h1SQuESrq=bMNeo0B71Sfw@mail.gmail.com
* Add static assertions about pg_control fitting into one disk sector.Tom Lane2017-07-19
| | | | | | | | | | | | | | | | | When pg_control was first designed, sizeof(ControlFileData) was small enough that a comment seemed like plenty to document the assumption that it'd fit into one disk sector. Now it's nearly 300 bytes, raising the possibility that somebody would carelessly add enough stuff to create a problem. Let's add a StaticAssertStmt() to ensure that the situation doesn't pass unnoticed if it ever occurs. While at it, rename PG_CONTROL_SIZE to PG_CONTROL_FILE_SIZE to make it clearer what that symbol means, and convert the existing runtime comparisons of sizeof(ControlFileData) vs. PG_CONTROL_FILE_SIZE to be static asserts --- we didn't have that technology when this code was first written. Discussion: https://postgr.es/m/9192.1500490591@sss.pgh.pa.us
* Fix broken link-command-line ordering for libpgfeutils.Tom Lane2017-07-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the frontend Makefiles that pull in libpgfeutils, we'd generally done it like this: LDFLAGS += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) That method is badly broken, as seen in bug #14742 from Chris Ruprecht. The -L flag for src/fe_utils ends up being placed after whatever random -L flags are in LDFLAGS already. That puts us at risk of pulling in libpgfeutils.a from some previous installation rather than the freshly built one in src/fe_utils. Also, the lack of an "override" is hazardous if someone tries to specify some LDFLAGS on the make command line. The correct way to do it is like this: override LDFLAGS := -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) $(LDFLAGS) so that libpgfeutils, along with libpq, libpgport, and libpgcommon, are guaranteed to be pulled in from the build tree and not from any referenced system directory, because their -L flags will appear first. In some places we'd been even lazier and done it like this: LDFLAGS += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpq which is subtly wrong in an additional way: on platforms where we can't restrict the symbols exported by libpq.so, it allows libpgfeutils to latch onto libpgport and libpgcommon symbols from libpq.so, rather than directly from those static libraries as intended. This carries hazards like those explained in the comments for the libpq_pgport macro. In addition to fixing the broken libpgfeutils usages, I tried to standardize on using $(libpq_pgport) like so: override LDFLAGS := $(libpq_pgport) $(LDFLAGS) even where libpgfeutils is not in the picture. This makes no difference right now but will hopefully discourage future mistakes of the same ilk. And it's more like the way we handle CPPFLAGS in libpq-using Makefiles. In passing, just for consistency, make pgbench include PTHREAD_LIBS the same way everyplace else does, ie just after LIBS rather than in some random place in the command line. This might have practical effect if there are -L switches in that macro on some platform. It looks to me like the MSVC build scripts are not affected by this error, but someone more familiar with them than I might want to double check. Back-patch to 9.6 where libpgfeutils was introduced. In 9.6, the hazard this error creates is that a reinstallation might link to the prior installation's copy of libpgfeutils.a and thereby fail to absorb a minor-version bug fix. Discussion: https://postgr.es/m/20170714125106.9231.13772@wrigleys.postgresql.org
* Translation updatesAlvaro Herrera2017-07-10
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: c5a8de3653bb1af6b0eb41cc6bf090c5522df52b
* Phase 3 of pgindent updates.Tom Lane2017-06-21
| | | | | | | | | | | | | | | | | | | | | | | | | Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
* Phase 2 of pgindent updates.Tom Lane2017-06-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
* Translation updatesPeter Eisentraut2017-05-15
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 398beeef4921df0956f917becd7b5669d2a8a5c4
* Standardize "WAL location" terminologyPeter Eisentraut2017-05-12
| | | | Other previously used terms were "WAL position" or "log position".
* Replace "transaction log" with "write-ahead log"Peter Eisentraut2017-05-12
| | | | | This makes documentation and error messages match the renaming of "xlog" to "wal" in APIs and file naming.
* Rename WAL-related functions and views to use "lsn" not "location".Tom Lane2017-05-11
| | | | | | | | | | | | | | | Per discussion, "location" is a rather vague term that could refer to multiple concepts. "LSN" is an unambiguous term for WAL locations and should be preferred. Some function names, view column names, and function output argument names used "lsn" already, but others used "location", as well as yet other terms such as "wal_position". Since we've already renamed a lot of things in this area from "xlog" to "wal" for v10, we may as well incur a bit more compatibility pain and make these names all consistent. David Rowley, minor additional docs hacking by me Discussion: https://postgr.es/m/CAKJS1f8O0njDKe8ePFQ-LK5-EjwThsDws6ohJ-+c6nWK+oUxtg@mail.gmail.com
* Fix new warnings from GCC 7Peter Eisentraut2017-04-17
| | | | | This addresses the new warning types -Wformat-truncation -Wformat-overflow that are part of -Wall, via -Wformat, in GCC 7.
* Remove useless trailing spaces in queries in C stringsPeter Eisentraut2017-04-13
| | | | Author: Alexander Law <exclusion@gmail.com>
* Fix inclusions of c.h from .h files.Tom Lane2017-03-08
| | | | | | | | | | We have a project policy that every .c file should start by including postgres.h, postgres_fe.h, or c.h as appropriate; and then there is no need for any .h file to explicitly include any of these. Fix a few headers that were violating this policy by including c.h. Discussion: https://postgr.es/m/CAEepm=2zCoeq3QxVwhS5DFeUh=yU6z81pbWMgfOB8OzyiBwxzw@mail.gmail.com Discussion: https://postgr.es/m/11634.1488932128@sss.pgh.pa.us
* Remove useless duplicate inclusions of system header files.Tom Lane2017-02-25
| | | | | | | | | | | | | | | | c.h #includes a number of core libc header files, such as <stdio.h>. There's no point in re-including these after having read postgres.h, postgres_fe.h, or c.h; so remove code that did so. While at it, also fix some places that were ignoring our standard pattern of "include postgres[_fe].h, then system header files, then other Postgres header files". While there's not any great magic in doing it that way rather than system headers last, it's silly to have just a few files deviating from the general pattern. (But I didn't attempt to enforce this globally, only in files I was touching anyway.) I'd be the first to say that this is mostly compulsive neatnik-ism, but over time it might save enough compile cycles to be useful.
* Remove all references to "xlog" from SQL-callable functions in pg_proc.Robert Haas2017-02-09
| | | | | | | | | | | | | | Commit f82ec32ac30ae7e3ec7c84067192535b2ff8ec0e renamed the pg_xlog directory to pg_wal. To make things consistent, and because "xlog" is terrible terminology for either "transaction log" or "write-ahead log" rename all SQL-callable functions that contain "xlog" in the name to instead contain "wal". (Note that this may pose an upgrade hazard for some users.) Similarly, rename the xlog_position argument of the functions that create slots to be called wal_position. Discussion: https://www.postgresql.org/message-id/CA+Tgmob=YmA=H3DbW1YuOXnFVgBheRmyDkWcD9M8f=5bGWYEoQ@mail.gmail.com
* Add WAL consistency checking facility.Robert Haas2017-02-08
| | | | | | | | | | | | | | When the new GUC wal_consistency_checking is set to a non-empty value, it triggers recording of additional full-page images, which are compared on the standby against the results of applying the WAL record (without regard to those full-page images). Allowable differences such as hints are masked out, and the resulting pages are compared; any difference results in a FATAL error on the standby. Kuntal Ghosh, based on earlier patches by Michael Paquier and Heikki Linnakangas. Extensively reviewed and revised by Michael Paquier and by me, with additional reviews and comments from Amit Kapila, Álvaro Herrera, Simon Riggs, and Peter Eisentraut.
* Update copyright via script for 2017Bruce Momjian2017-01-03
|
* pg_rewing pg_upgrade: Fix translation markersPeter Eisentraut2016-11-07
| | | | | In pg_log_v(), we need to translate the fmt before processing, not the formatted message afterwards.
* Rename "pg_xlog" directory to "pg_wal".Robert Haas2016-10-20
| | | | | | | | | | | | | | | | | | | | | "xlog" is not a particularly clear abbreviation for "write-ahead log", and it sometimes confuses users into believe that the contents of the "pg_xlog" directory are not critical data, leading to unpleasant consequences. So, rename the directory to "pg_wal". This patch modifies pg_upgrade and pg_basebackup to understand both the old and new directory layouts; the former is necessary given the purpose of the tool, while the latter merely avoids an unnecessary backward-compatibility break. We may wish to consider renaming other programs, switches, and functions which still use the old "xlog" naming to also refer to "wal". However, that's still under discussion, so let's do just this much for now. Discussion: CAB7nPqTeC-8+zux8_-4ZD46V7YPwooeFxgndfsq5Rg8ibLVm1A@mail.gmail.com Michael Paquier
* Use pg_ctl promote -w in TAP testsPeter Eisentraut2016-10-19
| | | | | | | | Switch TAP tests to use the new wait mode of pg_ctl promote. This allows avoiding extra logic with poll_query_until() to be sure that a promoted standby is ready for read-write queries. From: Michael Paquier <michael.paquier@gmail.com>
* Don't allow both --source-server and --source-target args to pg_rewind.Heikki Linnakangas2016-10-07
| | | | | | | | | They are supposed to be mutually exclusive, but there was no check for that. Michael Banck Discussion: <20161007103414.GD12247@nighthawk.caipicrew.dd-dns.de>
* Disable synchronous commits in pg_rewind.Heikki Linnakangas2016-10-06
| | | | | | | | | | | | | | | | | | If you point pg_rewind to a server that is using synchronous replication, with "pg_rewind --source-server=...", and the replication is not working for some reason, pg_rewind will get stuck because it creates a temporary table, which needs to be replicated. You could call broken replication a pilot error, but pg_rewind is often used in special circumstances, when there are changes to the replication setup. We don't do any "real" updates, and we don't care about fsyncing or replicating the operations on the temporary tables, so fix that by setting synchronous_commit off. Michael Banck, Michael Paquier. Backpatch to 9.5, where pg_rewind was introduced. Discussion: <20161005143938.GA12247@nighthawk.caipicrew.dd-dns.de>
* Add tests for various connection string issuesPeter Eisentraut2016-09-22
| | | | | | | | | | | | | | Add tests for consistent support of connection strings in frontend programs as well as proper handling of unusual characters in database and user names. These tests were developed for the issues of CVE-2016-5424. To allow testing of names with spaces, change the pg_regress command-line options --create-role and --dbname to split their arguments by comma only, not space or comma as before. Only commas were actually used in existing uses. Noah Misch, Michael Paquier, Peter Eisentraut
* Translation updatesPeter Eisentraut2016-08-08
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: cda21c1d7b160b303dc21dfe9d4169f2c8064c60
* Translation updatesPeter Eisentraut2016-07-18
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 3d71988dffd3c0798a8864c55ca4b7833b48abb1
* Translation updatesPeter Eisentraut2016-06-20
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 0c374f8d25ed31833a10d24252bc928d41438838
* Finish pgindent run for 9.6: Perl files.Noah Misch2016-06-12
|
* pgindent run for 9.6Robert Haas2016-06-09
|
* Message style and wording fixesPeter Eisentraut2016-06-07
|
* Translation updatesPeter Eisentraut2016-05-09
| | | | | Source-Git-URL: git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 17bf3e8564abf600274789fcc90e72532d5e7c05
* pg_rewind: Improve internationalizationAlvaro Herrera2016-03-28
| | | | | | | This is mostly cosmetic since two of the three changes are debug messages, and the third one is just a progress indicator. Author: Michaël Paquier
* pg_rewind: fsync target data directory.Andres Freund2016-03-27
| | | | | | | | | | | | | | | | | Previously pg_rewind did not fsync any files. That's problematic, given that the target directory is modified. If the database was started afterwards, 2ce439f33 luckily already caused the data directory to be synced to disk at postmaster startup; reducing the scope of the problem. To fix, use initdb -S, at the end of the pg_rewind run. It doesn't seem worthwhile to duplicate the code into pg_rewind, and initdb -S is already used that way by pg_upgrade. Reported-By: Andres Freund Author: Michael Paquier, somewhat edited by me Discussion: 20160310034352.iuqgvpmg5qmnxtkz@alap3.anarazel.de CAB7nPqSytVG1o4S3S2pA1O=692ekurJ+fckW2PywEG3sNw54Ow@mail.gmail.com Backpatch: 9.5, where pg_rewind was introduced
* pg_rewind: Close backup_label file descriptor.Andres Freund2016-03-27
| | | | | | | | | | This was a relatively harmless leak, as createBackupLabel() is only called once per pg_rewind invocation. Author: Michael Paquier Reported-By: Michael Paquier Discussion: CAB7nPqRnOw30gOXe2_SPLjh37bgm4V+txbYAPwoXb97nGQ297w@mail.gmail.com Backpatch: 9.5, where pg_rewind was introduced
* Move some code from RewindTest into PostgresNodeAlvaro Herrera2016-02-26
| | | | | | | | | | | | | | Some code in the RewindTest test suite is more generally useful than just for that suite, so put it where other test suites can reach it. Some postgresql.conf parameters change their default values when a cluster is initialized with 'allows_streaming' than the previous behavior; most notably, autovacuum is no longer turned off. (Also, we no longer call pg_ctl promote with -w, but that flag doesn't actually do anything in promote so there's no behavior change.) Author: Michael Paquier
* PostgresNode: Add names to nodesAlvaro Herrera2016-01-20
| | | | | | | | This makes the log files easier to follow when investigating a test failure. Author: Michael Paquier Review: Noah Misch
* Fix typo in comment.Robert Haas2016-01-04
| | | | Masahiko Sawada
* Update copyright for 2016Bruce Momjian2016-01-02
| | | | Backpatch certain files through 9.1
* Remove unnecessary row ordering dependency in pg_rewind test suite.Tom Lane2015-12-24
| | | | | | | t/002_databases.pl was expecting to see a specific physical order of the rows in pg_database. I broke that in HEAD with commit 01e386a325549b77, but I'd say it's a pretty fragile test methodology in any case, so fix it in 9.5 as well.
* Remove unreferenced function declarations.Tom Lane2015-12-17
| | | | | | datapagemap_create() and datapagemap_destroy() were declared extern, but they don't actually exist anywhere. Per YUriy Zhuravlev and Michael Paquier.
* Add missing cleanup logic in pg_rewind/t/005_same_timeline.pl test.Tom Lane2015-12-14
| | | | Per Michael Paquier
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-02 04:26:43 +0000