aboutsummaryrefslogtreecommitdiff
path: root/src/backend/port/win32_shmem.c
Commit message (Collapse)AuthorAge
* Consistently test for in-use shared memory.Noah Misch2019-04-12
| | | | | | | | | | | | | | | | | | | | | | postmaster startup scrutinizes any shared memory segment recorded in postmaster.pid, exiting if that segment matches the current data directory and has an attached process. When the postmaster.pid file was missing, a starting postmaster used weaker checks. Change to use the same checks in both scenarios. This increases the chance of a startup failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1 postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster will no longer stop if shmat() of an old segment fails with EACCES. A postmaster will no longer recycle segments pertaining to other data directories. That's good for production, but it's bad for integration tests that crash a postmaster and immediately delete its data directory. Such a test now leaks a segment indefinitely. No "make check-world" test does that. win32_shmem.c already avoided all these problems. In 9.6 and later, enhance PostgresNode to facilitate testing. Back-patch to 9.4 (all supported versions). Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI. Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
* Avoid "could not reattach" by providing space for concurrent allocation.Noah Misch2019-04-08
| | | | | | | | | | | | | | | | | We've long had reports of intermittent "could not reattach to shared memory" errors on Windows. Buildfarm member dory fails that way when PGSharedMemoryReAttach() execution overlaps with creation of a thread for the process's "default thread pool". Fix that by providing a second region to receive asynchronous allocations that would otherwise intrude into UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free reservations landing at incorrect addresses; the caller's next step has been to terminate the affected process. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. He also did much of the prerequisite research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion: https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com
* Revert "Consistently test for in-use shared memory."Noah Misch2019-04-05
| | | | | | | | | This reverts commits 2f932f71d9f2963bbd201129d7b971c8f5f077fd, 16ee6eaf80a40007a138b60bb5661660058d0422 and 6f0e190056fe441f7cf788ff19b62b13c94f68f3. The buildfarm has revealed several bugs. Back-patch like the original commits. Discussion: https://postgr.es/m/20190404145319.GA1720877@rfd.leadboat.com
* Consistently test for in-use shared memory.Noah Misch2019-04-03
| | | | | | | | | | | | | | | | | | | | | postmaster startup scrutinizes any shared memory segment recorded in postmaster.pid, exiting if that segment matches the current data directory and has an attached process. When the postmaster.pid file was missing, a starting postmaster used weaker checks. Change to use the same checks in both scenarios. This increases the chance of a startup failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1 postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster will no longer recycle segments pertaining to other data directories. That's good for production, but it's bad for integration tests that crash a postmaster and immediately delete its data directory. Such a test now leaks a segment indefinitely. No "make check-world" test does that. win32_shmem.c already avoided all these problems. In 9.6 and later, enhance PostgresNode to facilitate testing. Back-patch to 9.4 (all supported versions). Reviewed by Daniel Gustafsson and Kyotaro HORIGUCHI. Discussion: https://postgr.es/m/20130911033341.GD225735@tornado.leadboat.com
* Update copyright for 2019Bruce Momjian2019-01-02
| | | | Backpatch-through: certain files through 9.4
* Remove investigative code for can't-reattach-to-shared-memory errors.Tom Lane2018-05-01
| | | | | | | | | Revert commits 23078689a, 73042b8d1, ce07aff48, f7df8043f, 6ba0cc4bd, eb16011f4, 68e7e973d, 63ca350ef. We still have a problem here, but somebody who's actually a Windows developer will need to spend time on it. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Does it help to wait before reattaching?Tom Lane2018-04-30
| | | | | | | | | | | | | | | | Revert the map/unmap dance I tried in commit 73042b8d1; that helps not at all. Instead, speculate that the unwanted allocation is being done on another thread, and thus timing variations explain the apparent unpredictability. Temporarily add a 1-second sleep before the VirtualFree call, in hopes that any such other threads will quiesce and not jog our elbow. This is obviously not a desirable long-term fix, but as a means of investigation it seems useful. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Map and unmap the shared memory block before risking VirtualFree.Tom Lane2018-04-30
| | | | | | | | | | | | | | The idea here is to get Windows' userspace infrastructure to allocate whatever space it needs for MapViewOfFileEx() before we release the locked-down space that we want to map the shared memory block into. This is a fairly brute-force attempt, and would likely (for example) fail with large shared memory on 32-bit Windows. We could perhaps ameliorate that by mapping only part of the shared memory block in this way, but for the moment I just want to see if this approach will fix dory's problem. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Further effort at preventing memory map dump from affecting the results.Tom Lane2018-04-30
| | | | | | | | Rather than elog'ing immediately, push the map data into a preallocated StringInfo. Perhaps this will prevent some of the mid-operation allocations that are evidently happening now. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Remove Windows module-list-dumping code.Tom Lane2018-04-30
| | | | | | | | This code is evidently allocating memory and thus confusing matters even more. Let's see whether we can learn anything with just VirtualQuery. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Dump full memory maps around failing Windows reattach code.Tom Lane2018-04-30
| | | | | | | | | | | | | | This morning's results from buildfarm member dory make it pretty clear that something is getting mapped into the just-freed space, but not what that something is. Replace my minimalistic probes with a full dump of the process address space and module space, based on Noah's work at <20170403065106.GA2624300%40tornado.leadboat.com> This is all (probably) to get reverted once we have fixed the problem, but for now we need information. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Get still more info about Windows can't-reattach-to-shared-memory errors.Tom Lane2018-04-29
| | | | | | | | | | | | After some thought about the info captured so far, it seems possible that MapViewOfFileEx is itself causing some DLL to get loaded into the space just freed by VirtualFree. The previous commit here didn't capture enough info to really prove the case for that, so let's add one more VirtualQuery in between those steps. Also, be sure to capture the post-Map state before we emit any log entries, just in case elog() is invoking some code not previously loaded. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Get more info about Windows can't-reattach-to-shared-memory errors.Tom Lane2018-04-29
| | | | | | | Commit 63ca350ef neglected to probe the state of things *before* the VirtualFree call, which now looks like it might be interesting. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Try to get some info about Windows can't-reattach-to-shared-memory errors.Tom Lane2018-04-27
| | | | | | | | | | | | Add some debug printouts focused on the idea that MapViewOfFileEx might be rounding its virtual memory allocation up more than we expect (and, in particular, more than VirtualAllocEx does). Once we've seen what this reports in one of the failures on buildfarm members dory or jacana, we might revert this ... or perhaps just decrease the log level. Discussion: https://postgr.es/m/25495.1524517820@sss.pgh.pa.us
* Post-feature-freeze pgindent run.Tom Lane2018-04-26
| | | | Discussion: https://postgr.es/m/15719.1523984266@sss.pgh.pa.us
* Support huge pages on WindowsMagnus Hagander2018-01-21
| | | | | | | | | | | Add support for huge pages (called large pages on Windows) to the Windows build. This (probably) breaks compatibility with Windows versions prior to Windows 2003 or Windows Vista. Authors: Takayuki Tsunakawa and Thomas Munro Reviewed by: Magnus Hagander, Amit Kapila
* Update copyright for 2018Bruce Momjian2018-01-02
| | | | Backpatch-through: certain files through 9.3
* Phase 2 of pgindent updates.Tom Lane2017-06-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
* Update copyright via script for 2017Bruce Momjian2017-01-03
|
* Update copyright for 2016Bruce Momjian2016-01-02
| | | | Backpatch certain files through 9.1
* On Windows, ensure shared memory handle gets closed if not being used.Tom Lane2015-10-13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Postmaster child processes that aren't supposed to be attached to shared memory were not bothering to close the shared memory mapping handle they inherit from the postmaster process. That's mostly harmless, since the handle vanishes anyway when the child process exits -- but the syslogger process, if used, doesn't get killed and restarted during recovery from a backend crash. That meant that Windows doesn't see the shared memory mapping as becoming free, so it doesn't delete it and the postmaster is unable to create a new one, resulting in failure to recover from crashes whenever logging_collector is turned on. Per report from Dmitry Vasilyev. It's a bit astonishing that we'd not figured this out long ago, since it's been broken from the very beginnings of out native Windows support; probably some previously-unexplained trouble reports trace to this. A secondary problem is that on Cygwin (perhaps only in older versions?), exec() may not detach from the shared memory segment after all, in which case these child processes did remain attached to shared memory, posing the risk of an unexpected shared memory clobber if they went off the rails somehow. That may be a long-gone bug, but we can deal with it now if it's still live, by detaching within the infrastructure introduced here to deal with closing the handle. Back-patch to all supported branches. Tom Lane and Amit Kapila
* Update copyright for 2015Bruce Momjian2015-01-06
| | | | Backpatch certain files through 9.0
* pgindent run for 9.4Bruce Momjian2014-05-06
| | | | | This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
* Add missing include.Robert Haas2014-04-09
| | | | | | This is more cleanup from commit 11a65eed1637a05b03e174700799b024e104bfb4. Amit Kapila
* Fix silly oversight in patch to remove dsm state file.Robert Haas2014-04-08
| | | | | I'm not sure if this is what's causing the Windows buildfarm members to get unhappy, but I don't think it can be helping anything...
* Get rid of the dynamic shared memory state file.Robert Haas2014-04-08
| | | | | | | | | | | | | Instead of storing the ID of the dynamic shared memory control segment in a file within the data directory, store it in the main control segment. This avoids a number of nasty corner cases, most seriously that doing an online backup and then using it on the same machine (e.g. to fire up a standby) would result in the standby clobbering all of the master's dynamic shared memory segments. Per complaints from Heikki Linnakangas, Fujii Masao, and Tom Lane.
* Rename huge_tlb_pages to huge_pages, and improve docs.Heikki Linnakangas2014-03-03
| | | | Christian Kruse
* Allow using huge TLB pages on Linux (MAP_HUGETLB)Heikki Linnakangas2014-01-29
| | | | | | | | | | | | | | | | | | | | | | This patch adds an option, huge_tlb_pages, which allows requesting the shared memory segment to be allocated using huge pages, by using the MAP_HUGETLB flag in mmap(). This can improve performance. The default is 'try', which means that we will attempt using huge pages, and fall back to non-huge pages if it doesn't work. Currently, only Linux has MAP_HUGETLB. On other platforms, the default 'try' behaves the same as 'off'. In the passing, don't try to round the mmap() size to a multiple of pagesize. mmap() doesn't require that, and there's no particular reason for PostgreSQL to do that either. When using MAP_HUGETLB, however, round the request size up to nearest 2MB boundary. This is to work around a bug in some Linux kernel versions, but also to avoid wasting memory, because the kernel will round the size up anyway. Many people were involved in writing this patch, including Christian Kruse, Richard Poole, Abhijit Menon-Sen, reviewed by Peter Geoghegan, Andres Freund and me.
* Allow use of "z" flag in our printf calls, and use it where appropriate.Tom Lane2014-01-23
| | | | | | | | | | | | | | | | | | | | | | | | | Since C99, it's been standard for printf and friends to accept a "z" size modifier, meaning "whatever size size_t has". Up to now we've generally dealt with printing size_t values by explicitly casting them to unsigned long and using the "l" modifier; but this is really the wrong thing on platforms where pointers are wider than longs (such as Win64). So let's start using "z" instead. To ensure we can do that on all platforms, teach src/port/snprintf.c to understand "z", and add a configure test to force use of that implementation when the platform's version doesn't handle "z". Having done that, modify a bunch of places that were using the unsigned-long hack to use "z" instead. This patch doesn't pretend to have gotten everyplace that could benefit, but it catches many of them. I made an effort in particular to ensure that all uses of the same error message text were updated together, so as not to increase the number of translatable strings. It's possible that this change will result in format-string warnings from pre-C99 compilers. We might have to reconsider if there are any popular compilers that will warn about this; but let's start by seeing what the buildfarm thinks. Andres Freund, with a little additional work by me
* Update copyright for 2014Bruce Momjian2014-01-07
| | | | | Update all files in head, and files COPYRIGHT and legal.sgml in all back branches.
* Update copyrights for 2013Bruce Momjian2013-01-01
| | | | | Fully update git head, and update back branches in ./COPYRIGHT and legal.sgml files.
* Update copyright notices for year 2012.Bruce Momjian2012-01-01
|
* Use consistent format for reporting GetLastError()Peter Eisentraut2011-08-23
| | | | | | Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
* Stamp copyrights for year 2011.Bruce Momjian2011-01-01
|
* Remove cvs keywords from all files.Magnus Hagander2010-09-20
|
* pgindent run for 9.0Bruce Momjian2010-02-26
|
* Silence compiler warning about printf format for HANDLE.Itagaki Takahiro2010-01-04
|
* Update copyright for the year 2010.Bruce Momjian2010-01-02
|
* Support 64-bit shared memory when building on 64-bit Windows.Magnus Hagander2010-01-02
| | | | Tsutomu Yamada
* Reserve the shared memory region during backend startup on Windows, soMagnus Hagander2009-07-24
| | | | | | | | | | that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
* 8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef listBruce Momjian2009-06-11
| | | | provided by Andrew.
* Minor improvement: avoid assuming that GetLastError value cannot beTom Lane2009-05-05
| | | | affected by CloseHandle() or Sleep().
* Make the win32 shared memory code try 10 times instead of one ifMagnus Hagander2009-05-05
| | | | | | | it fails because the shared memory segment already exists. This means it can take up to 10 seconds before it reports the error if it *does* exist, but hopefully it will make the system capable of restarting even when the server is under high load.
* Call SetLastError(0) before calling the file mapping functionsMagnus Hagander2009-05-04
| | | | | | | to make sure that the error code is reset, as a precaution in case the API doesn't properly reset it on success. This could be necessary, since we check the error value even if the function doesn't fail for specific success cases.
* Update copyright for 2009.Bruce Momjian2009-01-01
|
* Revert previous patch to put the shared memory segment on win32Magnus Hagander2008-10-30
| | | | | | | | in the Global\ namespace, because it caused permission errors on a lot of platforms. We need to come up with something better for 8.4, but for now revert to the pre-8.3.4 behaviour.
* Fix a couple of bugs in win32 shmem name generation:Magnus Hagander2008-07-04
| | | | | * Don't cut off the prefix. With this fix, it's again readable. * Properly store it in the Global namespace as intended.
* Update copyrights in source tree to 2008.Bruce Momjian2008-01-01
|
* Add sentence-ending periods.Peter Eisentraut2007-11-08
|
* Fix compiler warnings in ereport messages on mingw.Magnus Hagander2007-03-28
| | | | ITAGAKI Takahiro
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-05 03:16:11 +0000