Change pg_ctl to detect server-ready by watching status in postmaster.pid.

Traditionally, "pg_ctl start -w" has waited for the server to become
ready to accept connections by attempting a connection once per second.
That has the major problem that connection issues (for instance, a
kernel packet filter blocking traffic) can't be reliably told apart
from server startup issues, and the minor problem that if server startup
isn't quick, we accumulate "the database system is starting up" spam
in the server log.  We've hacked around many of the possible connection
issues, but it resulted in ugly and complicated code in pg_ctl.c.

In commit c61559ec3, I changed the probe rate to every tenth of a second.
That prompted Jeff Janes to complain that the log-spam problem had become
much worse.  In the ensuing discussion, Andres Freund pointed out that
we could dispense with connection attempts altogether if the postmaster
were changed to report its status in postmaster.pid, which "pg_ctl start"
already relies on being able to read.  This patch implements that, teaching
postmaster.c to report a status string into the pidfile at the same
state-change points already identified as being of interest for systemd
status reporting (cf commit 7d17e683f).  pg_ctl no longer needs to link
with libpq at all; all its functions now depend on reading server files.

In support of this, teach AddToDataDirLockFile() to allow addition of
postmaster.pid lines in not-necessarily-sequential order.  This is needed
on Windows where the SHMEM_KEY line will never be written at all.  We still
have the restriction that we don't want to truncate the pidfile; document
the reasons for that a bit better.

Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode
is broken --- before, they'd just wait out the sixty seconds until
the loop gives up, and then report success anyway.  (Yes, I found that
out the hard way.)

While at it, arrange for pg_ctl to not need to #include miscadmin.h;
as a rather low-level backend header, requiring that to be compilable
client-side is pretty dubious.  This requires moving the #define's
associated with the pidfile into a new header file, and moving
PG_BACKEND_VERSIONSTR someplace else.  For lack of a clearly better
"someplace else", I put it into port.h, beside the declaration of
find_other_exec(), since most users of that macro are passing the value to
find_other_exec().  (initdb still depends on miscadmin.h, but at least
pg_ctl and pg_upgrade no longer do.)

In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the
output of "postgres -V", which remarkably it had never done before.

Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com

1 files changed, 55 insertions, 0 deletions

diff --git a/src/include/utils/pidfile.h b/src/include/utils/pidfile.h
new file mode 100644
index 00000000000..c3db4c46e39
--- /dev/null
+++ b/src/include/utils/pidfile.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pidfile.h
+ *	  Declarations describing the data directory lock file (postmaster.pid)
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/utils/pidfile.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef UTILS_PIDFILE_H
+#define UTILS_PIDFILE_H
+
+/*
+ * As of Postgres 10, the contents of the data-directory lock file are:
+ *
+ * line #
+ *		1	postmaster PID (or negative of a standalone backend's PID)
+ *		2	data directory path
+ *		3	postmaster start timestamp (time_t representation)
+ *		4	port number
+ *		5	first Unix socket directory path (empty if none)
+ *		6	first listen_address (IP address or "*"; empty if no TCP port)
+ *		7	shared memory key (empty on Windows)
+ *		8	postmaster status (see values below)
+ *
+ * Lines 6 and up are added via AddToDataDirLockFile() after initial file
+ * creation; also, line 5 is initially empty and is changed after the first
+ * Unix socket is opened.
+ *
+ * Socket lock file(s), if used, have the same contents as lines 1-5, with
+ * line 5 being their own directory.
+ */
+#define LOCK_FILE_LINE_PID			1
+#define LOCK_FILE_LINE_DATA_DIR		2
+#define LOCK_FILE_LINE_START_TIME	3
+#define LOCK_FILE_LINE_PORT			4
+#define LOCK_FILE_LINE_SOCKET_DIR	5
+#define LOCK_FILE_LINE_LISTEN_ADDR	6
+#define LOCK_FILE_LINE_SHMEM_KEY	7
+#define LOCK_FILE_LINE_PM_STATUS	8
+
+/*
+ * The PM_STATUS line may contain one of these values.  All these strings
+ * must be the same length, per comments for AddToDataDirLockFile().
+ * We pad with spaces as needed to make that true.
+ */
+#define PM_STATUS_STARTING		"starting"	/* still starting up */
+#define PM_STATUS_STOPPING		"stopping"	/* in shutdown sequence */
+#define PM_STATUS_READY			"ready   "	/* ready for connections */
+#define PM_STATUS_STANDBY		"standby "	/* up, won't accept connections */
+
+#endif							/* UTILS_PIDFILE_H */