From 7949d9594582ab49dee221e1db1aa5401ace49d4 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Sun, 4 Aug 2024 19:41:24 +0900 Subject: Introduce pluggable APIs for Cumulative Statistics This commit adds support in the backend for $subject, allowing out-of-core extensions to plug their own custom kinds of cumulative statistics. This feature has come up a few times into the lists, and the first, original, suggestion came from Andres Freund, about pg_stat_statements to use the cumulative statistics APIs in shared memory rather than its own less efficient internals. The advantage of this implementation is that this can be extended to any kind of statistics. The stats kinds are divided into two parts: - The in-core "builtin" stats kinds, with designated initializers, able to use IDs up to 128. - The "custom" stats kinds, able to use a range of IDs from 128 to 256 (128 slots available as of this patch), with information saved in TopMemoryContext. This can be made larger, if necessary. There are two types of cumulative statistics in the backend: - For fixed-numbered objects (like WAL, archiver, etc.). These are attached to the snapshot and pgstats shmem control structures for efficiency, and built-in stats kinds still do that to avoid any redirection penalty. The data of custom kinds is stored in a first array in snapshot structure and a second array in the shmem control structure, both indexed by their ID, acting as an equivalent of the builtin stats. - For variable-numbered objects (like tables, functions, etc.). These are stored in a dshash using the stats kind ID in the hash lookup key. Internally, the handling of the builtin stats is unchanged, and both fixed and variabled-numbered objects are supported. Structure definitions for builtin stats kinds are renamed to reflect better the differences with custom kinds. Like custom RMGRs, custom cumulative statistics can only be loaded with shared_preload_libraries at startup, and must allocate a unique ID shared across all the PostgreSQL extension ecosystem with the following wiki page to avoid conflicts: https://wiki.postgresql.org/wiki/CustomCumulativeStats This makes the detection of the stats kinds and their handling when reading and writing stats much easier than, say, allocating IDs for stats kinds from a shared memory counter, that may change the ID used by a stats kind across restarts. When under development, extensions can use PGSTAT_KIND_EXPERIMENTAL. Two examples that can be used as templates for fixed-numbered and variable-numbered stats kinds will be added in some follow-up commits, with tests to provide coverage. Some documentation is added to explain how to use this plugin facility. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m/Zmqm9j5EO0I4W8dx@paquier.xyz --- doc/src/sgml/xfunc.sgml | 57 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) (limited to 'doc/src') diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index bf76490cbc0..6b029a5a35f 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3864,6 +3864,63 @@ extern bool InjectionPointDetach(const char *name); + + Custom Cumulative Statistics + + + Is is possible for add-ins written in C-language to use custom types + of cumulative statistics registered in the + Cumulative Statistics System. + + + + First, define a PgStat_KindInfo that includes all + the information related to the custom type registered. For example: + +static const PgStat_KindInfo custom_stats = { + .name = "custom_stats", + .fixed_amount = false, + .shared_size = sizeof(PgStatShared_Custom), + .shared_data_off = offsetof(PgStatShared_Custom, stats), + .shared_data_len = sizeof(((PgStatShared_Custom *) 0)->stats), + .pending_size = sizeof(PgStat_StatCustomEntry), +} + + + Then, each backend that needs to use this custom type needs to register + it with pgstat_register_kind and a unique ID used to + store the entries related to this type of statistics: + +extern PgStat_Kind pgstat_add_kind(PgStat_Kind kind, + const PgStat_KindInfo *kind_info); + + While developing a new extension, use + PGSTAT_KIND_EXPERIMENTAL for + kind. When you are ready to release the extension + to users, reserve a kind ID at the + + Custom Cumulative Statistics page. + + + + The details of the API for PgStat_KindInfo can + be found in src/include/utils/pgstat_internal.h. + + + + The type of statistics registered is associated with a name and a unique + ID shared across the server in shared memory. Each backend using a + custom type of statistics maintains a local cache storing the information + of each custom PgStat_KindInfo. + + + + Place the extension module implementing the custom cumulative statistics + type in so that it will + be loaded early during PostgreSQL startup. + + + Using C++ for Extensibility -- cgit v1.2.3