path: root/doc/src/sgml/custom-scan.sgml

<!-- doc/src/sgml/custom-scan.sgml -->

<chapter id="custom-scan">
 <title>Writing A Custom Scan Provider</title>

 <indexterm zone="custom-scan">
  <primary>custom scan provider</primary>
  <secondary>handler for</secondary>
 </indexterm>

 <para>
  <productname>PostgreSQL</> supports a set of experimental facilities which
  are intended to allow extension modules to add new scan types to the system.
  Unlike a <link linkend="fdwhandler">foreign data wrapper</>, which is only
  responsible for knowing how to scan its own foreign tables, a custom scan
  provider can provide an alternative method of scanning any relation in the
  system.  Typically, the motivation for writing a custom scan provider will
  be to allow the use of some optimization not supported by the core
  system, such as caching or some form of hardware acceleration.  This chapter
  outlines how to write a new custom scan provider.
 </para>

 <para>
  Implementing a new type of custom scan is a three-step process.  First,
  during planning, it is necessary to generate access paths representing a
  scan using the proposed strategy.  Second, if one of those access paths
  is selected by the planner as the optimal strategy for scanning a
  particular relation, the access path must be converted to a plan.
  Finally, it must be possible to execute the plan and generate the same
  results that would have been generated for any other access path targeting
  the same relation.
 </para>

 <sect1 id="custom-scan-path">
  <title>Implementing Custom Paths</title>

  <para>
    A custom scan provider will typically add paths by setting the following
    hook, which is called after the core code has generated what it believes
    to be the complete and correct set of access paths for the relation.
<programlisting>
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
                                            RelOptInfo *rel,
                                            Index rti,
                                            RangeTblEntry *rte);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
</programlisting>
  </para>

  <para>
    Although this hook function can be used to examine, modify, or remove
    paths generated by the core system, a custom scan provider will typically
    confine itself to generating <structname>CustomPath</> objects and adding
    them to <literal>rel</> using <function>add_path</>.  The custom scan
    provider is responsible for initializing the <structname>CustomPath</>
    object, which is declared like this:
<programlisting>
typedef struct CustomPath
{
    Path      path;
    uint32    flags;
    List     *custom_private;
    const CustomPathMethods *methods;
} CustomPath;
</programlisting>
  </para>

  <para>
    <structfield>path</> must be initialized as for any other path, including
    the row-count estimate, start and total cost, and sort ordering provided
    by this path.  <structfield>flags</> is a bitmask, which should include
    <literal>CUSTOMPATH_SUPPORT_BACKWARD_SCAN</> if the custom path can support
    a backward scan and <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> if it
    can support mark and restore.  Both capabilities are optional.
    <structfield>custom_private</> can be used to store the custom path's
    private data.  Private data should be stored in a form that can be handled
    by <literal>nodeToString</>, so that debugging routines which attempt to
    print the custom path will work as designed.  <structfield>methods</> must
    point to a (usually statically allocated) object implementing the required
    custom path methods, of which there are currently only two, as further
    detailed below.
  </para>

  <para>
   A custom scan provider can also add join paths; in this case, the scan
   must produce the same output as would normally be produced by the join
   it replaces.  To do this, the join provider should set the following hook.
   This hook may be invoked repeatedly for the same pair of relations, with
   different combinations of inner and outer relations; it is the
   responsibility of the hook to minimize duplicated work.
<programlisting>
typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
                                             RelOptInfo *joinrel,
                                             RelOptInfo *outerrel,
                                             RelOptInfo *innerrel,
                                             List *restrictlist,
                                             JoinType jointype,
                                             SpecialJoinInfo *sjinfo,
                                             SemiAntiJoinFactors *semifactors,
                                             Relids param_source_rels,
                                             Relids extra_lateral_rels);
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
</programlisting>
  </para>

  <sect2 id="custom-scan-path-callbacks">
  <title>Custom Path Callbacks</title>

  <para>
<programlisting>
Plan *(*PlanCustomPath) (PlannerInfo *root,
                         RelOptInfo *rel,
                         CustomPath *best_path,
                         List *tlist,
                         List *clauses);
</programlisting>
    Convert a custom path to a finished plan.  The return value will generally
    be a <literal>CustomScan</> object, which the callback must allocate and
    initialize.  See <xref linkend="custom-scan-plan"> for more details.
   </para>

   <para>
<programlisting>
void (*TextOutCustomPath) (StringInfo str,
                           const CustomPath *node);
</programlisting>
    Generate additional output when <function>nodeToString</> is invoked on
    this custom path.  This callback is optional. Since
    <function>nodeToString</> will automatically dump all fields in the
    structure that it can see, including <structfield>custom_private</>, this
    is only useful if the <structname>CustomPath</> is actually embedded in a
    larger struct containing additional fields.
   </para>
  </sect2>
 </sect1>

 <sect1 id="custom-scan-plan">
  <title>Implementing Custom Plans</title>

  <para>
    A custom scan is represented in a finished plan tree using the following
    structure:
<programlisting>
typedef struct CustomScan
{
    Scan      scan;
    uint32    flags;
    List     *custom_exprs;
    List     *custom_ps_tlist;
    List     *custom_private;
    List     *custom_relids;
    const CustomScanMethods *methods;
} CustomScan;
</programlisting>
  </para>

  <para>
    <structfield>scan</> must be initialized as for any other scan, including
    estimated costs, target lists, qualifications, and so on.
    <structfield>flags</> is a bitmask with the same meaning as in
    <structname>CustomPath</>.  <structfield>custom_exprs</> should be used to
    store expression trees that will need to be fixed up by
    <filename>setrefs.c</> and <filename>subselect.c</>, while
    <literal>custom_private</> should be used to store other private data that
    is only used by the custom scan provider itself.  Plan trees must be able
    to be duplicated using <function>copyObject</>, so all the data stored
    within these two fields must consist of nodes that function can handle.
    <literal>custom_relids</> is set by the core code to the set of relations
    which this scan node must handle; except when this scan is replacing a
    join, it will have only one member.
    <structfield>methods</> must point to a (usually statically allocated)
    object implementing the required custom scan methods, which are further
    detailed below.
  </para>

  <para>
   When a <structname>CustomScan</> scans a single relation,
   <structfield>scan.scanrelid</> should be the range table index of the table
   to be scanned, and <structfield>custom_ps_tlist</> should be
   <literal>NULL</>.  When it replaces a join, <structfield>scan.scanrelid</>
   should be zero, and <structfield>custom_ps_tlist</> should be a list of
   <structname>TargetEntry</> nodes.  This is necessary because, when a join
   is replaced, the target list cannot be constructed from the table
   definition.  At execution time, this list will be used to initialize the
   tuple descriptor of the <structname>TupleTableSlot</>.  It will also be
   used by <command>EXPLAIN</>, when deparsing.
  </para>

  <sect2 id="custom-scan-plan-callbacks">
   <title>Custom Scan Callbacks</title>
   <para>
<programlisting>
Node *(*CreateCustomScanState) (CustomScan *cscan);
</programlisting>
    Allocate a <structname>CustomScanState</> for this
    <structname>CustomScan</>.  The actual allocation will often be larger than
    required for an ordinary <structname>CustomScanState</>, because many
    scan types will wish to embed that as the first field of a large structure.
    The value returned must have the node tag and <structfield>methods</>
    set appropriately, but the other fields need not be initialized at this
    stage; after <function>ExecInitCustomScan</> performs basic initialization,
    the <function>BeginCustomScan</> callback will be invoked to give the
    custom scan state a chance to do whatever else is needed.
   </para>

   <para>
<programlisting>
void (*TextOutCustomScan) (StringInfo str,
                           const CustomScan *node);
</programlisting>
    Generate additional output when <function>nodeToString</> is invoked on
    this custom plan.  This callback is optional.  Since a
    <structname>CustomScan</> must be copyable by <function>copyObject</>,
    custom scan providers cannot substitute a larger structure that embeds a
    <structname>CustomScan</> for the structure itself, as would be possible
    for a <structname>CustomPath</> or <structname>CustomScanState</>.
    Therefore, providing this callback is unlikely to be useful.
   </para>
  </sect2>
 </sect1>

 <sect1 id="custom-scan-scan">
  <title>Implementing Custom Scans</title>

  <para>
   When a <structfield>CustomScan</> is executed, its execution state is
   represented by a <structfield>CustomScanState</>, which is declared as
   follows.
<programlisting>
typedef struct CustomScanState
{
    ScanState ss;
    uint32    flags;
    const CustomExecMethods *methods;
} CustomScanState;
</programlisting>
  </para>

  <para>
   <structfield>ss</> must be initialized as for any other scanstate;
   <structfield>flags</> is a bitmask with the same meaning as in
   <structname>CustomPath</> and <structname>CustomScan</>.
   <structfield>methods</> must point to a (usually statically allocated)
   object implementing the required custom scan state methods, which are
   further detailed below.  Typically, a <structname>CustomScanState</>, which
   need not support <function>copyObject</>, will actually be a larger
   structure embedding the above as its first member.
  </para>

  <sect2 id="custom-scan-scan-callbacks">
   <title>Custom Execution-Time Callbacks</title>

   <para>
<programlisting>
void (*BeginCustomScan) (CustomScanState *node,
                         EState *estate,
                         int eflags);
</programlisting>
    Complete initialization of the supplied <structname>CustomScanState</>.
    Some initialization is performed by <function>ExecInitCustomScan</>, but
    any private fields should be initialized here.
   </para>

   <para>
<programlisting>
TupleTableSlot *(*ExecCustomScan) (CustomScanState *node);
</programlisting>
    Fetch the next scan tuple.  If any tuples remain, it should fill
    <literal>ps_ResultTupleSlot</> with the next tuple in the current scan
    direction, and then return the tuple slot.  If not,
    <literal>NULL</> or an empty slot should be returned.
   </para>

   <para>
<programlisting>
void (*EndCustomScan) (CustomScanState *node);
</programlisting>
    Clean up any private data associated with the <literal>CustomScanState</>.
    This method is required, but may not need to do anything if the associated
    data does not exist or will be cleaned up automatically.
   </para>

   <para>
<programlisting>
void (*ReScanCustomScan) (CustomScanState *node);
</programlisting>
    Rewind the current scan to the beginning and prepare to rescan the
    relation.
   </para>

   <para>
<programlisting>
void (*MarkPosCustomScan) (CustomScanState *node);
</programlisting>
    Save the current scan position so that it can subsequently be restored
    by the <function>RestrPosCustomScan</> callback.  This calback is optional,
    and need only be supplied if 
    <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set.
   </para>

   <para>
<programlisting>
void (*RestrPosCustomScan) (CustomScanState *node);
</programlisting>
    Restore the previous scan position as saved by the
    <function>MarkPosCustomScan</> callback.  This callback is optional,
    and need only be supplied if 
    <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set.
   </para>

   <para>
<programlisting>
void (*ExplainCustomScan) (CustomScanState *node,
                           List *ancestors,
                           ExplainState *es);
</programlisting>
    Output additional information on <command>EXPLAIN</> that involves
    custom-scan node.  This callback is optional.  Common data stored in the
    <structname>ScanState</>, such as the target list and scan relation, will
    be shown even without this callback, but the callback allows the display
    of additional, private state.
   </para>
  </sect2>
 </sect1>
</chapter>