Debian Codegraph @

src/backend/utils/fmgr/README

raw
1 src/backend/utils/fmgr/README
2
3 Function Manager
4 ================
5
6 [This file originally explained the transition from the V0 to the V1
7 interface. Now it just explains some internals and rationale for the V1
8 interface, while the V0 interface has been removed.]
9
10 The V1 Function-Manager Interface
11 ---------------------------------
12
13 The core of the design is data structures for representing the result of a
14 function lookup and for representing the parameters passed to a specific
15 function invocation. (We want to keep function lookup separate from
16 function call, since many parts of the system apply the same function over
17 and over; the lookup overhead should be paid once per query, not once per
18 tuple.)
19
20
21 When a function is looked up in pg_proc, the result is represented as
22
23 typedef struct
24 {
25 PGFunction fn_addr; /* pointer to function or handler to be called */
26 Oid fn_oid; /* OID of function (NOT of handler, if any) */
27 short fn_nargs; /* number of input args (0..FUNC_MAX_ARGS) */
28 bool fn_strict; /* function is "strict" (NULL in => NULL out) */
29 bool fn_retset; /* function returns a set (over multiple calls) */
30 unsigned char fn_stats; /* collect stats if track_functions > this */
31 void *fn_extra; /* extra space for use by handler */
32 MemoryContext fn_mcxt; /* memory context to store fn_extra in */
33 Node *fn_expr; /* expression parse tree for call, or NULL */
34 } FmgrInfo;
35
36 For an ordinary built-in function, fn_addr is just the address of the C
37 routine that implements the function. Otherwise it is the address of a
38 handler for the class of functions that includes the target function.
39 The handler can use the function OID and perhaps also the fn_extra slot
40 to find the specific code to execute. (fn_oid = InvalidOid can be used
41 to denote a not-yet-initialized FmgrInfo struct. fn_extra will always
42 be NULL when an FmgrInfo is first filled by the function lookup code, but
43 a function handler could set it to avoid making repeated lookups of its
44 own when the same FmgrInfo is used repeatedly during a query.) fn_nargs
45 is the number of arguments expected by the function, fn_strict is its
46 strictness flag, and fn_retset shows whether it returns a set; all of
47 these values come from the function's pg_proc entry. fn_stats is also
48 set up to control whether or not to track runtime statistics for calling
49 this function.
50
51 If the function is being called as part of a SQL expression, fn_expr will
52 point to the expression parse tree for the function call; this can be used
53 to extract parse-time knowledge about the actual arguments. Note that this
54 field really is information about the arguments rather than information
55 about the function, but it's proven to be more convenient to keep it in
56 FmgrInfo than in FunctionCallInfoBaseData where it might more logically go.
57
58
59 During a call of a function, the following data structure is created
60 and passed to the function:
61
62 typedef struct
63 {
64 FmgrInfo *flinfo; /* ptr to lookup info used for this call */
65 Node *context; /* pass info about context of call */
66 Node *resultinfo; /* pass or return extra info about result */
67 Oid fncollation; /* collation for function to use */
68 bool isnull; /* function must set true if result is NULL */
69 short nargs; /* # arguments actually passed */
70 NullableDatum args[]; /* Arguments passed to function */
71 } FunctionCallInfoBaseData;
72 typedef FunctionCallInfoBaseData* FunctionCallInfo;
73
74 flinfo points to the lookup info used to make the call. Ordinary functions
75 will probably ignore this field, but function class handlers will need it
76 to find out the OID of the specific function being called.
77
78 context is NULL for an "ordinary" function call, but may point to additional
79 info when the function is called in certain contexts. (For example, the
80 trigger manager will pass information about the current trigger event here.)
81 Further details appear in "Function Call Contexts" below.
82
83 resultinfo is NULL when calling any function from which a simple Datum
84 result is expected. It may point to some subtype of Node if the function
85 returns more than a Datum. (For example, resultinfo is used when calling a
86 function that returns a set, as discussed below.) Like the context field,
87 resultinfo is a hook for expansion; fmgr itself doesn't constrain the use
88 of the field.
89
90 fncollation is the input collation derived by the parser, or InvalidOid
91 when there are no inputs of collatable types or they don't share a common
92 collation. This is effectively a hidden additional argument, which
93 collation-sensitive functions can use to determine their behavior.
94
95 nargs and args[] hold the arguments being passed to the function.
96 Notice that all the arguments passed to a function (as well as its result
97 value) will now uniformly be of type Datum. As discussed below, callers
98 and callees should apply the standard Datum-to-and-from-whatever macros
99 to convert to the actual argument types of a particular function. The
100 value in args[i].value is unspecified when args[i].isnull is true.
101
102 It is generally the responsibility of the caller to ensure that the
103 number of arguments passed matches what the callee is expecting; except
104 for callees that take a variable number of arguments, the callee will
105 typically ignore the nargs field and just grab values from args[].
106
107 The isnull field will be initialized to "false" before the call. On
108 return from the function, isnull is the null flag for the function result:
109 if it is true the function's result is NULL, regardless of the actual
110 function return value. Note that simple "strict" functions can ignore
111 both isnull and args[i].isnull, since they won't even get called when there
112 are any TRUE values in args[].isnull.
113
114 FunctionCallInfo replaces FmgrValues plus a bunch of ad-hoc parameter
115 conventions, global variables (fmgr_pl_finfo and CurrentTriggerData at
116 least), and other uglinesses.
117
118
119 Callees, whether they be individual functions or function handlers,
120 shall always have this signature:
121
122 Datum function (FunctionCallInfo fcinfo);
123
124 which is represented by the typedef
125
126 typedef Datum (*PGFunction) (FunctionCallInfo fcinfo);
127
128 The function is responsible for setting fcinfo->isnull appropriately
129 as well as returning a result represented as a Datum. Note that since
130 all callees will now have exactly the same signature, and will be called
131 through a function pointer declared with exactly that signature, we
132 should have no portability or optimization problems.
133
134
135 Function Coding Conventions
136 ---------------------------
137
138 Here are the proposed macros and coding conventions:
139
140 The definition of an fmgr-callable function will always look like
141
142 Datum
143 function_name(PG_FUNCTION_ARGS)
144 {
145 ...
146 }
147
148 "PG_FUNCTION_ARGS" just expands to "FunctionCallInfo fcinfo". The main
149 reason for using this macro is to make it easy for scripts to spot function
150 definitions. However, if we ever decide to change the calling convention
151 again, it might come in handy to have this macro in place.
152
153 A nonstrict function is responsible for checking whether each individual
154 argument is null or not, which it can do with PG_ARGISNULL(n) (which is
155 just "fcinfo->args[n].isnull"). It should avoid trying to fetch the value
156 of any argument that is null.
157
158 Both strict and nonstrict functions can return NULL, if needed, with
159 PG_RETURN_NULL();
160 which expands to
161 { fcinfo->isnull = true; return (Datum) 0; }
162
163 Argument values are ordinarily fetched using code like
164 int32 name = PG_GETARG_INT32(number);
165
166 For float4, float8, and int8, the PG_GETARG macros will hide whether the
167 types are pass-by-value or pass-by-reference. For example, if float8 is
168 pass-by-reference then PG_GETARG_FLOAT8 expands to
169 (* (float8 *) DatumGetPointer(fcinfo->args[number].value))
170 and would typically be called like this:
171 float8 arg = PG_GETARG_FLOAT8(0);
172 For what are now historical reasons, the float-related typedefs and macros
173 express the type width in bytes (4 or 8), whereas we prefer to label the
174 widths of integer types in bits.
175
176 Non-null values are returned with a PG_RETURN_XXX macro of the appropriate
177 type. For example, PG_RETURN_INT32 expands to
178 return Int32GetDatum(x)
179 PG_RETURN_FLOAT4, PG_RETURN_FLOAT8, and PG_RETURN_INT64 hide whether their
180 data types are pass-by-value or pass-by-reference, by doing a palloc if
181 needed.
182
183 fmgr.h will provide PG_GETARG and PG_RETURN macros for all the basic data
184 types. Modules or header files that define specialized SQL datatypes
185 (eg, timestamp) should define appropriate macros for those types, so that
186 functions manipulating the types can be coded in the standard style.
187
188 For non-primitive data types (particularly variable-length types) it won't
189 be very practical to hide the pass-by-reference nature of the data type,
190 so the PG_GETARG and PG_RETURN macros for those types won't do much more
191 than DatumGetPointer/PointerGetDatum plus the appropriate typecast (but see
192 TOAST discussion, below). Functions returning such types will need to
193 palloc() their result space explicitly. I recommend naming the GETARG and
194 RETURN macros for such types to end in "_P", as a reminder that they
195 produce or take a pointer. For example, PG_GETARG_TEXT_P yields "text *".
196
197 When a function needs to access fcinfo->flinfo or one of the other auxiliary
198 fields of FunctionCallInfo, it should just do it. I doubt that providing
199 syntactic-sugar macros for these cases is useful.
200
201
202 Support for TOAST-Able Data Types
203 ---------------------------------
204
205 For TOAST-able data types, the PG_GETARG macro will deliver a de-TOASTed
206 data value. There might be a few cases where the still-toasted value is
207 wanted, but the vast majority of cases want the de-toasted result, so
208 that will be the default. To get the argument value without causing
209 de-toasting, use PG_GETARG_RAW_VARLENA_P(n).
210
211 Some functions require a modifiable copy of their input values. In these
212 cases, it's silly to do an extra copy step if we copied the data anyway
213 to de-TOAST it. Therefore, each toastable datatype has an additional
214 fetch macro, for example PG_GETARG_TEXT_P_COPY(n), which delivers a
215 guaranteed-fresh copy, combining this with the detoasting step if possible.
216
217 There is also a PG_FREE_IF_COPY(ptr,n) macro, which pfree's the given
218 pointer if and only if it is different from the original value of the n'th
219 argument. This can be used to free the de-toasted value of the n'th
220 argument, if it was actually de-toasted. Currently, doing this is not
221 necessary for the majority of functions because the core backend code
222 releases temporary space periodically, so that memory leaked in function
223 execution isn't a big problem. However, as of 7.1 memory leaks in
224 functions that are called by index searches will not be cleaned up until
225 end of transaction. Therefore, functions that are listed in pg_amop or
226 pg_amproc should be careful not to leak detoasted copies, and so these
227 functions do need to use PG_FREE_IF_COPY() for toastable inputs.
228
229 A function should never try to re-TOAST its result value; it should just
230 deliver an untoasted result that's been palloc'd in the current memory
231 context. When and if the value is actually stored into a tuple, the
232 tuple toaster will decide whether toasting is needed.
233
234
235 Function Call Contexts
236 ----------------------
237
238 If a caller passes a non-NULL pointer in fcinfo->context, it should point
239 to some subtype of Node; the particular kind of context is indicated by the
240 node type field. (A callee should always check the node type, via IsA(),
241 before assuming it knows what kind of context is being passed.) fmgr
242 itself puts no other restrictions on the use of this field.
243
244 Current uses of this convention include:
245
246 * Trigger functions are passed an instance of struct TriggerData,
247 containing information about the trigger context. (The trigger function
248 does not receive any normal arguments.) See commands/trigger.h for
249 more information and macros that are commonly used by trigger functions.
250
251 * Aggregate functions (or to be precise, their transition and final
252 functions) are passed an instance of struct AggState, that is the executor
253 state node for the calling Agg plan node; or if they are called as window
254 functions, they receive an instance of struct WindowAggState. It is
255 recommended that these pointers be used only via AggCheckCallContext()
256 and sibling functions, which are declared in fmgr.h but are documented
257 only with their source code in src/backend/executor/nodeAgg.c. Typically
258 these context nodes are only of interest when the transition and final
259 functions wish to optimize execution based on knowing that they are being
260 used within an aggregate rather than as standalone SQL functions.
261
262 * True window functions receive an instance of struct WindowObject.
263 (Like trigger functions, they don't receive any normal arguments.)
264 See windowapi.h for more information.
265
266 * Procedures are passed an instance of struct CallContext, containing
267 information about the context of the CALL statement, particularly
268 whether it is within an "atomic" execution context.
269
270 * Some callers of datatype input functions (and in future perhaps
271 other classes of functions) pass an instance of ErrorSaveContext.
272 This indicates that the caller wishes to handle "soft" errors without
273 a transaction-terminating exception being thrown: instead, the callee
274 should store information about the error cause in the ErrorSaveContext
275 struct and return a dummy result value. Further details appear in
276 "Handling Soft Errors" below.
277
278
279 Handling Soft Errors
280 --------------------
281
282 Postgres' standard mechanism for reporting errors (ereport() or elog())
283 is used for all sorts of error conditions. This means that throwing
284 an exception via ereport(ERROR) requires an expensive transaction or
285 subtransaction abort and cleanup, since the exception catcher dare not
286 make many assumptions about what has gone wrong. There are situations
287 where we would rather have a lighter-weight mechanism for dealing
288 with errors that are known to be safe to recover from without a full
289 transaction cleanup. SQL-callable functions can support this need
290 using the ErrorSaveContext context mechanism.
291
292 To report a "soft" error, a SQL-callable function should call
293 errsave(fcinfo->context, ...)
294 where it would previously have done
295 ereport(ERROR, ...)
296 If the passed "context" is NULL or is not an ErrorSaveContext node,
297 then errsave behaves precisely as ereport(ERROR): the exception is
298 thrown via longjmp, so that control does not return. If "context"
299 is an ErrorSaveContext node, then the error information included in
300 errsave's subsidiary reporting calls is stored into the context node
301 and control returns from errsave normally. The function should then
302 return a dummy value to its caller. (SQL NULL is recommendable as
303 the dummy value; but anything will do, since the caller is expected
304 to ignore the function's return value once it sees that an error has
305 been reported in the ErrorSaveContext node.)
306
307 If there is nothing to do except return after calling errsave(),
308 you can save a line or two by writing
309 ereturn(fcinfo->context, dummy_value, ...)
310 to perform errsave() and then "return dummy_value".
311
312 An error reported "softly" must be safe, in the sense that there is
313 no question about our ability to continue normal processing of the
314 transaction. Error conditions that should NOT be handled this way
315 include out-of-memory, unexpected internal errors, or anything that
316 cannot easily be cleaned up after. Such cases should still be thrown
317 with ereport, as they have been in the past.
318
319 Considering datatype input functions as examples, typical "soft" error
320 conditions include input syntax errors and out-of-range values. An
321 input function typically detects such cases with simple if-tests and
322 can easily change the ensuing ereport call to an errsave or ereturn.
323 Because of this restriction, it's typically not necessary to pass
324 the ErrorSaveContext pointer down very far, as errors reported by
325 low-level functions are typically reasonable to consider internal.
326 (Another way to frame the distinction is that input functions should
327 report all invalid-input conditions softly, but internal problems are
328 hard errors.)
329
330 Because no transaction cleanup will occur, a function that is exiting
331 after errsave() returns will bear responsibility for resource cleanup.
332 It is not necessary to be concerned about small leakages of palloc'd
333 memory, since the caller should be running the function in a short-lived
334 memory context. However, resources such as locks, open files, or buffer
335 pins must be closed out cleanly, as they would be in the non-error code
336 path.
337
338 Conventions for callers that use the ErrorSaveContext mechanism
339 to trap errors are discussed with the declaration of that struct,
340 in nodes/miscnodes.h.
341
342
343 Functions Accepting or Returning Sets
344 -------------------------------------
345
346 If a function is marked in pg_proc as returning a set, then it is called
347 with fcinfo->resultinfo pointing to a node of type ReturnSetInfo. A
348 function that desires to return a set should raise an error "called in
349 context that does not accept a set result" if resultinfo is NULL or does
350 not point to a ReturnSetInfo node.
351
352 There are currently two modes in which a function can return a set result:
353 value-per-call, or materialize. In value-per-call mode, the function returns
354 one value each time it is called, and finally reports "done" when it has no
355 more values to return. In materialize mode, the function's output set is
356 instantiated in a Tuplestore object; all the values are returned in one call.
357 Additional modes might be added in future.
358
359 ReturnSetInfo contains a field "allowedModes" which is set (by the caller)
360 to a bitmask that's the OR of the modes the caller can support. The actual
361 mode used by the function is returned in another field "returnMode". For
362 backwards-compatibility reasons, returnMode is initialized to value-per-call
363 and need only be changed if the function wants to use a different mode.
364 The function should ereport() if it cannot use any of the modes the caller is
365 willing to support.
366
367 Value-per-call mode works like this: ReturnSetInfo contains a field
368 "isDone", which should be set to one of these values:
369
370 ExprSingleResult /* expression does not return a set */
371 ExprMultipleResult /* this result is an element of a set */
372 ExprEndResult /* there are no more elements in the set */
373
374 (the caller will initialize it to ExprSingleResult). If the function simply
375 returns a Datum without touching ReturnSetInfo, then the call is over and a
376 single-item set has been returned. To return a set, the function must set
377 isDone to ExprMultipleResult for each set element. After all elements have
378 been returned, the next call should set isDone to ExprEndResult and return a
379 null result. (Note it is possible to return an empty set by doing this on
380 the first call.)
381
382 Value-per-call functions MUST NOT assume that they will be run to completion;
383 the executor might simply stop calling them, for example because of a LIMIT.
384 Therefore, it's unsafe to attempt to perform any resource cleanup in the
385 final call. It's usually not necessary to clean up memory, anyway. If it's
386 necessary to clean up other types of resources, such as file descriptors,
387 one can register a shutdown callback function in the ExprContext pointed to
388 by the ReturnSetInfo node. (But note that file descriptors are a limited
389 resource, so it's generally unwise to hold those open across calls; SRFs
390 that need file access are better written to do it in a single call using
391 Materialize mode.)
392
393 Materialize mode works like this: the function creates a Tuplestore holding
394 the (possibly empty) result set, and returns it. There are no multiple calls.
395 The function must also return a TupleDesc that indicates the tuple structure.
396 The Tuplestore and TupleDesc should be created in the context
397 econtext->ecxt_per_query_memory (note this will *not* be the context the
398 function is called in). The function stores pointers to the Tuplestore and
399 TupleDesc into ReturnSetInfo, sets returnMode to indicate materialize mode,
400 and returns null. isDone is not used and should be left at ExprSingleResult.
401
402 The Tuplestore must be created with randomAccess = true if
403 SFRM_Materialize_Random is set in allowedModes, but it can (and preferably
404 should) be created with randomAccess = false if not. Callers that can support
405 both ValuePerCall and Materialize mode will set SFRM_Materialize_Preferred,
406 or not, depending on which mode they prefer.
407
408 If available, the expected tuple descriptor is passed in ReturnSetInfo;
409 in other contexts the expectedDesc field will be NULL. The function need
410 not pay attention to expectedDesc, but it may be useful in special cases.
411
412 InitMaterializedSRF() is a helper function able to setup the function's
413 ReturnSetInfo for a single call, filling in the Tuplestore and the
414 TupleDesc with the proper configuration for Materialize mode.
415
416 There is no support for functions accepting sets; instead, the function will
417 be called multiple times, once for each element of the input set.
418
419
420 Notes About Function Handlers
421 -----------------------------
422
423 Handlers for classes of functions should find life much easier and
424 cleaner in this design. The OID of the called function is directly
425 reachable from the passed parameters; we don't need the global variable
426 fmgr_pl_finfo anymore. Also, by modifying fcinfo->flinfo->fn_extra,
427 the handler can cache lookup info to avoid repeat lookups when the same
428 function is invoked many times. (fn_extra can only be used as a hint,
429 since callers are not required to re-use an FmgrInfo struct.
430 But in performance-critical paths they normally will do so.)
431
432 If the handler wants to allocate memory to hold fn_extra data, it should
433 NOT do so in CurrentMemoryContext, since the current context may well be
434 much shorter-lived than the context where the FmgrInfo is. Instead,
435 allocate the memory in context flinfo->fn_mcxt, or in a long-lived cache
436 context. fn_mcxt normally points at the context that was
437 CurrentMemoryContext at the time the FmgrInfo structure was created;
438 in any case it is required to be a context at least as long-lived as the
439 FmgrInfo itself.