| 1 |
NAME |
| 2 |
|
| 3 |
XS::Parse::Sublike - XS functions to assist in parsing sub-like syntax |
| 4 |
|
| 5 |
DESCRIPTION |
| 6 |
|
| 7 |
This module provides some XS functions to assist in writing parsers for |
| 8 |
sub-like syntax, primarily for authors of keyword plugins using the |
| 9 |
PL_keyword_plugin hook mechanism. It is unlikely to be of much use to |
| 10 |
anyone else; and highly unlikely to be any use when writing perl code |
| 11 |
using these. Unless you are writing a keyword plugin using XS, this |
| 12 |
module is not for you. |
| 13 |
|
| 14 |
This module is also currently experimental, and the design is still |
| 15 |
evolving and subject to change. Later versions may break ABI |
| 16 |
compatibility, requiring changes or at least a rebuild of any module |
| 17 |
that depends on it. |
| 18 |
|
| 19 |
XS FUNCTIONS |
| 20 |
|
| 21 |
boot_xs_parse_sublike |
| 22 |
|
| 23 |
void boot_xs_parse_sublike(double ver) |
| 24 |
|
| 25 |
Call this function from your BOOT section in order to initialise the |
| 26 |
module and parsing hooks. |
| 27 |
|
| 28 |
ver should either be 0 or a decimal number for the module version |
| 29 |
requirement; e.g. |
| 30 |
|
| 31 |
boot_xs_parse_sublike(0.04); |
| 32 |
|
| 33 |
xs_parse_sublike |
| 34 |
|
| 35 |
int xs_parse_sublike(const struct XSParseSublikeHooks *hooks, void *hookdata, OP **op_ptr) |
| 36 |
|
| 37 |
This function performs the actual parsing of a sub-like keyword. It |
| 38 |
expects the lexer to be at a position just after the introduction |
| 39 |
keyword has been consumed, and will proceed to parse an optional name, |
| 40 |
list of attributes, signature (if enabled by use feature 'signatures'), |
| 41 |
and code body. The return value and op_ptr can be used directly from |
| 42 |
the keyword plugin function. It is intended this function be invoked |
| 43 |
from it, and the result returned directly. |
| 44 |
|
| 45 |
For a more automated handling of keywords, see |
| 46 |
"register_xs_parse_sublike". |
| 47 |
|
| 48 |
hooks should be a structure that can provide optional function pointers |
| 49 |
used to customise the parsing process at various stages. hookdata is an |
| 50 |
opaque pointer which is passed through to each of the hook stage |
| 51 |
functions. |
| 52 |
|
| 53 |
register_xs_parse_sublike |
| 54 |
|
| 55 |
void register_xs_parse_sublike(const char *keyword, |
| 56 |
const struct XSParseSublikeHooks *hooks, void *hookdata) |
| 57 |
|
| 58 |
This function installs a set of parsing hooks to be associated with the |
| 59 |
given keyword. Such a keyword will then be handled automatically by a |
| 60 |
keyword parser installed by XS::Parse::Sublike itself. |
| 61 |
|
| 62 |
When the keyword is encountered, the hook's permit function is first |
| 63 |
tested to see if the keyword is permitted at this point. If the |
| 64 |
function returns true then the keyword is consumed and parsed as per |
| 65 |
"xs_parse_sublike". |
| 66 |
|
| 67 |
hookdata is an opaque pointer which is passed through to each of the |
| 68 |
hook stage functions when they are invoked. |
| 69 |
|
| 70 |
xs_parse_sublike_any |
| 71 |
|
| 72 |
int xs_parse_sublike_any(const struct XSParseSublikeHooks *hooks, void *hookdata, |
| 73 |
OP **op_ptr) |
| 74 |
|
| 75 |
This function expects to consume an introduction keyword at the lexer |
| 76 |
position which is either sub or the name of another sub-like keyword, |
| 77 |
which has been previously registered using "register_xs_parse_sublike". |
| 78 |
It then proceeds to parse the subsequent syntax similar to how it would |
| 79 |
have parsed if encountered by the module's own keyword parser plugin, |
| 80 |
except that the second set of hooks given here also take effect. |
| 81 |
|
| 82 |
If a regular sub is encountered, then this is parsed using the hooks in |
| 83 |
a similar way to xs_parse_sublike(). |
| 84 |
|
| 85 |
If a different registered sub-like keyword is encountered, then parsing |
| 86 |
is performed using both sets of hooks - the ones given to this function |
| 87 |
as well as the ones registered with the keyword. This allows their |
| 88 |
effects to combined. The hooks given by the hooks argument are |
| 89 |
considered to be on the "outside" from those of the registered keyword |
| 90 |
"inside". The outside ones run first for all stages, except |
| 91 |
pre_blockend which runs them inside-out. |
| 92 |
|
| 93 |
hookdata is an opaque pointer which is passed through to each of the |
| 94 |
hook stage functions when they are invoked. |
| 95 |
|
| 96 |
Note that this function is now vaguely discouraged, in favour of using |
| 97 |
a prefixing keyword instead, by using the XS_PARSE_SUBLIKE_FLAG_PREFIX |
| 98 |
flag. |
| 99 |
|
| 100 |
xps_signature_add_param |
| 101 |
|
| 102 |
void xps_signature_add_param(struct XSParseSublikeContext *ctx, |
| 103 |
struct XPSSignatureParamDetails *details); |
| 104 |
|
| 105 |
Since version 0.31; experimental. |
| 106 |
|
| 107 |
This experimental function may only be called during the |
| 108 |
start_signature or finish_signature hook stages. It is used to insert |
| 109 |
extra signature parameters, either at the beginning (when called by the |
| 110 |
start hook), or at the end (when called by the finish hook). It takes |
| 111 |
details of the parameter to add from an addressed structure, which has |
| 112 |
the following fields. |
| 113 |
|
| 114 |
struct XPSSignatureParamDetails { |
| 115 |
U32 ver; |
| 116 |
|
| 117 |
char sigil; |
| 118 |
PADOFFSET padix; |
| 119 |
}; |
| 120 |
|
| 121 |
The caller must set the ver field equal to XSPARSESUBLIKE_ABI_VERSION. |
| 122 |
|
| 123 |
The sigil field gives the leading sigil of the parameter; $ for |
| 124 |
mandatory scalars, @ or % for a final slurpy. The padix field gives the |
| 125 |
pad offset for a pad variable to store the value into. The caller is |
| 126 |
(currently) responsible for creating that pad variable. |
| 127 |
|
| 128 |
At the present version, this API cannot create optional, or named |
| 129 |
parameters. These abilities may be added in a later version which |
| 130 |
expands on the structure's definition to add new fields to support |
| 131 |
this. |
| 132 |
|
| 133 |
xps_signature_query_* |
| 134 |
|
| 135 |
IV xps_signature_query_params(struct XSParseSublikeContext *ctx); |
| 136 |
|
| 137 |
IV xps_signature_query_optparams(struct XSParseSublikeContext *ctx); |
| 138 |
|
| 139 |
char xps_signature_query_slurpy(struct XSParseSublikeContext *ctx); |
| 140 |
|
| 141 |
Since version 0.37; experimental. |
| 142 |
|
| 143 |
These experimental functions may only be called during the |
| 144 |
start_signature or finish_signature hook stages. They are used to query |
| 145 |
details about the accumulated set of signature parameters. The _params |
| 146 |
query returns the total number of defined parameters - counting all |
| 147 |
mandatory, optional, and a final slurpy if present. _optparams returns |
| 148 |
a count of only the optional parameters. _slurpy returns the sigil |
| 149 |
character of a final slurpy parameter (@ or %), or zero if no slurpy |
| 150 |
parameter is present. |
| 151 |
|
| 152 |
PARSE CONTEXT |
| 153 |
|
| 154 |
The various hook stages all share state about the ongoing parse process |
| 155 |
using various fields of the XSParseSublikeContext structure. |
| 156 |
|
| 157 |
struct XSParseSublikeContext { |
| 158 |
SV *name; |
| 159 |
OP *attrs; |
| 160 |
OP *body; |
| 161 |
CV *cv; |
| 162 |
U32 actions; |
| 163 |
HV *moddata; |
| 164 |
} |
| 165 |
|
| 166 |
The actions field will contain a bitmask of action flags that control |
| 167 |
the various steps that XS::Parse::Sublike might take inbetween invoking |
| 168 |
hook stages. The initial value of this field is set after the |
| 169 |
name-parsing stage, depending on whether or not a name is found. Stage |
| 170 |
hook functions may modify the field to adjust the subsequent behaviour. |
| 171 |
|
| 172 |
At the current ABI version, a module will have to set the |
| 173 |
XS_PARSE_SUBLIKE_COMPAT_FLAG_DYNAMIC_ACTIONS bit of the flags field in |
| 174 |
order to make use of the actions field. A future ABI version may remove |
| 175 |
this restriction. |
| 176 |
|
| 177 |
XS_PARSE_SUBLIKE_ACTION_CVf_ANON |
| 178 |
|
| 179 |
If set, the start_subparse() call will be set up for an anonymous |
| 180 |
function protosub; if not it will be set for a named function. This |
| 181 |
is set by default if a name was not found. |
| 182 |
|
| 183 |
XS_PARSE_SUBLIKE_ACTION_SET_CVNAME |
| 184 |
|
| 185 |
If set, the newly-constructed CV will have the given name set on it. |
| 186 |
This is set by default if a name was found. |
| 187 |
|
| 188 |
On Perl versions 5.22 and above, this flag can be set even if |
| 189 |
XS_PARSE_SUBLIKE_ACTION_INSTALL_SYMBOL is not. In this case, the CV |
| 190 |
will not be reachable via the symbol table, even though it knows its |
| 191 |
own name and pretends that it is. On earlier versions of perl this |
| 192 |
flag will be ignored in that case. |
| 193 |
|
| 194 |
XS_PARSE_SUBLIKE_ACTION_INSTALL_SYMBOL |
| 195 |
|
| 196 |
If set, the newly-constructed CV will be installed into the symbol |
| 197 |
table at its given name. Note that it is not possible to enable this |
| 198 |
flag without also enabling XS_PARSE_SUBLIKE_ACTION_SET_CVNAME. This |
| 199 |
is set by default if a name was found. |
| 200 |
|
| 201 |
XS_PARSE_SUBLIKE_ACTION_INSTALL_LEXICAL |
| 202 |
|
| 203 |
If set, the newly-constructed CV will be installed into the currently |
| 204 |
compiling lexical pad as its given name. This is only available on |
| 205 |
Perl version 5.18 or above, and conflicts with the alternative of |
| 206 |
XS_PARSE_SUBLIKE_ACTION_INSTALL_SYMBOL. This is set by default if a |
| 207 |
name was found and the my keyword appeared before the construction. |
| 208 |
|
| 209 |
XS_PARSE_SUBLIKE_ACTION_REFGEN_ANONCODE |
| 210 |
|
| 211 |
If set, the syntax will yield the OP_REFGEN / OP_ANONCODE optree |
| 212 |
fragment typical of anonymous code expressions; if not it will be |
| 213 |
OP_NULL. This is set by default if a name was not found. |
| 214 |
|
| 215 |
XS_PARSE_SUBLIKE_ACTION_RET_EXPR |
| 216 |
|
| 217 |
If set, the syntax will parse like an expression; if not it will |
| 218 |
parse like a statement. This is set by default if a name was not |
| 219 |
found. |
| 220 |
|
| 221 |
The moddata field will point towards an HV that modules can used to |
| 222 |
store extra data between stages. As a naming convention a module should |
| 223 |
prefix its keys with its own module name and a slash character, |
| 224 |
"Some::Module/field". The field will point to a newly-created HV for |
| 225 |
every parse invocation, and will be released when each parse is |
| 226 |
complete. |
| 227 |
|
| 228 |
PARSE HOOKS |
| 229 |
|
| 230 |
The XSParseSublikeHooks structure provides the following hook stages, |
| 231 |
which are invoked in the given order. |
| 232 |
|
| 233 |
The structure has a flags field, which controls various optional parts |
| 234 |
of operation. The following flags are defined. |
| 235 |
|
| 236 |
XS_PARSE_SUBLIKE_FLAG_BODY_OPTIONAL |
| 237 |
|
| 238 |
If not set, the require_parts field will imply the |
| 239 |
XS_PARSE_SUBLIKE_PART_BODY flag, making the body part required. By |
| 240 |
setting this flag this will no longer happen. If all hooks agree, |
| 241 |
then the body will become optional. |
| 242 |
|
| 243 |
XS_PARSE_SUBLIKE_FLAG_PREFIX |
| 244 |
|
| 245 |
If set, the keyword is considered to be a prefix that can be placed |
| 246 |
in front of sub or another sub-like keyword, to add its set of hooks |
| 247 |
in addition to those of the following keyword. These prefices may be |
| 248 |
further stacked. |
| 249 |
|
| 250 |
XS_PARSE_SUBLIKE_FLAG_ALLOW_PKGNAME |
| 251 |
|
| 252 |
Since version 0.31. |
| 253 |
|
| 254 |
If not set, then fully-qualified identifiers that include a package |
| 255 |
name are not allowed when declaring a named function. If all hooks |
| 256 |
agree by all setting the flag, then the name may be fully-qualified |
| 257 |
to add the newly-declared function into a different package. |
| 258 |
|
| 259 |
XS_PARSE_SUBLIKE_FLAG_SIGNATURE_NAMED_PARAMS |
| 260 |
|
| 261 |
If set, use the extended signature parser of this module when parsing |
| 262 |
a signature and additionally permit the use of named parameter |
| 263 |
syntax, as documented in Sublike::Extended. |
| 264 |
|
| 265 |
XS_PARSE_SUBLIKE_FLAG_SIGNATURE_PARAM_ATTRIBUTES |
| 266 |
|
| 267 |
If set, use the extended signature parser of this module when parsing |
| 268 |
a signature and additionally permit the use of attribute declarations |
| 269 |
on parameter variables, as documented in Sublike::Extended. |
| 270 |
|
| 271 |
XS_PARSE_SUBLIKE_FLAG_SIGNATURE_REFALIAS |
| 272 |
|
| 273 |
If set, use the extended signature parser of this module when parsing |
| 274 |
a signature and additionally permit the use of refalias parameter |
| 275 |
syntax, as documented in Sublike::Extended. |
| 276 |
|
| 277 |
In addition there are two U8 fields named require_parts and skip_parts |
| 278 |
which control the behaviour of various parts of the syntax which are |
| 279 |
usually optional. Any parts with bits set in require_parts become |
| 280 |
non-optional, and an error if they are missing. Any parts with bits set |
| 281 |
in skip_parts will skip the relevant part of the parsing process. |
| 282 |
|
| 283 |
When multiple sets of hooks are combined by the xs_parse_sublike_any |
| 284 |
function, or as part of parsing prefixing keywords, these bitmasks are |
| 285 |
accumulated together with inclusive or. Any part required by any set of |
| 286 |
hooks will still be required; any step skipped by either will be |
| 287 |
skipped entirely. |
| 288 |
|
| 289 |
If the same bit is set in both fields then the relevant parsing step |
| 290 |
will not be performed but it will still be an error for that section to |
| 291 |
be missing. This is likely not useful. |
| 292 |
|
| 293 |
Note that for skipped parts, only the actual parsing steps are skipped. |
| 294 |
A hook function can still set the relevant fields in the context |
| 295 |
structure anyway to force a particular value for those parts. |
| 296 |
|
| 297 |
XS_PARSE_SUBLIKE_PART_NAME |
| 298 |
|
| 299 |
The name of the function. |
| 300 |
|
| 301 |
XS_PARSE_SUBLIKE_PART_ATTRS |
| 302 |
|
| 303 |
The attributes of the function. |
| 304 |
|
| 305 |
This part can be skipped, but the bit is ignored when in |
| 306 |
require_parts. It is always permitted to not provide any additional |
| 307 |
attributes to a function definition. |
| 308 |
|
| 309 |
XS_PARSE_SUBLIKE_PART_SIGNATURE |
| 310 |
|
| 311 |
The parameter signature of the function. |
| 312 |
|
| 313 |
This part can be skipped, but it is always permitted not to provide a |
| 314 |
signature for a function definition even if the bit it set in |
| 315 |
require_parts. This is because such syntax only applies when use |
| 316 |
feature 'signatures' is in effect, and only on supporting perl |
| 317 |
versions. |
| 318 |
|
| 319 |
However, setting the bit in require_parts instead has the effect of |
| 320 |
enabling use feature 'signatures' (at least on supporting perl |
| 321 |
versions), thus permitting the syntax to use a signature even if the |
| 322 |
signatures feature was not previously enabled. |
| 323 |
|
| 324 |
XS_PARSE_SUBLIKE_PART_BODY |
| 325 |
|
| 326 |
The actual body of the function, expressed as a brace-delimited |
| 327 |
block. |
| 328 |
|
| 329 |
This part cannot be skipped, but it can be made optional by omitting |
| 330 |
it from the require_parts field. Instead of the block, it is |
| 331 |
permitted to place a single semicolon (;) to act as a statement |
| 332 |
terminator; thus giving the same syntax as a subroutine forward |
| 333 |
declaration. |
| 334 |
|
| 335 |
In this case, the body and cv fields of the context structure will |
| 336 |
remain NULL. |
| 337 |
|
| 338 |
This flag is currently implied on the require_parts field if the hook |
| 339 |
does not supply the XS_PARSE_SUBLIKE_FLAG_BODY_OPTIONAL flag; meaning |
| 340 |
that most use-cases will make it a required part. |
| 341 |
|
| 342 |
The permit Stage |
| 343 |
|
| 344 |
const char *permit_hintkey |
| 345 |
bool (*permit)(pTHX_ void *hookdata) |
| 346 |
|
| 347 |
Called by the installed keyword parser hook which is used to handle |
| 348 |
keywords registered by "register_xs_parse_sublike". |
| 349 |
|
| 350 |
As a shortcut for the common case, the permit_hintkey may point to a |
| 351 |
string to look up from the hints hash. If the given key name is not |
| 352 |
found in the hints hash then the keyword is not permitted. If the key |
| 353 |
is present then the permit function is invoked as normal. |
| 354 |
|
| 355 |
If not rejected by a hint key that was not found in the hints hash, the |
| 356 |
function part of the stage is called next and should inspect whether |
| 357 |
the keyword is permitted at this time perhaps by inspecting other |
| 358 |
lexical clues, and return true only if the keyword is permitted. |
| 359 |
|
| 360 |
Both the string and the function are optional. Either or both may be |
| 361 |
present. If neither is present then the keyword is always permitted - |
| 362 |
which is likely not what you wanted to do. |
| 363 |
|
| 364 |
Parse Name |
| 365 |
|
| 366 |
At this point, the optional name is parsed and filled into the name |
| 367 |
field of the context. |
| 368 |
|
| 369 |
The pre_subparse Stage |
| 370 |
|
| 371 |
void (*pre_subparse)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata) |
| 372 |
|
| 373 |
Invoked just before start_subparse() is called. |
| 374 |
|
| 375 |
Parse Attrs |
| 376 |
|
| 377 |
At this point the optional sub attributes are parsed and filled into |
| 378 |
the attrs field of the context, then block_start() is called. |
| 379 |
|
| 380 |
The filter_attr Stage |
| 381 |
|
| 382 |
bool (*filter_attr)(pTHX_ struct XSParseSublikeContext *ctx, |
| 383 |
SV *attr, SV *val, void *hookdata); |
| 384 |
|
| 385 |
If this optional stage is defined, then each individual attribute is |
| 386 |
passed through this optional filter function immediately as each is |
| 387 |
parsed. attr will be a string SV containing the name of the attribute, |
| 388 |
and val will either be NULL, or a string SV containing the contents of |
| 389 |
the parens after its name (without the parens themselves). |
| 390 |
|
| 391 |
If the filter returns true, it indicates that it has in some way |
| 392 |
handled the attribute and it should not be added to the list given to |
| 393 |
newATTRSUB(). If the filter returns false it will be handled in the |
| 394 |
usual way; equivalent to the case where the filter function did not |
| 395 |
exist. |
| 396 |
|
| 397 |
The post_blockstart Stage |
| 398 |
|
| 399 |
void (*post_blockstart)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata) |
| 400 |
|
| 401 |
Invoked after the block_start() function has been called. This hook |
| 402 |
stage may wish to perform any alterations of PL_compcv or related, |
| 403 |
inspect or alter the lexical pad, provide hints hash values, or any |
| 404 |
other tasks before the signature and code body are parsed. |
| 405 |
|
| 406 |
Parse Signature |
| 407 |
|
| 408 |
If the perl version supports subroutine signatures, and the feature is |
| 409 |
enabled at this point, then an optional signatured is expected. |
| 410 |
|
| 411 |
Since version 0.31: if the open parenthesis of a signature declaration |
| 412 |
is found, then the start_signature stage is invoked. Once the signature |
| 413 |
has been parsed, the finish_signature stage is invoked. |
| 414 |
|
| 415 |
void (*start_signature)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata); |
| 416 |
|
| 417 |
void (*finish_signature)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata); |
| 418 |
|
| 419 |
Code in either of these hook stages is permitted to call |
| 420 |
"xps_signature_add_param", or any of the xps_signature_query_*() |
| 421 |
functions. |
| 422 |
|
| 423 |
Parse Body |
| 424 |
|
| 425 |
At this point, the main body of the function is parsed and the optree |
| 426 |
is stored in the body field of the context. If a subroutine signature |
| 427 |
was found, the body will be prefixed with the signature ops as well. |
| 428 |
|
| 429 |
The pre_blockend Stage |
| 430 |
|
| 431 |
void (*pre_blockend)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata) |
| 432 |
|
| 433 |
Invoked just before the block_end() function is invoked. The hook stage |
| 434 |
may wish to inspect or alter the optree stored in the body context |
| 435 |
field. |
| 436 |
|
| 437 |
The post_newcv Stage |
| 438 |
|
| 439 |
void (*post_newcv)(pTHX_ struct XSParseSublikeContext *ctx, void *hookdata) |
| 440 |
|
| 441 |
Invoked just after newATTRSUB() has been invoked on the optree. The |
| 442 |
hook stage may wish to inspect or alter the CV stored in the cv context |
| 443 |
field. |
| 444 |
|
| 445 |
AUTHOR |
| 446 |
|
| 447 |
Paul Evans <[email protected]> |
| 448 |
|