Debian Codegraph @

upstream/fedora-44/man1/perlguts.1

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\"
56 .\" Required to disable full justification in groff 1.23.0.
57 .if n .ds AD l
58 .\" ========================================================================
59 .\"
60 .IX Title "PERLGUTS 1"
61 .TH PERLGUTS 1 2026-03-09 "perl v5.42.1" "Perl Programmers Reference Guide"
62 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
63 .\" way too many mistakes in technical documents.
64 .if n .ad l
65 .nh
66 .SH NAME
67 perlguts \- Introduction to the Perl API
68 .SH DESCRIPTION
69 .IX Header "DESCRIPTION"
70 This document attempts to describe how to use the Perl API, as well as
71 to provide some info on the basic workings of the Perl core. It is far
72 from complete and probably contains many errors. Please refer any
73 questions or comments to the author below.
74 .SH Variables
75 .IX Header "Variables"
76 .SS Datatypes
77 .IX Subsection "Datatypes"
78 Perl has three typedefs that handle Perl\*(Aqs three main data types:
79 .PP
80 .Vb 3
81 \& SV Scalar Value
82 \& AV Array Value
83 \& HV Hash Value
84 .Ve
85 .PP
86 Each typedef has specific routines that manipulate the various data types.
87 .SS "What is an ""IV""?"
88 .IX Subsection "What is an ""IV""?"
89 Perl uses a special typedef IV which is a simple signed integer type that is
90 guaranteed to be large enough to hold a pointer, but may be larger,
91 depending on how perl is Configured.
92 Additionally, there is the UV, which is simply an unsigned IV.
93 .PP
94 Perl also uses several special typedefs to declare variables to hold
95 integers of (at least) a given size.
96 Use I8, I16, I32, and I64 to declare a signed integer variable which has
97 at least as many bits as the number in its name. These all evaluate to
98 the native C type that is closest to the given number of bits, but no
99 smaller than that number. For example, on many platforms, a \f(CW\*(C`short\*(C'\fR is
100 16 bits long, and if so, I16 will evaluate to a \f(CW\*(C`short\*(C'\fR. But on
101 platforms where a \f(CW\*(C`short\*(C'\fR isn\*(Aqt exactly 16 bits, Perl will use the
102 smallest type that contains 16 bits or more.
103 .PP
104 U8, U16, U32, and U64 are to declare the corresponding unsigned integer
105 types.
106 .PP
107 If the platform doesn\*(Aqt support 64\-bit integers, both I64 and U64 will
108 be undefined. Use IV and UV to declare the largest practicable, and
109 \&\f(CW\*(C`"WIDEST_UTYPE" in perlapi\*(C'\fR for the absolute maximum unsigned, but which
110 may not be usable in all circumstances.
111 .PP
112 A numeric constant can be specified with "\f(CW\*(C`INT16_C\*(C'\fR" in perlapi,
113 "\f(CW\*(C`UINTMAX_C\*(C'\fR" in perlapi, and similar.
114 .SS "Working with SVs"
115 .IX Subsection "Working with SVs"
116 An SV can be created and loaded with one command. There are five types of
117 values that can be loaded: an integer value (IV), an unsigned integer
118 value (UV), a double (NV), a string (PV), and another scalar (SV).
119 ("PV" stands for "Pointer Value". You might think that it is misnamed
120 because it is described as pointing only to strings. However, it is
121 possible to have it point to other things. For example, it could point
122 to an array of UVs. But,
123 using it for non\-strings requires care, as the underlying assumption of
124 much of the internals is that PVs are just for strings. Often, for
125 example, a trailing \f(CW\*(C`NUL\*(C'\fR is tacked on automatically. The non\-string use
126 is documented only in this paragraph.)
127 .PP
128 There are more than a few routines that create and initialize SVs, with
129 new ones occasionally added. These are all documented in "SV
130 Handling" in perlapi; each one\*(Aqs name begins with \f(CW\*(C`newSV\*(C'\fR. Here are the basic
131 ones:
132 .PP
133 .Vb 10
134 \& SV* newSVbool(boolean_value);
135 \& SV* newSViv(IV);
136 \& SV* newSVuv(UV);
137 \& SV* newSVnv(double);
138 \& SV* newSVpv(const char*, STRLEN);
139 \& SV* newSVpvf_nocontext(const char * const pat, ...)
140 \& SV* newSVpvn(const char*, STRLEN);
141 \& SV* newSVpvs("literal string")
142 \& SV* newSVpvz(STRLEN)
143 \& SV* newSVsv(SV*);
144 .Ve
145 .PP
146 \&\f(CW\*(C`STRLEN\*(C'\fR is an integer type (\f(CW\*(C`Size_t\*(C'\fR, usually defined as \f(CW\*(C`size_t\*(C'\fR in
147 \&\fIconfig.h\fR) guaranteed to be large enough to represent the size of
148 any string that perl can handle.
149 .PP
150 \&\f(CW"literal\ string"\fR is a string enclosed in double quote characters,
151 and the created SV is of string type (a PV), initialized to contain a
152 copy of the literal string parameter. \f(CWnewSVpvs("")\fR creates an SV
153 containing an empty (zero\-length) string and returns a pointer to it.
154 .PP
155 \&\f(CW\*(C`newSVpvz\*(C'\fR is like \f(CWnewSVpvs("")\fR, but it reserves the number of bytes
156 specified by its parameter for future growth, without having to
157 reallocate memory. It is useful in situations where you don\*(Aqt know what
158 the string eventually will be, but you have a pretty good idea of its
159 eventual minimum size.
160 .PP
161 In the case of an SV requiring more complex initialization, you
162 can create an empty SV with newSV(len). If \f(CW\*(C`len\*(C'\fR is 0 an empty SV of
163 type NULL is returned, else an SV of type PV is returned with len + 1 (for
164 the \f(CW\*(C`NUL\*(C'\fR) bytes of storage allocated, accessible via SvPVX. In both cases
165 the SV has the undef value.
166 .PP
167 .Vb 3
168 \& SV *sv = newSV(0); /* no storage allocated */
169 \& SV *sv = newSV(10); /* 10 (+1) bytes of uninitialised storage
170 \& * allocated */
171 .Ve
172 .PP
173 To change the value of an \fIalready\-existing\fR SV, these are the basic
174 routines (with more specified in "SV Handling" in perlapi):
175 .PP
176 .Vb 10
177 \& void sv_setbool(SV*, bool);
178 \& void sv_setiv(SV*, IV);
179 \& void sv_setuv(SV*, UV);
180 \& void sv_setnv(SV*, double);
181 \& void sv_setpv(SV*, const char*);
182 \& void sv_setpvn(SV*, const char*, STRLEN)
183 \& void sv_setpvf(SV*, const char*, ...);
184 \& void sv_vsetpvfn(SV*, const char*, STRLEN, va_list *,
185 \& SV **, Size_t, bool *);
186 \& void sv_setsv(SV*, SV*);
187 .Ve
188 .PP
189 Notice that you can choose to specify the length of the string to be
190 assigned by using \f(CW\*(C`sv_setpvn\*(C'\fR, \f(CW\*(C`newSVpvn\*(C'\fR, or \f(CW\*(C`newSVpv\*(C'\fR, or you may
191 allow Perl to calculate the length by using \f(CW\*(C`sv_setpv\*(C'\fR or by specifying
192 0 as the second argument to \f(CW\*(C`newSVpv\*(C'\fR. Be warned, though, that Perl will
193 determine the string\*(Aqs length by using \f(CW\*(C`strlen\*(C'\fR, which depends on the
194 string terminating with a \f(CW\*(C`NUL\*(C'\fR character, and not otherwise containing
195 NULs.
196 .PP
197 The arguments of \f(CW\*(C`sv_setpvf\*(C'\fR are processed like \f(CW\*(C`sprintf\*(C'\fR, and the
198 formatted output becomes the value.
199 .PP
200 \&\f(CW\*(C`sv_vsetpvfn\*(C'\fR is an analogue of \f(CW\*(C`vsprintf\*(C'\fR, but it allows you to specify
201 either a pointer to a variable argument list or the address and length of
202 an array of SVs. The last argument points to a boolean; on return, if that
203 boolean is true, then locale\-specific information has been used to format
204 the string, and the string\*(Aqs contents are therefore untrustworthy (see
205 perlsec). This pointer may be NULL if that information is not
206 important. Note that this function requires you to specify the length of
207 the format.
208 .PP
209 The \f(CW\*(C`sv_set*()\*(C'\fR functions are not generic enough to operate on values
210 that have "magic". See "Magic Virtual Tables" later in this document.
211 .PP
212 All SVs that contain strings should be terminated with a \f(CW\*(C`NUL\*(C'\fR character.
213 If it is not \f(CW\*(C`NUL\*(C'\fR\-terminated there is a risk of
214 core dumps and corruptions from code which passes the string to C
215 functions or system calls which expect a \f(CW\*(C`NUL\*(C'\fR\-terminated string.
216 Perl\*(Aqs own functions typically add a trailing \f(CW\*(C`NUL\*(C'\fR for this reason.
217 Nevertheless, you should be very careful when you pass a string stored
218 in an SV to a C function or system call.
219 .PP
220 To access the actual value that an SV points to, Perl\*(Aqs API exposes
221 several macros that coerce the actual scalar type into an IV, UV, double,
222 or string:
223 .IP \(bu 4
224 \&\f(CWSvIV(SV*)\fR (\f(CW\*(C`IV\*(C'\fR) and \f(CWSvUV(SV*)\fR (\f(CW\*(C`UV\*(C'\fR)
225 .IP \(bu 4
226 \&\f(CWSvNV(SV*)\fR (\f(CW\*(C`double\*(C'\fR)
227 .IP \(bu 4
228 Strings are a bit complicated:
229 .RS 4
230 .IP \(bu 4
231 Byte string: \f(CW\*(C`SvPVbyte(SV*, STRLEN len)\*(C'\fR or \f(CWSvPVbyte_nolen(SV*)\fR
232 .Sp
233 If the Perl string is \f(CW"\exff\exff"\fR, then this returns a 2\-byte \f(CW\*(C`char*\*(C'\fR.
234 .Sp
235 This is suitable for Perl strings that represent bytes.
236 .IP \(bu 4
237 UTF\-8 string: \f(CW\*(C`SvPVutf8(SV*, STRLEN len)\*(C'\fR or \f(CWSvPVutf8_nolen(SV*)\fR
238 .Sp
239 If the Perl string is \f(CW"\exff\exff"\fR, then this returns a 4\-byte \f(CW\*(C`char*\*(C'\fR.
240 .Sp
241 This is suitable for Perl strings that represent characters.
242 .Sp
243 \&\fBCAVEAT\fR: That \f(CW\*(C`char*\*(C'\fR will be encoded via Perl\*(Aqs internal UTF\-8 variant,
244 which means that if the SV contains non\-Unicode code points (e.g.,
245 0x110000), then the result may contain extensions over valid UTF\-8.
246 See "is_strict_utf8_string" in perlapi for some methods Perl gives
247 you to check the UTF\-8 validity of these macros\*(Aq returns.
248 .IP \(bu 4
249 You can also use \f(CW\*(C`SvPV(SV*, STRLEN len)\*(C'\fR or \f(CWSvPV_nolen(SV*)\fR
250 to fetch the SV\*(Aqs raw internal buffer. This is tricky, though; if your Perl
251 string
252 is \f(CW"\exff\exff"\fR, then depending on the SV\*(Aqs internal encoding you might get
253 back a 2\-byte \fBOR\fR a 4\-byte \f(CW\*(C`char*\*(C'\fR.
254 Moreover, if it\*(Aqs the 4\-byte string, that could come from either Perl
255 \&\f(CW"\exff\exff"\fR stored UTF\-8 encoded, or Perl \f(CW"\exc3\exbf\exc3\exbf"\fR stored
256 as raw octets. To differentiate between these you \fBMUST\fR look up the
257 SV\*(Aqs UTF8 bit (cf. \f(CW\*(C`SvUTF8\*(C'\fR) to know whether the source Perl string
258 is 2 characters (\f(CW\*(C`SvUTF8\*(C'\fR would be on) or 4 characters (\f(CW\*(C`SvUTF8\*(C'\fR would be
259 off).
260 .Sp
261 \&\fBIMPORTANT:\fR Use of \f(CW\*(C`SvPV\*(C'\fR, \f(CW\*(C`SvPV_nolen\*(C'\fR, or
262 similarly\-named macros \fIwithout\fR looking up the SV\*(Aqs UTF8 bit is
263 almost certainly a bug if non\-ASCII input is allowed.
264 .Sp
265 When the UTF8 bit is on, the same \fBCAVEAT\fR about UTF\-8 validity applies
266 here as for \f(CW\*(C`SvPVutf8\*(C'\fR.
267 .RE
268 .RS 4
269 .Sp
270 (See "How do I pass a Perl string to a C library?" for more details.)
271 .Sp
272 In \f(CW\*(C`SvPVbyte\*(C'\fR, \f(CW\*(C`SvPVutf8\*(C'\fR, and \f(CW\*(C`SvPV\*(C'\fR, the length of the \f(CW\*(C`char*\*(C'\fR returned
273 is placed into the
274 variable \f(CW\*(C`len\*(C'\fR (these are macros, so you do \fInot\fR use \f(CW&len\fR). If you do
275 not care what the length of the data is, use \f(CW\*(C`SvPVbyte_nolen\*(C'\fR,
276 \&\f(CW\*(C`SvPVutf8_nolen\*(C'\fR, or \f(CW\*(C`SvPV_nolen\*(C'\fR instead.
277 The global variable \f(CW\*(C`PL_na\*(C'\fR can also be given to
278 \&\f(CW\*(C`SvPVbyte\*(C'\fR/\f(CW\*(C`SvPVutf8\*(C'\fR/\f(CW\*(C`SvPV\*(C'\fR
279 in this case. But that can be quite inefficient because \f(CW\*(C`PL_na\*(C'\fR must
280 be accessed in thread\-local storage in threaded Perl. In any case, remember
281 that Perl allows arbitrary strings of data that may both contain NULs and
282 might not be terminated by a \f(CW\*(C`NUL\*(C'\fR.
283 .Sp
284 Also remember that C doesn\*(Aqt allow you to safely say \f(CW\*(C`foo(SvPVbyte(s, len),
285 len);\*(C'\fR. It might work with your
286 compiler, but it won\*(Aqt work for everyone.
287 Break this sort of statement up into separate assignments:
288 .Sp
289 .Vb 5
290 \& SV *s;
291 \& STRLEN len;
292 \& char *ptr;
293 \& ptr = SvPVbyte(s, len);
294 \& foo(ptr, len);
295 .Ve
296 .RE
297 .PP
298 If you want to know if the scalar value is TRUE, you can use:
299 .PP
300 .Vb 1
301 \& SvTRUE(SV*)
302 .Ve
303 .PP
304 Although Perl will automatically grow strings for you, if you need to force
305 Perl to allocate more memory for your SV, you can use the macro
306 .PP
307 .Vb 1
308 \& SvGROW(SV*, STRLEN newlen)
309 .Ve
310 .PP
311 which will determine if more memory needs to be allocated. If so, it will
312 call the function \f(CW\*(C`sv_grow\*(C'\fR. Note that \f(CW\*(C`SvGROW\*(C'\fR can only increase, not
313 decrease, the allocated memory of an SV and that it does not automatically
314 add space for the trailing \f(CW\*(C`NUL\*(C'\fR byte (perl\*(Aqs own string functions typically do
315 \&\f(CW\*(C`SvGROW(sv, len + 1)\*(C'\fR).
316 .PP
317 If you want to write to an existing SV\*(Aqs buffer and set its value to a
318 string, use \fBSvPVbyte_force()\fR or one of its variants to force the SV to be
319 a PV. This will remove any of various types of non\-stringness from
320 the SV while preserving the content of the SV in the PV. This can be
321 used, for example, to append data from an API function to a buffer
322 without extra copying:
323 .PP
324 .Vb 11
325 \& (void)SvPVbyte_force(sv, len);
326 \& s = SvGROW(sv, len + needlen + 1);
327 \& /* something that modifies up to needlen bytes at s+len, but
328 \& modifies newlen bytes
329 \& eg. newlen = read(fd, s + len, needlen);
330 \& ignoring errors for these examples
331 \& */
332 \& s[len + newlen] = \*(Aq\e0\*(Aq;
333 \& SvCUR_set(sv, len + newlen);
334 \& SvUTF8_off(sv);
335 \& SvSETMAGIC(sv);
336 .Ve
337 .PP
338 If you already have the data in memory or if you want to keep your
339 code simple, you can use one of the sv_cat*() variants, such as
340 \&\fBsv_catpvn()\fR. If you want to insert anywhere in the string you can use
341 \&\fBsv_insert()\fR or \fBsv_insert_flags()\fR.
342 .PP
343 If you don\*(Aqt need the existing content of the SV, you can avoid some
344 copying with:
345 .PP
346 .Vb 10
347 \& SvPVCLEAR(sv);
348 \& s = SvGROW(sv, needlen + 1);
349 \& /* something that modifies up to needlen bytes at s, but modifies
350 \& newlen bytes
351 \& eg. newlen = read(fd, s, needlen);
352 \& */
353 \& s[newlen] = \*(Aq\e0\*(Aq;
354 \& SvCUR_set(sv, newlen);
355 \& SvPOK_only(sv); /* also clears SVf_UTF8 */
356 \& SvSETMAGIC(sv);
357 .Ve
358 .PP
359 Again, if you already have the data in memory or want to avoid the
360 complexity of the above, you can use \fBsv_setpvn()\fR.
361 .PP
362 If you have a buffer allocated with \fBNewx()\fR and want to set that as the
363 SV\*(Aqs value, you can use \fBsv_usepvn_flags()\fR. That has some requirements
364 if you want to avoid perl re\-allocating the buffer to fit the trailing
365 NUL:
366 .PP
367 .Vb 5
368 \& Newx(buf, somesize+1, char);
369 \& /* ... fill in buf ... */
370 \& buf[somesize] = \*(Aq\e0\*(Aq;
371 \& sv_usepvn_flags(sv, buf, somesize, SV_SMAGIC | SV_HAS_TRAILING_NUL);
372 \& /* buf now belongs to perl, don\*(Aqt release it */
373 .Ve
374 .PP
375 If you have an SV and want to know what kind of data Perl thinks is stored
376 in it, you can use the following macros to check the type of SV you have.
377 .PP
378 .Vb 3
379 \& SvIOK(SV*)
380 \& SvNOK(SV*)
381 \& SvPOK(SV*)
382 .Ve
383 .PP
384 Be aware that retrieving the numeric value of an SV can set IOK or NOK
385 on that SV, even when the SV started as a string. Prior to Perl
386 5.36.0 retrieving the string value of an integer could set POK, but
387 this can no longer occur. From 5.36.0 this can be used to distinguish
388 the original representation of an SV and is intended to make life
389 simpler for serializers:
390 .PP
391 .Vb 10
392 \& /* references handled elsewhere */
393 \& if (SvIsBOOL(sv)) {
394 \& /* originally boolean */
395 \& ...
396 \& }
397 \& else if (SvPOK(sv)) {
398 \& /* originally a string */
399 \& ...
400 \& }
401 \& else if (SvNIOK(sv)) {
402 \& /* originally numeric */
403 \& ...
404 \& }
405 \& else {
406 \& /* something special or undef */
407 \& }
408 .Ve
409 .PP
410 You can get and set the current length of the string stored in an SV with
411 the following macros:
412 .PP
413 .Vb 2
414 \& SvCUR(SV*)
415 \& SvCUR_set(SV*, I32 val)
416 .Ve
417 .PP
418 You can also get a pointer to the end of the string stored in the SV
419 with the macro:
420 .PP
421 .Vb 1
422 \& SvEND(SV*)
423 .Ve
424 .PP
425 But note that these last three macros are valid only if \f(CWSvPOK()\fR is true.
426 .PP
427 If you want to append something to the end of string stored in an \f(CW\*(C`SV*\*(C'\fR,
428 you can use the following functions:
429 .PP
430 .Vb 6
431 \& void sv_catpv(SV*, const char*);
432 \& void sv_catpvn(SV*, const char*, STRLEN);
433 \& void sv_catpvf(SV*, const char*, ...);
434 \& void sv_vcatpvfn(SV*, const char*, STRLEN, va_list *, SV **,
435 \& I32, bool);
436 \& void sv_catsv(SV*, SV*);
437 .Ve
438 .PP
439 The first function calculates the length of the string to be appended by
440 using \f(CW\*(C`strlen\*(C'\fR. In the second, you specify the length of the string
441 yourself. The third function processes its arguments like \f(CW\*(C`sprintf\*(C'\fR and
442 appends the formatted output. The fourth function works like \f(CW\*(C`vsprintf\*(C'\fR.
443 You can specify the address and length of an array of SVs instead of the
444 va_list argument. The fifth function
445 extends the string stored in the first
446 SV with the string stored in the second SV. It also forces the second SV
447 to be interpreted as a string.
448 .PP
449 The \f(CW\*(C`sv_cat*()\*(C'\fR functions are not generic enough to operate on values that
450 have "magic". See "Magic Virtual Tables" later in this document.
451 .PP
452 If you know the name of a scalar variable, you can get a pointer to its SV
453 by using the following:
454 .PP
455 .Vb 1
456 \& SV* get_sv("package::varname", 0);
457 .Ve
458 .PP
459 This returns NULL if the variable does not exist.
460 .PP
461 If you want to know if this variable (or any other SV) is actually \f(CW\*(C`defined\*(C'\fR,
462 you can call:
463 .PP
464 .Vb 1
465 \& SvOK(SV*)
466 .Ve
467 .PP
468 The scalar \f(CW\*(C`undef\*(C'\fR value is stored in an SV instance called \f(CW\*(C`PL_sv_undef\*(C'\fR.
469 .PP
470 Its address can be used whenever an \f(CW\*(C`SV*\*(C'\fR is needed. Make sure that
471 you don\*(Aqt try to compare a random sv with \f(CW&PL_sv_undef\fR. For example
472 when interfacing Perl code, it\*(Aqll work correctly for:
473 .PP
474 .Vb 1
475 \& foo(undef);
476 .Ve
477 .PP
478 But won\*(Aqt work when called as:
479 .PP
480 .Vb 2
481 \& $x = undef;
482 \& foo($x);
483 .Ve
484 .PP
485 So to repeat always use \fBSvOK()\fR to check whether an sv is defined.
486 .PP
487 Also you have to be careful when using \f(CW&PL_sv_undef\fR as a value in
488 AVs or HVs (see "AVs, HVs and undefined values").
489 .PP
490 There are also the two values \f(CW\*(C`PL_sv_yes\*(C'\fR and \f(CW\*(C`PL_sv_no\*(C'\fR, which contain
491 boolean TRUE and FALSE values, respectively. Like \f(CW\*(C`PL_sv_undef\*(C'\fR, their
492 addresses can be used whenever an \f(CW\*(C`SV*\*(C'\fR is needed.
493 .PP
494 Do not be fooled into thinking that \f(CW\*(C`(SV *) 0\*(C'\fR is the same as \f(CW&PL_sv_undef\fR.
495 Take this code:
496 .PP
497 .Vb 5
498 \& SV* sv = (SV*) 0;
499 \& if (I\-am\-to\-return\-a\-real\-value) {
500 \& sv = sv_2mortal(newSViv(42));
501 \& }
502 \& sv_setsv(ST(0), sv);
503 .Ve
504 .PP
505 This code tries to return a new SV (which contains the value 42) if it should
506 return a real value, or undef otherwise. Instead it has returned a NULL
507 pointer which, somewhere down the line, will cause a segmentation violation,
508 bus error, or just weird results. Change the zero to \f(CW&PL_sv_undef\fR in the
509 first line and all will be well.
510 .PP
511 To free an SV that you\*(Aqve created, call \f(CWSvREFCNT_dec(SV*)\fR. Normally this
512 call is not necessary (see "Reference Counts and Mortality").
513 .SS Offsets
514 .IX Subsection "Offsets"
515 Perl provides the function \f(CW\*(C`sv_chop\*(C'\fR to efficiently remove characters
516 from the beginning of a string; you give it an SV and a pointer to
517 somewhere inside the PV, and it discards everything before the
518 pointer. The efficiency comes by means of a little hack: instead of
519 actually removing the characters, \f(CW\*(C`sv_chop\*(C'\fR sets the flag \f(CW\*(C`OOK\*(C'\fR
520 (offset OK) to signal to other functions that the offset hack is in
521 effect, and it moves the PV pointer (called \f(CW\*(C`SvPVX\*(C'\fR) forward
522 by the number of bytes chopped off, and adjusts \f(CW\*(C`SvCUR\*(C'\fR and \f(CW\*(C`SvLEN\*(C'\fR
523 accordingly. (A portion of the space between the old and new PV
524 pointers is used to store the count of chopped bytes.)
525 .PP
526 Hence, at this point, the start of the buffer that we allocated lives
527 at \f(CW\*(C`SvPVX(sv) \- SvIV(sv)\*(C'\fR in memory and the PV pointer is pointing
528 into the middle of this allocated storage.
529 .PP
530 This is best demonstrated by example. Normally copy\-on\-write will prevent
531 the substitution from operator from using this hack, but if you can craft a
532 string for which copy\-on\-write is not possible, you can see it in play. In
533 the current implementation, the final byte of a string buffer is used as a
534 copy\-on\-write reference count. If the buffer is not big enough, then
535 copy\-on\-write is skipped. First have a look at an empty string:
536 .PP
537 .Vb 7
538 \& % ./perl \-Ilib \-MDevel::Peek \-le \*(Aq$a=""; $a .= ""; Dump $a\*(Aq
539 \& SV = PV(0x7ffb7c008a70) at 0x7ffb7c030390
540 \& REFCNT = 1
541 \& FLAGS = (POK,pPOK)
542 \& PV = 0x7ffb7bc05b50 ""\e0
543 \& CUR = 0
544 \& LEN = 10
545 .Ve
546 .PP
547 Notice here the LEN is 10. (It may differ on your platform.) Extend the
548 length of the string to one less than 10, and do a substitution:
549 .PP
550 .Vb 9
551 \& % ./perl \-Ilib \-MDevel::Peek \-le \*(Aq$a=""; $a.="123456789"; $a=~s/.//; \e
552 \& Dump($a)\*(Aq
553 \& SV = PV(0x7ffa04008a70) at 0x7ffa04030390
554 \& REFCNT = 1
555 \& FLAGS = (POK,OOK,pPOK)
556 \& OFFSET = 1
557 \& PV = 0x7ffa03c05b61 ( "\e1" . ) "23456789"\e0
558 \& CUR = 8
559 \& LEN = 9
560 .Ve
561 .PP
562 Here the number of bytes chopped off (1) is shown next as the OFFSET. The
563 portion of the string between the "real" and the "fake" beginnings is
564 shown in parentheses, and the values of \f(CW\*(C`SvCUR\*(C'\fR and \f(CW\*(C`SvLEN\*(C'\fR reflect
565 the fake beginning, not the real one. (The first character of the string
566 buffer happens to have changed to "\e1" here, not "1", because the current
567 implementation stores the offset count in the string buffer. This is
568 subject to change.)
569 .PP
570 Something similar to the offset hack is performed on AVs to enable
571 efficient shifting and splicing off the beginning of the array; while
572 \&\f(CW\*(C`AvARRAY\*(C'\fR points to the first element in the array that is visible from
573 Perl, \f(CW\*(C`AvALLOC\*(C'\fR points to the real start of the C array. These are
574 usually the same, but a \f(CW\*(C`shift\*(C'\fR operation can be carried out by
575 increasing \f(CW\*(C`AvARRAY\*(C'\fR by one and decreasing \f(CW\*(C`AvFILL\*(C'\fR and \f(CW\*(C`AvMAX\*(C'\fR.
576 Again, the location of the real start of the C array only comes into
577 play when freeing the array. See \f(CW\*(C`av_shift\*(C'\fR in \fIav.c\fR.
578 .SS "What\*(Aqs Really Stored in an SV?"
579 .IX Subsection "What's Really Stored in an SV?"
580 Recall that the usual method of determining the type of scalar you have is
581 to use \f(CW\*(C`Sv*OK\*(C'\fR macros. Because a scalar can be both a number and a string,
582 usually these macros will always return TRUE and calling the \f(CW\*(C`Sv*V\*(C'\fR
583 macros will do the appropriate conversion of string to integer/double or
584 integer/double to string.
585 .PP
586 If you \fIreally\fR need to know if you have an integer, double, or string
587 pointer in an SV, you can use the following three macros instead:
588 .PP
589 .Vb 3
590 \& SvIOKp(SV*)
591 \& SvNOKp(SV*)
592 \& SvPOKp(SV*)
593 .Ve
594 .PP
595 These will tell you if you truly have an integer, double, or string pointer
596 stored in your SV. The "p" stands for private.
597 .PP
598 There are various ways in which the private and public flags may differ.
599 For example, in perl 5.16 and earlier a tied SV may have a valid
600 underlying value in the IV slot (so SvIOKp is true), but the data
601 should be accessed via the FETCH routine rather than directly,
602 so SvIOK is false. (In perl 5.18 onwards, tied scalars use
603 the flags the same way as untied scalars.) Another is when
604 numeric conversion has occurred and precision has been lost: only the
605 private flag is set on \*(Aqlossy\*(Aq values. So when an NV is converted to an
606 IV with loss, SvIOKp, SvNOKp and SvNOK will be set, while SvIOK won\*(Aqt be.
607 .PP
608 In general, though, it\*(Aqs best to use the \f(CW\*(C`Sv*V\*(C'\fR macros.
609 .SS "Read\-Only Values"
610 .IX Subsection "Read-Only Values"
611 In Perl 5.16 and earlier, copy\-on\-write (see the next section) shared a
612 flag bit with read\-only scalars. So the only way to test whether
613 \&\f(CW\*(C`sv_setsv\*(C'\fR, etc., will raise a "Modification of a read\-only value" error
614 in those versions is:
615 .PP
616 .Vb 1
617 \& SvREADONLY(sv) && !SvIsCOW(sv)
618 .Ve
619 .PP
620 Under Perl 5.18 and later, SvREADONLY only applies to read\-only variables,
621 and, under 5.20, copy\-on\-write scalars can also be read\-only, so the above
622 check is incorrect. You just want:
623 .PP
624 .Vb 1
625 \& SvREADONLY(sv)
626 .Ve
627 .PP
628 If you need to do this check often, define your own macro like this:
629 .PP
630 .Vb 5
631 \& #if PERL_VERSION >= 18
632 \& # define SvTRULYREADONLY(sv) SvREADONLY(sv)
633 \& #else
634 \& # define SvTRULYREADONLY(sv) (SvREADONLY(sv) && !SvIsCOW(sv))
635 \& #endif
636 .Ve
637 .PP
638 Or better yet, read about THINKFIRST macros below.
639 .SS "Copy on Write"
640 .IX Subsection "Copy on Write"
641 Perl implements a copy\-on\-write (COW) mechanism for scalars, in which
642 string copies are not immediately made when requested, but are deferred
643 until made necessary by one or the other scalar changing. This is mostly
644 transparent, but one must take care not to modify string buffers that are
645 shared by multiple SVs.
646 .PP
647 You can test whether an SV is using copy\-on\-write with \f(CWSvIsCOW(sv)\fR.
648 .PP
649 You can force an SV to make its own copy of its string buffer by calling
650 \&\f(CWsv_force_normal(sv)\fR or SvPV_force_nolen(sv).
651 .PP
652 If you want to make the SV drop its string buffer, use
653 \&\f(CW\*(C`sv_force_normal_flags(sv, SV_COW_DROP_PV)\*(C'\fR or simply
654 \&\f(CW\*(C`sv_setsv(sv, NULL)\*(C'\fR.
655 .PP
656 All of these functions will croak on read\-only scalars (see the previous
657 section for more on those).
658 .PP
659 To test that your code is behaving correctly and not modifying COW buffers,
660 on systems that support \fBmmap\fR\|(2) (e.g. Unix, Linux, BSDs, macOS) you can
661 configure perl with \f(CW\*(C`\-Accflags=\-DPERL_DEBUG_READONLY_COW\*(C'\fR and it will turn
662 buffer violations into crashes. You will find it to be marvellously slow,
663 so you may want to skip perl\*(Aqs own tests.
664 .SS "THINKFIRST before writing to a string buffer"
665 .IX Subsection "THINKFIRST before writing to a string buffer"
666 You are more likely to (and usually \fIshould\fR) see the \f(CWSvTHINKFIRST(sv)\fR
667 macro used to check whether an SV is ready to be written to, as this is a
668 combined single check to see if the SV:
669 .IP \(bu 4
670 Points to a COW buffer
671 .IP \(bu 4
672 Is READONLY
673 .IP \(bu 4
674 Contains a reference
675 .IP \(bu 4
676 Has some relevant magic assigned
677 .PP
678 Two related macros can be used to perform those checks and, if required,
679 also perform some action:
680 .IP \(bu 4
681 \&\f(CWSV_CHECK_THINKFIRST(sv)\fR calls \f(CWsv_force_normal(sv)\fR to copy the
682 string buffer, turning it into a mutable string.
683 .IP \(bu 4
684 \&\f(CWSV_CHECK_THINKFIRST_COW_DROP(sv)\fR calls \f(CW\*(C`sv_force_normal_flags(sv, SV_COW_DROP_PV)\*(C'\fR
685 which drops any string buffer pointed to by \f(CW\*(C`sv\*(C'\fR. This is usually
686 used to avoid copying any COW string to a new buffer, prior to
687 writing some completely new string to \f(CW\*(C`sv\*(C'\fR.
688 .SS "Working with AVs"
689 .IX Subsection "Working with AVs"
690 There are two main, longstanding ways to create and load an AV. The first
691 method creates an empty AV:
692 .PP
693 .Vb 1
694 \& AV* newAV();
695 .Ve
696 .PP
697 The second method both creates the AV and initially populates it with SVs:
698 .PP
699 .Vb 1
700 \& AV* av_make(SSize_t num, SV **ptr);
701 .Ve
702 .PP
703 The second argument points to an array containing \f(CW\*(C`num\*(C'\fR \f(CW\*(C`SV*\*(C'\fR\*(Aqs. Once the
704 AV has been created, the SVs can be destroyed, if so desired.
705 .PP
706 Perl v5.36 added two new ways to create an AV and allocate a SV** array
707 without populating it. These are more efficient than a \fBnewAV()\fR followed by an
708 \&\fBav_extend()\fR.
709 .PP
710 .Vb 4
711 \& /* Creates but does not initialize (Zero) the SV** array */
712 \& AV *av = newAV_alloc_x(1);
713 \& /* Creates and does initialize (Zero) the SV** array */
714 \& AV *av = newAV_alloc_xz(1);
715 .Ve
716 .PP
717 The numerical argument refers to the number of array elements to allocate, not
718 an array index, and must be >0. The first form must only ever be used when all
719 elements will be initialized before any read occurs. Reading a non\-initialized
720 SV* \- i.e. treating a random memory address as a SV* \- is a serious bug.
721 .PP
722 Once the AV has been created, the following operations are possible on it:
723 .PP
724 .Vb 4
725 \& void av_push(AV*, SV*);
726 \& SV* av_pop(AV*);
727 \& SV* av_shift(AV*);
728 \& void av_unshift(AV*, SSize_t num);
729 .Ve
730 .PP
731 These should be familiar operations, with the exception of \f(CW\*(C`av_unshift\*(C'\fR.
732 This routine adds \f(CW\*(C`num\*(C'\fR elements at the front of the array with the \f(CW\*(C`undef\*(C'\fR
733 value. You must then use \f(CW\*(C`av_store\*(C'\fR (described below) to assign values
734 to these new elements.
735 .PP
736 Here are some other functions:
737 .PP
738 .Vb 4
739 \& Size_t av_count(AV*);
740 \& SSize_t av_top_index(AV*);
741 \& SV** av_fetch(AV*, SSize_t key, I32 lval);
742 \& SV** av_store(AV*, SSize_t key, SV* val);
743 .Ve
744 .PP
745 \&\f(CW\*(C`av_count\*(C'\fR returns the number of elements in the array (including
746 any empty slots (undefined ones) that are intermixed with filled\-in ones).
747 The \f(CW\*(C`av_top_index\*(C'\fR function returns the highest index value in an array (just
748 like $#array in Perl). If the array is empty, \-1 is returned. It is
749 always equal to \f(CW\*(C`av_count()\ \-\ 1\*(C'\fR. The
750 \&\f(CW\*(C`av_fetch\*(C'\fR function returns the value at index \f(CW\*(C`key\*(C'\fR, but if \f(CW\*(C`lval\*(C'\fR
751 is non\-zero, then \f(CW\*(C`av_fetch\*(C'\fR will store an undef value at that index.
752 The \f(CW\*(C`av_store\*(C'\fR function stores the value \f(CW\*(C`val\*(C'\fR at index \f(CW\*(C`key\*(C'\fR, and does
753 not increment the reference count of \f(CW\*(C`val\*(C'\fR. Thus the caller is responsible
754 for taking care of that, and if \f(CW\*(C`av_store\*(C'\fR returns NULL, the caller will
755 have to decrement the reference count to avoid a memory leak. Note that
756 \&\f(CW\*(C`av_fetch\*(C'\fR and \f(CW\*(C`av_store\*(C'\fR both return \f(CW\*(C`SV**\*(C'\fR\*(Aqs, not \f(CW\*(C`SV*\*(C'\fR\*(Aqs as their
757 return value.
758 .PP
759 A few more:
760 .PP
761 .Vb 3
762 \& void av_clear(AV*);
763 \& void av_undef(AV*);
764 \& void av_extend(AV*, SSize_t key);
765 .Ve
766 .PP
767 The \f(CW\*(C`av_clear\*(C'\fR function deletes all the elements in the AV* array, but
768 does not actually delete the array itself. The \f(CW\*(C`av_undef\*(C'\fR function will
769 delete all the elements in the array plus the array itself. The
770 \&\f(CW\*(C`av_extend\*(C'\fR function extends the array so that it contains at least \f(CW\*(C`key+1\*(C'\fR
771 elements. If \f(CW\*(C`key+1\*(C'\fR is less than the currently allocated length of the array,
772 then nothing is done.
773 .PP
774 If you know the name of an array variable, you can get a pointer to its AV
775 by using the following:
776 .PP
777 .Vb 1
778 \& AV* get_av("package::varname", 0);
779 .Ve
780 .PP
781 This returns NULL if the variable does not exist.
782 .PP
783 See "Understanding the Magic of Tied Hashes and Arrays" for more
784 information on how to use the array access functions on tied arrays.
785 .PP
786 \fIMore efficient working with new or vanilla AVs\fR
787 .IX Subsection "More efficient working with new or vanilla AVs"
788 .PP
789 Perl v5.36 and v5.38 introduced streamlined, inlined versions of some
790 functions:
791 .IP \(bu 4
792 \&\f(CW\*(C`av_store_simple\*(C'\fR
793 .IP \(bu 4
794 \&\f(CW\*(C`av_fetch_simple\*(C'\fR
795 .IP \(bu 4
796 \&\f(CW\*(C`av_push_simple\*(C'\fR
797 .PP
798 These are drop\-in replacements, but can only be used on straightforward
799 AVs that meet the following criteria:
800 .IP \(bu 4
801 are not magical
802 .IP \(bu 4
803 are not readonly
804 .IP \(bu 4
805 are "real" AVs (see next section)
806 .IP \(bu 4
807 have an av_top_index value > \-2
808 .PP
809 AVs created using \f(CWnewAV()\fR, \f(CW\*(C`av_make\*(C'\fR, \f(CW\*(C`newAV_alloc_x\*(C'\fR, and
810 \&\f(CW\*(C`newAV_alloc_xz\*(C'\fR are all compatible at the time of creation. It is
811 only if they are declared readonly or unreal, have magic attached, or
812 are otherwise configured unusually that they will stop being compatible.
813 .PP
814 Note that some interpreter functions may attach magic to an AV as part
815 of normal operations. It is therefore safest, unless you are sure of the
816 lifecycle of an AV, to only use these new functions close to the point
817 of AV creation.
818 .PP
819 \fIReal AVs \- and those that are not\fR
820 .IX Subsection "Real AVs - and those that are not"
821 .PP
822 The standard or typical AV is sometime referred to as being a "real" AV,
823 a state that can be checked for with the \f(CW\*(C`AvREAL\*(C'\fR macro. What this state
824 basically signifies is that:
825 .IP \(bu 4
826 Every accessible element in the array is either initialized but
827 empty or contains a pointer to a live SV.
828 .IP \(bu 4
829 When a SV* is assigned to an array element, the reference count on
830 the SV is incremented. Conversely, when the element is unset, or assigned
831 a different SV*, the reference count of the expelled SV is decremented.
832 .PP
833 "Fake" AVs are intended for specific, limited use cases only. Use outside
834 of core perl is strongly discouraged.
835 .PP
836 For example, the interpreter\*(Aqs argument stack is implemented as an AV,
837 but it is not \f(CW\*(C`AvREAL\*(C'\fR. Its elements were historically not reference
838 counted, which gives rise to the "stack\-not\-refcounted" class of bugs
839 when SVs are freed during the execution of a statement but still needed
840 later in that statement. Efforts are currently underway to move to a
841 refcounted stack to fix this class of bugs: see the "Reference\-counted argument stack"
842 section in this document.
843 .PP
844 A "fake" AV can be converted into a "real" AV by \f(CW\*(C`av_reify\*(C'\fR. Conversion
845 must occur if the AV has the \f(CW\*(C`SVpav_REIFY\*(C'\fR flag set, which can be checked
846 for using the \f(CWAvREIFY()\fR macro. Trying to store a SV* into, or delete
847 from, an AV usually involves an \f(CWAvREIFY()\fR check and conversion if needed.
848 .PP
849 See comments in \fIav.h\fR and the functions in \fIav.c\fR if you really
850 want to know more.
851 .SS "Working with HVs"
852 .IX Subsection "Working with HVs"
853 To create an HV, you use the following routine:
854 .PP
855 .Vb 1
856 \& HV* newHV();
857 .Ve
858 .PP
859 Once the HV has been created, the following operations are possible on it:
860 .PP
861 .Vb 2
862 \& SV** hv_store(HV*, const char* key, U32 klen, SV* val, U32 hash);
863 \& SV** hv_fetch(HV*, const char* key, U32 klen, I32 lval);
864 .Ve
865 .PP
866 The \f(CW\*(C`klen\*(C'\fR parameter is the length of the key being passed in (Note that
867 you cannot pass 0 in as a value of \f(CW\*(C`klen\*(C'\fR to tell Perl to measure the
868 length of the key). The \f(CW\*(C`val\*(C'\fR argument contains the SV pointer to the
869 scalar being stored, and \f(CW\*(C`hash\*(C'\fR is the precomputed hash value (zero if
870 you want \f(CW\*(C`hv_store\*(C'\fR to calculate it for you). The \f(CW\*(C`lval\*(C'\fR parameter
871 indicates whether this fetch is actually a part of a store operation, in
872 which case a new undefined value will be added to the HV with the supplied
873 key and \f(CW\*(C`hv_fetch\*(C'\fR will return as if the value had already existed.
874 .PP
875 Remember that \f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_fetch\*(C'\fR return \f(CW\*(C`SV**\*(C'\fR\*(Aqs and not just
876 \&\f(CW\*(C`SV*\*(C'\fR. To access the scalar value, you must first dereference the return
877 value. However, you should check to make sure that the return value is
878 not NULL before dereferencing it.
879 .PP
880 The first of these two functions checks if a hash table entry exists, and the
881 second deletes it.
882 .PP
883 .Vb 2
884 \& bool hv_exists(HV*, const char* key, U32 klen);
885 \& SV* hv_delete(HV*, const char* key, U32 klen, I32 flags);
886 .Ve
887 .PP
888 If \f(CW\*(C`flags\*(C'\fR does not include the \f(CW\*(C`G_DISCARD\*(C'\fR flag then \f(CW\*(C`hv_delete\*(C'\fR will
889 create and return a mortal copy of the deleted value.
890 .PP
891 And more miscellaneous functions:
892 .PP
893 .Vb 2
894 \& void hv_clear(HV*);
895 \& void hv_undef(HV*);
896 .Ve
897 .PP
898 Like their AV counterparts, \f(CW\*(C`hv_clear\*(C'\fR deletes all the entries in the hash
899 table but does not actually delete the hash table. The \f(CW\*(C`hv_undef\*(C'\fR deletes
900 both the entries and the hash table itself.
901 .PP
902 Perl keeps the actual data in a linked list of structures with a typedef of HE.
903 These contain the actual key and value pointers (plus extra administrative
904 overhead). The key is a string pointer; the value is an \f(CW\*(C`SV*\*(C'\fR. However,
905 once you have an \f(CW\*(C`HE*\*(C'\fR, to get the actual key and value, use the routines
906 specified below.
907 .PP
908 .Vb 10
909 \& I32 hv_iterinit(HV*);
910 \& /* Prepares starting point to traverse hash table */
911 \& HE* hv_iternext(HV*);
912 \& /* Get the next entry, and return a pointer to a
913 \& structure that has both the key and value */
914 \& char* hv_iterkey(HE* entry, I32* retlen);
915 \& /* Get the key from an HE structure and also return
916 \& the length of the key string */
917 \& SV* hv_iterval(HV*, HE* entry);
918 \& /* Return an SV pointer to the value of the HE
919 \& structure */
920 \& SV* hv_iternextsv(HV*, char** key, I32* retlen);
921 \& /* This convenience routine combines hv_iternext,
922 \& hv_iterkey, and hv_iterval. The key and retlen
923 \& arguments are return values for the key and its
924 \& length. The value is returned in the SV* argument */
925 .Ve
926 .PP
927 If you know the name of a hash variable, you can get a pointer to its HV
928 by using the following:
929 .PP
930 .Vb 1
931 \& HV* get_hv("package::varname", 0);
932 .Ve
933 .PP
934 This returns NULL if the variable does not exist.
935 .PP
936 The hash algorithm is defined in the \f(CW\*(C`PERL_HASH\*(C'\fR macro:
937 .PP
938 .Vb 1
939 \& PERL_HASH(hash, key, klen)
940 .Ve
941 .PP
942 The exact implementation of this macro varies by architecture and version
943 of perl, and the return value may change per invocation, so the value
944 is only valid for the duration of a single perl process.
945 .PP
946 See "Understanding the Magic of Tied Hashes and Arrays" for more
947 information on how to use the hash access functions on tied hashes.
948 .SS "Hash API Extensions"
949 .IX Subsection "Hash API Extensions"
950 Beginning with version 5.004, the following functions are also supported:
951 .PP
952 .Vb 2
953 \& HE* hv_fetch_ent (HV* tb, SV* key, I32 lval, U32 hash);
954 \& HE* hv_store_ent (HV* tb, SV* key, SV* val, U32 hash);
955 \&
956 \& bool hv_exists_ent (HV* tb, SV* key, U32 hash);
957 \& SV* hv_delete_ent (HV* tb, SV* key, I32 flags, U32 hash);
958 \&
959 \& SV* hv_iterkeysv (HE* entry);
960 .Ve
961 .PP
962 Note that these functions take \f(CW\*(C`SV*\*(C'\fR keys, which simplifies writing
963 of extension code that deals with hash structures. These functions
964 also allow passing of \f(CW\*(C`SV*\*(C'\fR keys to \f(CW\*(C`tie\*(C'\fR functions without forcing
965 you to stringify the keys (unlike the previous set of functions).
966 .PP
967 They also return and accept whole hash entries (\f(CW\*(C`HE*\*(C'\fR), making their
968 use more efficient (since the hash number for a particular string
969 doesn\*(Aqt have to be recomputed every time). See perlapi for detailed
970 descriptions.
971 .PP
972 The following macros must always be used to access the contents of hash
973 entries. Note that the arguments to these macros must be simple
974 variables, since they may get evaluated more than once. See
975 perlapi for detailed descriptions of these macros.
976 .PP
977 .Vb 6
978 \& HePV(HE* he, STRLEN len)
979 \& HeVAL(HE* he)
980 \& HeHASH(HE* he)
981 \& HeSVKEY(HE* he)
982 \& HeSVKEY_force(HE* he)
983 \& HeSVKEY_set(HE* he, SV* sv)
984 .Ve
985 .PP
986 These two lower level macros are defined, but must only be used when
987 dealing with keys that are not \f(CW\*(C`SV*\*(C'\fRs:
988 .PP
989 .Vb 2
990 \& HeKEY(HE* he)
991 \& HeKLEN(HE* he)
992 .Ve
993 .PP
994 Note that both \f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_store_ent\*(C'\fR do not increment the
995 reference count of the stored \f(CW\*(C`val\*(C'\fR, which is the caller\*(Aqs responsibility.
996 If these functions return a NULL value, the caller will usually have to
997 decrement the reference count of \f(CW\*(C`val\*(C'\fR to avoid a memory leak.
998 .SS "AVs, HVs and undefined values"
999 .IX Subsection "AVs, HVs and undefined values"
1000 Sometimes you have to store undefined values in AVs or HVs. Although
1001 this may be a rare case, it can be tricky. That\*(Aqs because you\*(Aqre
1002 used to using \f(CW&PL_sv_undef\fR if you need an undefined SV.
1003 .PP
1004 For example, intuition tells you that this XS code:
1005 .PP
1006 .Vb 2
1007 \& AV *av = newAV();
1008 \& av_store( av, 0, &PL_sv_undef );
1009 .Ve
1010 .PP
1011 is equivalent to this Perl code:
1012 .PP
1013 .Vb 2
1014 \& my @av;
1015 \& $av[0] = undef;
1016 .Ve
1017 .PP
1018 Unfortunately, this isn\*(Aqt true. In perl 5.18 and earlier, AVs use \f(CW&PL_sv_undef\fR as a marker
1019 for indicating that an array element has not yet been initialized.
1020 Thus, \f(CW\*(C`exists $av[0]\*(C'\fR would be true for the above Perl code, but
1021 false for the array generated by the XS code. In perl 5.20, storing
1022 &PL_sv_undef will create a read\-only element, because the scalar
1023 &PL_sv_undef itself is stored, not a copy.
1024 .PP
1025 Similar problems can occur when storing \f(CW&PL_sv_undef\fR in HVs:
1026 .PP
1027 .Vb 1
1028 \& hv_store( hv, "key", 3, &PL_sv_undef, 0 );
1029 .Ve
1030 .PP
1031 This will indeed make the value \f(CW\*(C`undef\*(C'\fR, but if you try to modify
1032 the value of \f(CW\*(C`key\*(C'\fR, you\*(Aqll get the following error:
1033 .PP
1034 .Vb 1
1035 \& Modification of non\-creatable hash value attempted
1036 .Ve
1037 .PP
1038 In perl 5.8.0, \f(CW&PL_sv_undef\fR was also used to mark placeholders
1039 in restricted hashes. This caused such hash entries not to appear
1040 when iterating over the hash or when checking for the keys
1041 with the \f(CW\*(C`hv_exists\*(C'\fR function.
1042 .PP
1043 You can run into similar problems when you store \f(CW&PL_sv_yes\fR or
1044 \&\f(CW&PL_sv_no\fR into AVs or HVs. Trying to modify such elements
1045 will give you the following error:
1046 .PP
1047 .Vb 1
1048 \& Modification of a read\-only value attempted
1049 .Ve
1050 .PP
1051 To make a long story short, you can use the special variables
1052 \&\f(CW&PL_sv_undef\fR, \f(CW&PL_sv_yes\fR and \f(CW&PL_sv_no\fR with AVs and
1053 HVs, but you have to make sure you know what you\*(Aqre doing.
1054 .PP
1055 Generally, if you want to store an undefined value in an AV
1056 or HV, you should not use \f(CW&PL_sv_undef\fR, but rather create a
1057 new undefined value using the \f(CW\*(C`newSV\*(C'\fR function, for example:
1058 .PP
1059 .Vb 2
1060 \& av_store( av, 42, newSV(0) );
1061 \& hv_store( hv, "foo", 3, newSV(0), 0 );
1062 .Ve
1063 .SS References
1064 .IX Subsection "References"
1065 References are a special type of scalar that point to other data types
1066 (including other references).
1067 .PP
1068 To create a reference, use either of the following functions:
1069 .PP
1070 .Vb 2
1071 \& SV* newRV_inc((SV*) thing);
1072 \& SV* newRV_noinc((SV*) thing);
1073 .Ve
1074 .PP
1075 The \f(CW\*(C`thing\*(C'\fR argument can be any of an \f(CW\*(C`SV*\*(C'\fR, \f(CW\*(C`AV*\*(C'\fR, or \f(CW\*(C`HV*\*(C'\fR. The
1076 functions are identical except that \f(CW\*(C`newRV_inc\*(C'\fR increments the reference
1077 count of the \f(CW\*(C`thing\*(C'\fR, while \f(CW\*(C`newRV_noinc\*(C'\fR does not. For historical
1078 reasons, \f(CW\*(C`newRV\*(C'\fR is a synonym for \f(CW\*(C`newRV_inc\*(C'\fR.
1079 .PP
1080 Once you have a reference, you can use the following macro to dereference
1081 the reference:
1082 .PP
1083 .Vb 1
1084 \& SvRV(SV*)
1085 .Ve
1086 .PP
1087 then call the appropriate routines, casting the returned \f(CW\*(C`SV*\*(C'\fR to either an
1088 \&\f(CW\*(C`AV*\*(C'\fR or \f(CW\*(C`HV*\*(C'\fR, if required.
1089 .PP
1090 To determine if an SV is a reference, you can use the following macro:
1091 .PP
1092 .Vb 1
1093 \& SvROK(SV*)
1094 .Ve
1095 .PP
1096 To discover what type of value the reference refers to, use the following
1097 macro and then check the return value.
1098 .PP
1099 .Vb 1
1100 \& SvTYPE(SvRV(SV*))
1101 .Ve
1102 .PP
1103 The most useful types that will be returned are:
1104 .PP
1105 .Vb 4
1106 \& SVt_PVAV Array
1107 \& SVt_PVHV Hash
1108 \& SVt_PVCV Code
1109 \& SVt_PVGV Glob (possibly a file handle)
1110 .Ve
1111 .PP
1112 Any numerical value returned which is less than SVt_PVAV will be a scalar
1113 of some form.
1114 .PP
1115 See "svtype" in perlapi for more details.
1116 .SS "Blessed References and Class Objects"
1117 .IX Subsection "Blessed References and Class Objects"
1118 References are also used to support object\-oriented programming. In perl\*(Aqs
1119 OO lexicon, an object is simply a reference that has been blessed into a
1120 package (or class). Once blessed, the programmer may now use the reference
1121 to access the various methods in the class.
1122 .PP
1123 A reference can be blessed into a package with the following function:
1124 .PP
1125 .Vb 1
1126 \& SV* sv_bless(SV* sv, HV* stash);
1127 .Ve
1128 .PP
1129 The \f(CW\*(C`sv\*(C'\fR argument must be a reference value. The \f(CW\*(C`stash\*(C'\fR argument
1130 specifies which class the reference will belong to. See
1131 "Stashes and Globs" for information on converting class names into stashes.
1132 .PP
1133 /* Still under construction */
1134 .PP
1135 The following function upgrades rv to reference if not already one.
1136 Creates a new SV for rv to point to. If \f(CW\*(C`classname\*(C'\fR is non\-null, the SV
1137 is blessed into the specified class. SV is returned.
1138 .PP
1139 .Vb 1
1140 \& SV* newSVrv(SV* rv, const char* classname);
1141 .Ve
1142 .PP
1143 The following three functions copy integer, unsigned integer or double
1144 into an SV whose reference is \f(CW\*(C`rv\*(C'\fR. SV is blessed if \f(CW\*(C`classname\*(C'\fR is
1145 non\-null.
1146 .PP
1147 .Vb 3
1148 \& SV* sv_setref_iv(SV* rv, const char* classname, IV iv);
1149 \& SV* sv_setref_uv(SV* rv, const char* classname, UV uv);
1150 \& SV* sv_setref_nv(SV* rv, const char* classname, NV iv);
1151 .Ve
1152 .PP
1153 The following function copies the pointer value (\fIthe address, not the
1154 string!\fR) into an SV whose reference is rv. SV is blessed if \f(CW\*(C`classname\*(C'\fR
1155 is non\-null.
1156 .PP
1157 .Vb 1
1158 \& SV* sv_setref_pv(SV* rv, const char* classname, void* pv);
1159 .Ve
1160 .PP
1161 The following function copies a string into an SV whose reference is \f(CW\*(C`rv\*(C'\fR.
1162 Set length to 0 to let Perl calculate the string length. SV is blessed if
1163 \&\f(CW\*(C`classname\*(C'\fR is non\-null.
1164 .PP
1165 .Vb 2
1166 \& SV* sv_setref_pvn(SV* rv, const char* classname, char* pv,
1167 \& STRLEN length);
1168 .Ve
1169 .PP
1170 The following function tests whether the SV is blessed into the specified
1171 class. It does not check inheritance relationships.
1172 .PP
1173 .Vb 1
1174 \& int sv_isa(SV* sv, const char* name);
1175 .Ve
1176 .PP
1177 The following function tests whether the SV is a reference to a blessed object.
1178 .PP
1179 .Vb 1
1180 \& int sv_isobject(SV* sv);
1181 .Ve
1182 .PP
1183 The following function tests whether the SV is derived from the specified
1184 class. SV can be either a reference to a blessed object or a string
1185 containing a class name. This is the function implementing the
1186 \&\f(CW\*(C`UNIVERSAL::isa\*(C'\fR functionality.
1187 .PP
1188 .Vb 1
1189 \& bool sv_derived_from(SV* sv, const char* name);
1190 .Ve
1191 .PP
1192 To check if you\*(Aqve got an object derived from a specific class you have
1193 to write:
1194 .PP
1195 .Vb 1
1196 \& if (sv_isobject(sv) && sv_derived_from(sv, class)) { ... }
1197 .Ve
1198 .SS "Creating New Variables"
1199 .IX Subsection "Creating New Variables"
1200 To create a new Perl variable with an undef value which can be accessed from
1201 your Perl script, use the following routines, depending on the variable type.
1202 .PP
1203 .Vb 3
1204 \& SV* get_sv("package::varname", GV_ADD);
1205 \& AV* get_av("package::varname", GV_ADD);
1206 \& HV* get_hv("package::varname", GV_ADD);
1207 .Ve
1208 .PP
1209 Notice the use of GV_ADD as the second parameter. The new variable can now
1210 be set, using the routines appropriate to the data type.
1211 .PP
1212 There are additional macros whose values may be bitwise OR\*(Aqed with the
1213 \&\f(CW\*(C`GV_ADD\*(C'\fR argument to enable certain extra features. Those bits are:
1214 .IP GV_ADDMULTI 4
1215 .IX Item "GV_ADDMULTI"
1216 Marks the variable as multiply defined, thus preventing the:
1217 .Sp
1218 .Vb 1
1219 \& Name <varname> used only once: possible typo
1220 .Ve
1221 .Sp
1222 warning.
1223 .IP GV_ADDWARN 4
1224 .IX Item "GV_ADDWARN"
1225 Issues the warning:
1226 .Sp
1227 .Vb 1
1228 \& Had to create <varname> unexpectedly
1229 .Ve
1230 .Sp
1231 if the variable did not exist before the function was called.
1232 .PP
1233 If you do not specify a package name, the variable is created in the current
1234 package.
1235 .SS "Reference Counts and Mortality"
1236 .IX Subsection "Reference Counts and Mortality"
1237 Perl uses a reference count\-driven garbage collection mechanism. SVs,
1238 AVs, or HVs (xV for short in the following) start their life with a
1239 reference count of 1. If the reference count of an xV ever drops to 0,
1240 then it will be destroyed and its memory made available for reuse.
1241 At the most basic internal level, reference counts can be manipulated
1242 with the following macros:
1243 .PP
1244 .Vb 3
1245 \& int SvREFCNT(SV* sv);
1246 \& SV* SvREFCNT_inc(SV* sv);
1247 \& void SvREFCNT_dec(SV* sv);
1248 .Ve
1249 .PP
1250 (There are also suffixed versions of the increment and decrement macros,
1251 for situations where the full generality of these basic macros can be
1252 exchanged for some performance.)
1253 .PP
1254 However, the way a programmer should think about references is not so
1255 much in terms of the bare reference count, but in terms of \fIownership\fR
1256 of references. A reference to an xV can be owned by any of a variety
1257 of entities: another xV, the Perl interpreter, an XS data structure,
1258 a piece of running code, or a dynamic scope. An xV generally does not
1259 know what entities own the references to it; it only knows how many
1260 references there are, which is the reference count.
1261 .PP
1262 To correctly maintain reference counts, it is essential to keep track
1263 of what references the XS code is manipulating. The programmer should
1264 always know where a reference has come from and who owns it, and be
1265 aware of any creation or destruction of references, and any transfers
1266 of ownership. Because ownership isn\*(Aqt represented explicitly in the xV
1267 data structures, only the reference count need be actually maintained
1268 by the code, and that means that this understanding of ownership is not
1269 actually evident in the code. For example, transferring ownership of a
1270 reference from one owner to another doesn\*(Aqt change the reference count
1271 at all, so may be achieved with no actual code. (The transferring code
1272 doesn\*(Aqt touch the referenced object, but does need to ensure that the
1273 former owner knows that it no longer owns the reference, and that the
1274 new owner knows that it now does.)
1275 .PP
1276 An xV that is visible at the Perl level should not become unreferenced
1277 and thus be destroyed. Normally, an object will only become unreferenced
1278 when it is no longer visible, often by the same means that makes it
1279 invisible. For example, a Perl reference value (RV) owns a reference to
1280 its referent, so if the RV is overwritten that reference gets destroyed,
1281 and the no\-longer\-reachable referent may be destroyed as a result.
1282 .PP
1283 Many functions have some kind of reference manipulation as
1284 part of their purpose. Sometimes this is documented in terms
1285 of ownership of references, and sometimes it is (less helpfully)
1286 documented in terms of changes to reference counts. For example, the
1287 \&\fBnewRV_inc()\fR function is documented to create a new RV
1288 (with reference count 1) and increment the reference count of the referent
1289 that was supplied by the caller. This is best understood as creating
1290 a new reference to the referent, which is owned by the created RV,
1291 and returning to the caller ownership of the sole reference to the RV.
1292 The \fBnewRV_noinc()\fR function instead does not
1293 increment the reference count of the referent, but the RV nevertheless
1294 ends up owning a reference to the referent. It is therefore implied
1295 that the caller of \f(CWnewRV_noinc()\fR is relinquishing a reference to the
1296 referent, making this conceptually a more complicated operation even
1297 though it does less to the data structures.
1298 .PP
1299 For example, imagine you want to return a reference from an XSUB
1300 function. Inside the XSUB routine, you create an SV which initially
1301 has just a single reference, owned by the XSUB routine. This reference
1302 needs to be disposed of before the routine is complete, otherwise it
1303 will leak, preventing the SV from ever being destroyed. So to create
1304 an RV referencing the SV, it is most convenient to pass the SV to
1305 \&\f(CWnewRV_noinc()\fR, which consumes that reference. Now the XSUB routine
1306 no longer owns a reference to the SV, but does own a reference to the RV,
1307 which in turn owns a reference to the SV. The ownership of the reference
1308 to the RV is then transferred by the process of returning the RV from
1309 the XSUB.
1310 .PP
1311 There are some convenience functions available that can help with the
1312 destruction of xVs. These functions introduce the concept of "mortality".
1313 Much documentation speaks of an xV itself being mortal, but this is
1314 misleading. It is really \fIa reference to\fR an xV that is mortal, and it
1315 is possible for there to be more than one mortal reference to a single xV.
1316 For a reference to be mortal means that it is owned by the temps stack,
1317 one of perl\*(Aqs many internal stacks, which will destroy that reference
1318 "a short time later". Usually the "short time later" is the end of
1319 the current Perl statement. However, it gets more complicated around
1320 dynamic scopes: there can be multiple sets of mortal references hanging
1321 around at the same time, with different death dates. Internally, the
1322 actual determinant for when mortal xV references are destroyed depends
1323 on two macros, SAVETMPS and FREETMPS. See perlcall and perlxs
1324 and "Temporaries Stack" below for more details on these macros.
1325 .PP
1326 Mortal references are mainly used for xVs that are placed on perl\*(Aqs
1327 main stack. The stack is problematic for reference tracking, because it
1328 contains a lot of xV references, but doesn\*(Aqt own those references: they
1329 are not counted. Currently, there are many bugs resulting from xVs being
1330 destroyed while referenced by the stack, because the stack\*(Aqs uncounted
1331 references aren\*(Aqt enough to keep the xVs alive. So when putting an
1332 (uncounted) reference on the stack, it is vitally important to ensure that
1333 there will be a counted reference to the same xV that will last at least
1334 as long as the uncounted reference. But it\*(Aqs also important that that
1335 counted reference be cleaned up at an appropriate time, and not unduly
1336 prolong the xV\*(Aqs life. For there to be a mortal reference is often the
1337 best way to satisfy this requirement, especially if the xV was created
1338 especially to be put on the stack and would otherwise be unreferenced.
1339 .PP
1340 To create a mortal reference, use the functions:
1341 .PP
1342 .Vb 3
1343 \& SV* sv_newmortal()
1344 \& SV* sv_mortalcopy(SV*)
1345 \& SV* sv_2mortal(SV*)
1346 .Ve
1347 .PP
1348 \&\f(CWsv_newmortal()\fR creates an SV (with the undefined value) whose sole
1349 reference is mortal. \f(CWsv_mortalcopy()\fR creates an xV whose value is a
1350 copy of a supplied xV and whose sole reference is mortal. \f(CWsv_2mortal()\fR
1351 mortalises an existing xV reference: it transfers ownership of a reference
1352 from the caller to the temps stack. Because \f(CW\*(C`sv_newmortal\*(C'\fR gives the new
1353 SV no value, it must normally be given one via \f(CW\*(C`sv_setpv\*(C'\fR, \f(CW\*(C`sv_setiv\*(C'\fR,
1354 etc. :
1355 .PP
1356 .Vb 2
1357 \& SV *tmp = sv_newmortal();
1358 \& sv_setiv(tmp, an_integer);
1359 .Ve
1360 .PP
1361 As that is multiple C statements it is quite common so see this idiom instead:
1362 .PP
1363 .Vb 1
1364 \& SV *tmp = sv_2mortal(newSViv(an_integer));
1365 .Ve
1366 .PP
1367 The mortal routines are not just for SVs; AVs and HVs can be
1368 made mortal by passing their address (type\-casted to \f(CW\*(C`SV*\*(C'\fR) to the
1369 \&\f(CW\*(C`sv_2mortal\*(C'\fR or \f(CW\*(C`sv_mortalcopy\*(C'\fR routines.
1370 .SS "Stashes and Globs"
1371 .IX Subsection "Stashes and Globs"
1372 A \fBstash\fR is a hash that contains all variables that are defined
1373 within a package. Each key of the stash is a symbol
1374 name (shared by all the different types of objects that have the same
1375 name), and each value in the hash table is a GV (Glob Value). This GV
1376 in turn contains references to the various objects of that name,
1377 including (but not limited to) the following:
1378 .PP
1379 .Vb 6
1380 \& Scalar Value
1381 \& Array Value
1382 \& Hash Value
1383 \& I/O Handle
1384 \& Format
1385 \& Subroutine
1386 .Ve
1387 .PP
1388 There is a single stash called \f(CW\*(C`PL_defstash\*(C'\fR that holds the items that exist
1389 in the \f(CW\*(C`main\*(C'\fR package. To get at the items in other packages, append the
1390 string "::" to the package name. The items in the \f(CW\*(C`Foo\*(C'\fR package are in
1391 the stash \f(CW\*(C`Foo::\*(C'\fR in PL_defstash. The items in the \f(CW\*(C`Bar::Baz\*(C'\fR package are
1392 in the stash \f(CW\*(C`Baz::\*(C'\fR in \f(CW\*(C`Bar::\*(C'\fR\*(Aqs stash.
1393 .PP
1394 To get the stash pointer for a particular package, use the function:
1395 .PP
1396 .Vb 2
1397 \& HV* gv_stashpv(const char* name, I32 flags)
1398 \& HV* gv_stashsv(SV*, I32 flags)
1399 .Ve
1400 .PP
1401 The first function takes a literal string, the second uses the string stored
1402 in the SV. Remember that a stash is just a hash table, so you get back an
1403 \&\f(CW\*(C`HV*\*(C'\fR. The \f(CW\*(C`flags\*(C'\fR flag will create a new package if it is set to GV_ADD.
1404 .PP
1405 The name that \f(CW\*(C`gv_stash*v\*(C'\fR wants is the name of the package whose symbol table
1406 you want. The default package is called \f(CW\*(C`main\*(C'\fR. If you have multiply nested
1407 packages, pass their names to \f(CW\*(C`gv_stash*v\*(C'\fR, separated by \f(CW\*(C`::\*(C'\fR as in the Perl
1408 language itself.
1409 .PP
1410 Alternately, if you have an SV that is a blessed reference, you can find
1411 out the stash pointer by using:
1412 .PP
1413 .Vb 1
1414 \& HV* SvSTASH(SvRV(SV*));
1415 .Ve
1416 .PP
1417 then use the following to get the package name itself:
1418 .PP
1419 .Vb 1
1420 \& char* HvNAME(HV* stash);
1421 .Ve
1422 .PP
1423 If you need to bless or re\-bless an object you can use the following
1424 function:
1425 .PP
1426 .Vb 1
1427 \& SV* sv_bless(SV*, HV* stash)
1428 .Ve
1429 .PP
1430 where the first argument, an \f(CW\*(C`SV*\*(C'\fR, must be a reference, and the second
1431 argument is a stash. The returned \f(CW\*(C`SV*\*(C'\fR can now be used in the same way
1432 as any other SV.
1433 .PP
1434 For more information on references and blessings, consult perlref.
1435 .SS "I/O Handles"
1436 .IX Subsection "I/O Handles"
1437 Like AVs and HVs, IO objects are another type of non\-scalar SV which
1438 may contain input and output PerlIO objects or a \f(CW\*(C`DIR *\*(C'\fR
1439 from \fBopendir()\fR.
1440 .PP
1441 You can create a new IO object:
1442 .PP
1443 .Vb 1
1444 \& IO* newIO();
1445 .Ve
1446 .PP
1447 Unlike other SVs, a new IO object is automatically blessed into the
1448 IO::File class.
1449 .PP
1450 The IO object contains an input and output PerlIO handle:
1451 .PP
1452 .Vb 2
1453 \& PerlIO *IoIFP(IO *io);
1454 \& PerlIO *IoOFP(IO *io);
1455 .Ve
1456 .PP
1457 Typically if the IO object has been opened on a file, the input handle
1458 is always present, but the output handle is only present if the file
1459 is open for output. For a file, if both are present they will be the
1460 same PerlIO object.
1461 .PP
1462 Distinct input and output PerlIO objects are created for sockets and
1463 character devices.
1464 .PP
1465 The IO object also contains other data associated with Perl I/O
1466 handles:
1467 .PP
1468 .Vb 12
1469 \& IV IoLINES(io); /* $. */
1470 \& IV IoPAGE(io); /* $% */
1471 \& IV IoPAGE_LEN(io); /* $= */
1472 \& IV IoLINES_LEFT(io); /* $\- */
1473 \& char *IoTOP_NAME(io); /* $^ */
1474 \& GV *IoTOP_GV(io); /* $^ */
1475 \& char *IoFMT_NAME(io); /* $~ */
1476 \& GV *IoFMT_GV(io); /* $~ */
1477 \& char *IoBOTTOM_NAME(io);
1478 \& GV *IoBOTTOM_GV(io);
1479 \& char IoTYPE(io);
1480 \& U8 IoFLAGS(io);
1481 .Ve
1482 .PP
1483 Most of these are involved with formats.
1484 .PP
1485 \&\fBIoFLAGs()\fR may contain a combination of flags, the most interesting of
1486 which are \f(CW\*(C`IOf_FLUSH\*(C'\fR (\f(CW$|\fR) for autoflush and \f(CW\*(C`IOf_UNTAINT\*(C'\fR,
1487 settable with IO::Handle\*(Aqs \fBuntaint()\fR method.
1488 .PP
1489 The IO object may also contain a directory handle:
1490 .PP
1491 .Vb 1
1492 \& DIR *IoDIRP(io);
1493 .Ve
1494 .PP
1495 suitable for use with \fBPerlDir_read()\fR etc.
1496 .PP
1497 All of these accessors macros are lvalues, there are no distinct
1498 \&\f(CW_set()\fR macros to modify the members of the IO object.
1499 .SS "Double\-Typed SVs"
1500 .IX Subsection "Double-Typed SVs"
1501 Scalar variables normally contain only one type of value, an integer,
1502 double, pointer, or reference. Perl will automatically convert the
1503 actual scalar data from the stored type into the requested type.
1504 .PP
1505 Some scalar variables contain more than one type of scalar data. For
1506 example, the variable \f(CW$!\fR contains either the numeric value of \f(CW\*(C`errno\*(C'\fR
1507 or its string equivalent from either \f(CW\*(C`strerror\*(C'\fR or \f(CW\*(C`sys_errlist[]\*(C'\fR.
1508 .PP
1509 To force multiple data values into an SV, you must do two things: use the
1510 \&\f(CW\*(C`sv_set*v\*(C'\fR routines to add the additional scalar type, then set a flag
1511 so that Perl will believe it contains more than one type of data. The
1512 four macros to set the flags are:
1513 .PP
1514 .Vb 4
1515 \& SvIOK_on
1516 \& SvNOK_on
1517 \& SvPOK_on
1518 \& SvROK_on
1519 .Ve
1520 .PP
1521 The particular macro you must use depends on which \f(CW\*(C`sv_set*v\*(C'\fR routine
1522 you called first. This is because every \f(CW\*(C`sv_set*v\*(C'\fR routine turns on
1523 only the bit for the particular type of data being set, and turns off
1524 all the rest.
1525 .PP
1526 For example, to create a new Perl variable called "dberror" that contains
1527 both the numeric and descriptive string error values, you could use the
1528 following code:
1529 .PP
1530 .Vb 2
1531 \& extern int dberror;
1532 \& extern char *dberror_list;
1533 \&
1534 \& SV* sv = get_sv("dberror", GV_ADD);
1535 \& sv_setiv(sv, (IV) dberror);
1536 \& sv_setpv(sv, dberror_list[dberror]);
1537 \& SvIOK_on(sv);
1538 .Ve
1539 .PP
1540 If the order of \f(CW\*(C`sv_setiv\*(C'\fR and \f(CW\*(C`sv_setpv\*(C'\fR had been reversed, then the
1541 macro \f(CW\*(C`SvPOK_on\*(C'\fR would need to be called instead of \f(CW\*(C`SvIOK_on\*(C'\fR.
1542 .SS "Magic Variables"
1543 .IX Subsection "Magic Variables"
1544 [This section still under construction. Ignore everything here. Post no
1545 bills. Everything not permitted is forbidden.]
1546 .PP
1547 Any SV may be magical, that is, it has special features that a normal
1548 SV does not have. These features are stored in the SV structure in a
1549 linked list of \f(CW\*(C`struct magic\*(C'\fR\*(Aqs, typedef\*(Aqed to \f(CW\*(C`MAGIC\*(C'\fR.
1550 .PP
1551 .Vb 10
1552 \& struct magic {
1553 \& MAGIC* mg_moremagic;
1554 \& MGVTBL* mg_virtual;
1555 \& U16 mg_private;
1556 \& char mg_type;
1557 \& U8 mg_flags;
1558 \& I32 mg_len;
1559 \& SV* mg_obj;
1560 \& char* mg_ptr;
1561 \& };
1562 .Ve
1563 .PP
1564 Note this is current as of patchlevel 0, and could change at any time.
1565 .SS "Assigning Magic"
1566 .IX Subsection "Assigning Magic"
1567 Perl adds magic to an SV using the sv_magic function:
1568 .PP
1569 .Vb 1
1570 \& void sv_magic(SV* sv, SV* obj, int how, const char* name, I32 namlen);
1571 .Ve
1572 .PP
1573 The \f(CW\*(C`sv\*(C'\fR argument is a pointer to the SV that is to acquire a new magical
1574 feature.
1575 .PP
1576 If \f(CW\*(C`sv\*(C'\fR is not already magical, Perl uses the \f(CW\*(C`SvUPGRADE\*(C'\fR macro to
1577 convert \f(CW\*(C`sv\*(C'\fR to type \f(CW\*(C`SVt_PVMG\*(C'\fR.
1578 Perl then continues by adding new magic
1579 to the beginning of the linked list of magical features. Any prior entry
1580 of the same type of magic is deleted. Note that this can be overridden,
1581 and multiple instances of the same type of magic can be associated with an
1582 SV.
1583 .PP
1584 The \f(CW\*(C`name\*(C'\fR and \f(CW\*(C`namlen\*(C'\fR arguments are used to associate a string with
1585 the magic, typically the name of a variable. \f(CW\*(C`namlen\*(C'\fR is stored in the
1586 \&\f(CW\*(C`mg_len\*(C'\fR field and if \f(CW\*(C`name\*(C'\fR is non\-null then either a \f(CW\*(C`savepvn\*(C'\fR copy of
1587 \&\f(CW\*(C`name\*(C'\fR or \f(CW\*(C`name\*(C'\fR itself is stored in the \f(CW\*(C`mg_ptr\*(C'\fR field, depending on
1588 whether \f(CW\*(C`namlen\*(C'\fR is greater than zero or equal to zero respectively. As a
1589 special case, if \f(CW\*(C`(name && namlen == HEf_SVKEY)\*(C'\fR then \f(CW\*(C`name\*(C'\fR is assumed
1590 to contain an \f(CW\*(C`SV*\*(C'\fR and is stored as\-is with its REFCNT incremented.
1591 .PP
1592 The sv_magic function uses \f(CW\*(C`how\*(C'\fR to determine which, if any, predefined
1593 "Magic Virtual Table" should be assigned to the \f(CW\*(C`mg_virtual\*(C'\fR field.
1594 See the "Magic Virtual Tables" section below. The \f(CW\*(C`how\*(C'\fR argument is also
1595 stored in the \f(CW\*(C`mg_type\*(C'\fR field. The value of
1596 \&\f(CW\*(C`how\*(C'\fR should be chosen from the set of macros
1597 \&\f(CW\*(C`PERL_MAGIC_foo\*(C'\fR found in \fIperl.h\fR. Note that before
1598 these macros were added, Perl internals used to directly use character
1599 literals, so you may occasionally come across old code or documentation
1600 referring to \*(AqU\*(Aq magic rather than \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR for example.
1601 .PP
1602 The \f(CW\*(C`obj\*(C'\fR argument is stored in the \f(CW\*(C`mg_obj\*(C'\fR field of the \f(CW\*(C`MAGIC\*(C'\fR
1603 structure. If it is not the same as the \f(CW\*(C`sv\*(C'\fR argument, the reference
1604 count of the \f(CW\*(C`obj\*(C'\fR object is incremented. If it is the same, or if
1605 the \f(CW\*(C`how\*(C'\fR argument is \f(CW\*(C`PERL_MAGIC_arylen\*(C'\fR, \f(CW\*(C`PERL_MAGIC_regdatum\*(C'\fR,
1606 \&\f(CW\*(C`PERL_MAGIC_regdata\*(C'\fR, or if it is a NULL pointer, then \f(CW\*(C`obj\*(C'\fR is merely
1607 stored, without the reference count being incremented.
1608 .PP
1609 See also \f(CW\*(C`sv_magicext\*(C'\fR in perlapi for a more flexible way to add magic
1610 to an SV.
1611 .PP
1612 There is also a function to add magic to an \f(CW\*(C`HV\*(C'\fR:
1613 .PP
1614 .Vb 1
1615 \& void hv_magic(HV *hv, GV *gv, int how);
1616 .Ve
1617 .PP
1618 This simply calls \f(CW\*(C`sv_magic\*(C'\fR and coerces the \f(CW\*(C`gv\*(C'\fR argument into an \f(CW\*(C`SV\*(C'\fR.
1619 .PP
1620 To remove the magic from an SV, call the function sv_unmagic:
1621 .PP
1622 .Vb 1
1623 \& int sv_unmagic(SV *sv, int type);
1624 .Ve
1625 .PP
1626 The \f(CW\*(C`type\*(C'\fR argument should be equal to the \f(CW\*(C`how\*(C'\fR value when the \f(CW\*(C`SV\*(C'\fR
1627 was initially made magical.
1628 .PP
1629 However, note that \f(CW\*(C`sv_unmagic\*(C'\fR removes all magic of a certain \f(CW\*(C`type\*(C'\fR from the
1630 \&\f(CW\*(C`SV\*(C'\fR. If you want to remove only certain
1631 magic of a \f(CW\*(C`type\*(C'\fR based on the magic
1632 virtual table, use \f(CW\*(C`sv_unmagicext\*(C'\fR instead:
1633 .PP
1634 .Vb 1
1635 \& int sv_unmagicext(SV *sv, int type, MGVTBL *vtbl);
1636 .Ve
1637 .SS "Magic Virtual Tables"
1638 .IX Subsection "Magic Virtual Tables"
1639 The \f(CW\*(C`mg_virtual\*(C'\fR field in the \f(CW\*(C`MAGIC\*(C'\fR structure is a pointer to an
1640 \&\f(CW\*(C`MGVTBL\*(C'\fR, which is a structure of function pointers and stands for
1641 "Magic Virtual Table" to handle the various operations that might be
1642 applied to that variable.
1643 .PP
1644 The \f(CW\*(C`MGVTBL\*(C'\fR has five (or sometimes eight) pointers to the following
1645 routine types:
1646 .PP
1647 .Vb 5
1648 \& int (*svt_get) (pTHX_ SV* sv, MAGIC* mg);
1649 \& int (*svt_set) (pTHX_ SV* sv, MAGIC* mg);
1650 \& U32 (*svt_len) (pTHX_ SV* sv, MAGIC* mg);
1651 \& int (*svt_clear)(pTHX_ SV* sv, MAGIC* mg);
1652 \& int (*svt_free) (pTHX_ SV* sv, MAGIC* mg);
1653 \&
1654 \& int (*svt_copy) (pTHX_ SV *sv, MAGIC* mg, SV *nsv,
1655 \& const char *name, I32 namlen);
1656 \& int (*svt_dup) (pTHX_ MAGIC *mg, CLONE_PARAMS *param);
1657 \& int (*svt_local)(pTHX_ SV *nsv, MAGIC *mg);
1658 .Ve
1659 .PP
1660 This MGVTBL structure is set at compile\-time in \fIperl.h\fR and there are
1661 currently 32 types. These different structures contain pointers to various
1662 routines that perform additional actions depending on which function is
1663 being called.
1664 .PP
1665 .Vb 8
1666 \& Function pointer Action taken
1667 \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-
1668 \& svt_get Do something before the value of the SV is
1669 \& retrieved.
1670 \& svt_set Do something after the SV is assigned a value.
1671 \& svt_len Report on the SV\*(Aqs length.
1672 \& svt_clear Clear something the SV represents.
1673 \& svt_free Free any extra storage associated with the SV.
1674 \&
1675 \& svt_copy copy tied variable magic to a tied element
1676 \& svt_dup duplicate a magic structure during thread cloning
1677 \& svt_local copy magic to local value during \*(Aqlocal\*(Aq
1678 .Ve
1679 .PP
1680 For instance, the MGVTBL structure called \f(CW\*(C`vtbl_sv\*(C'\fR (which corresponds
1681 to an \f(CW\*(C`mg_type\*(C'\fR of \f(CW\*(C`PERL_MAGIC_sv\*(C'\fR) contains:
1682 .PP
1683 .Vb 1
1684 \& { magic_get, magic_set, magic_len, 0, 0 }
1685 .Ve
1686 .PP
1687 Thus, when an SV is determined to be magical and of type \f(CW\*(C`PERL_MAGIC_sv\*(C'\fR,
1688 if a get operation is being performed, the routine \f(CW\*(C`magic_get\*(C'\fR is
1689 called. All the various routines for the various magical types begin
1690 with \f(CW\*(C`magic_\*(C'\fR. NOTE: the magic routines are not considered part of
1691 the Perl API, and may not be exported by the Perl library.
1692 .PP
1693 The last three slots are a recent addition, and for source code
1694 compatibility they are only checked for if one of the three flags
1695 \&\f(CW\*(C`MGf_COPY\*(C'\fR, \f(CW\*(C`MGf_DUP\*(C'\fR, or \f(CW\*(C`MGf_LOCAL\*(C'\fR is set in mg_flags.
1696 This means that most code can continue declaring
1697 a vtable as a 5\-element value. These three are
1698 currently used exclusively by the threading code, and are highly subject
1699 to change.
1700 .PP
1701 The current kinds of Magic Virtual Tables are:
1702 .PP
1703 .Vb 10
1704 \& mg_type
1705 \& (old\-style char and macro) MGVTBL Type of magic
1706 \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-
1707 \& \e0 PERL_MAGIC_sv vtbl_sv Special scalar variable
1708 \& # PERL_MAGIC_arylen vtbl_arylen Array length ($#ary)
1709 \& % PERL_MAGIC_rhash (none) Extra data for restricted
1710 \& hashes
1711 \& * PERL_MAGIC_debugvar vtbl_debugvar $DB::single, signal, trace
1712 \& vars
1713 \& . PERL_MAGIC_pos vtbl_pos pos() lvalue
1714 \& : PERL_MAGIC_symtab (none) Extra data for symbol
1715 \& tables
1716 \& < PERL_MAGIC_backref vtbl_backref For weak ref data
1717 \& @ PERL_MAGIC_arylen_p (none) To move arylen out of XPVAV
1718 \& B PERL_MAGIC_bm vtbl_regexp Boyer\-Moore
1719 \& (fast string search)
1720 \& c PERL_MAGIC_overload_table vtbl_ovrld Holds overload table
1721 \& (AMT) on stash
1722 \& D PERL_MAGIC_regdata vtbl_regdata Regex match position data
1723 \& (@+ and @\- vars)
1724 \& d PERL_MAGIC_regdatum vtbl_regdatum Regex match position data
1725 \& element
1726 \& E PERL_MAGIC_env vtbl_env %ENV hash
1727 \& e PERL_MAGIC_envelem vtbl_envelem %ENV hash element
1728 \& f PERL_MAGIC_fm vtbl_regexp Formline
1729 \& (\*(Aqcompiled\*(Aq format)
1730 \& g PERL_MAGIC_regex_global vtbl_mglob m//g target
1731 \& H PERL_MAGIC_hints vtbl_hints %^H hash
1732 \& h PERL_MAGIC_hintselem vtbl_hintselem %^H hash element
1733 \& I PERL_MAGIC_isa vtbl_isa @ISA array
1734 \& i PERL_MAGIC_isaelem vtbl_isaelem @ISA array element
1735 \& k PERL_MAGIC_nkeys vtbl_nkeys scalar(keys()) lvalue
1736 \& L PERL_MAGIC_dbfile (none) Debugger %_<filename
1737 \& l PERL_MAGIC_dbline vtbl_dbline Debugger %_<filename
1738 \& element
1739 \& N PERL_MAGIC_shared (none) Shared between threads
1740 \& n PERL_MAGIC_shared_scalar (none) Shared between threads
1741 \& o PERL_MAGIC_collxfrm vtbl_collxfrm Locale transformation
1742 \& P PERL_MAGIC_tied vtbl_pack Tied array or hash
1743 \& p PERL_MAGIC_tiedelem vtbl_packelem Tied array or hash element
1744 \& q PERL_MAGIC_tiedscalar vtbl_packelem Tied scalar or handle
1745 \& r PERL_MAGIC_qr vtbl_regexp Precompiled qr// regex
1746 \& S PERL_MAGIC_sig vtbl_sig %SIG hash
1747 \& s PERL_MAGIC_sigelem vtbl_sigelem %SIG hash element
1748 \& t PERL_MAGIC_taint vtbl_taint Taintedness
1749 \& U PERL_MAGIC_uvar vtbl_uvar Available for use by
1750 \& extensions
1751 \& u PERL_MAGIC_uvar_elem (none) Reserved for use by
1752 \& extensions
1753 \& V PERL_MAGIC_vstring (none) SV was vstring literal
1754 \& v PERL_MAGIC_vec vtbl_vec vec() lvalue
1755 \& w PERL_MAGIC_utf8 vtbl_utf8 Cached UTF\-8 information
1756 \& X PERL_MAGIC_destruct vtbl_destruct destruct callback
1757 \& x PERL_MAGIC_substr vtbl_substr substr() lvalue
1758 \& Y PERL_MAGIC_nonelem vtbl_nonelem Array element that does not
1759 \& exist
1760 \& y PERL_MAGIC_defelem vtbl_defelem Shadow "foreach" iterator
1761 \& variable / smart parameter
1762 \& vivification
1763 \& Z PERL_MAGIC_hook vtbl_hook %{^HOOK} hash
1764 \& z PERL_MAGIC_hookelem vtbl_hookelem %{^HOOK} hash element
1765 \& \e PERL_MAGIC_lvref vtbl_lvref Lvalue reference
1766 \& constructor
1767 \& ] PERL_MAGIC_checkcall vtbl_checkcall Inlining/mutation of call
1768 \& to this CV
1769 \& ^ PERL_MAGIC_extvalue (none) Value magic available for
1770 \& use by extensions
1771 \& ~ PERL_MAGIC_ext (none) Variable magic available
1772 \& for use by extensions
1773 .Ve
1774 .PP
1775 When an uppercase and lowercase letter both exist in the table, then the
1776 uppercase letter is typically used to represent some kind of composite type
1777 (a list or a hash), and the lowercase letter is used to represent an element
1778 of that composite type. Some internals code makes use of this case
1779 relationship. However, \*(Aqv\*(Aq and \*(AqV\*(Aq (vec and v\-string) are in no way related.
1780 .PP
1781 The \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR, \f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR and \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic types
1782 are defined specifically for use by extensions and will not be used by perl
1783 itself. Extensions can use \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR or \f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR magic to
1784 \&\*(Aqattach\*(Aq private information to variables (typically objects). This is
1785 especially useful because there is no way for normal perl code to corrupt this
1786 private information (unlike using extra elements of a hash object).
1787 \&\f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR is value magic (unlike \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR and
1788 \&\f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR) meaning that on localization the new value will not be
1789 magical.
1790 .PP
1791 Similarly, \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic can be used much like \fBtie()\fR to call a
1792 C function any time a scalar\*(Aqs value is used or changed. The \f(CW\*(C`MAGIC\*(C'\fR\*(Aqs
1793 \&\f(CW\*(C`mg_ptr\*(C'\fR field points to a \f(CW\*(C`ufuncs\*(C'\fR structure:
1794 .PP
1795 .Vb 5
1796 \& struct ufuncs {
1797 \& I32 (*uf_val)(pTHX_ IV, SV*);
1798 \& I32 (*uf_set)(pTHX_ IV, SV*);
1799 \& IV uf_index;
1800 \& };
1801 .Ve
1802 .PP
1803 When the SV is read from or written to, the \f(CW\*(C`uf_val\*(C'\fR or \f(CW\*(C`uf_set\*(C'\fR
1804 function will be called with \f(CW\*(C`uf_index\*(C'\fR as the first arg and a pointer to
1805 the SV as the second. A simple example of how to add \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR
1806 magic is shown below. Note that the ufuncs structure is copied by
1807 sv_magic, so you can safely allocate it on the stack.
1808 .PP
1809 .Vb 10
1810 \& void
1811 \& Umagic(sv)
1812 \& SV *sv;
1813 \& PREINIT:
1814 \& struct ufuncs uf;
1815 \& CODE:
1816 \& uf.uf_val = &my_get_fn;
1817 \& uf.uf_set = &my_set_fn;
1818 \& uf.uf_index = 0;
1819 \& sv_magic(sv, 0, PERL_MAGIC_uvar, (char*)&uf, sizeof(uf));
1820 .Ve
1821 .PP
1822 Attaching \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR to arrays is permissible but has no effect.
1823 .PP
1824 For hashes there is a specialized hook that gives control over hash
1825 keys (but not values). This hook calls \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR \*(Aqget\*(Aq magic
1826 if the "set" function in the \f(CW\*(C`ufuncs\*(C'\fR structure is NULL. The hook
1827 is activated whenever the hash is accessed with a key specified as
1828 an \f(CW\*(C`SV\*(C'\fR through the functions \f(CW\*(C`hv_store_ent\*(C'\fR, \f(CW\*(C`hv_fetch_ent\*(C'\fR,
1829 \&\f(CW\*(C`hv_delete_ent\*(C'\fR, and \f(CW\*(C`hv_exists_ent\*(C'\fR. Accessing the key as a string
1830 through the functions without the \f(CW\*(C`..._ent\*(C'\fR suffix circumvents the
1831 hook. See "GUTS" in Hash::Util::FieldHash for a detailed description.
1832 .PP
1833 Note that because multiple extensions may be using \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR
1834 or \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic, it is important for extensions to take
1835 extra care to avoid conflict. Typically only using the magic on
1836 objects blessed into the same class as the extension is sufficient.
1837 For \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR magic, it is usually a good idea to define an
1838 \&\f(CW\*(C`MGVTBL\*(C'\fR, even if all its fields will be \f(CW0\fR, so that individual
1839 \&\f(CW\*(C`MAGIC\*(C'\fR pointers can be identified as a particular kind of magic
1840 using their magic virtual table. \f(CW\*(C`mg_findext\*(C'\fR provides an easy way
1841 to do that:
1842 .PP
1843 .Vb 1
1844 \& STATIC MGVTBL my_vtbl = { 0, 0, 0, 0, 0, 0, 0, 0 };
1845 \&
1846 \& MAGIC *mg;
1847 \& if ((mg = mg_findext(sv, PERL_MAGIC_ext, &my_vtbl))) {
1848 \& /* this is really ours, not another module\*(Aqs PERL_MAGIC_ext */
1849 \& my_priv_data_t *priv = (my_priv_data_t *)mg\->mg_ptr;
1850 \& ...
1851 \& }
1852 .Ve
1853 .PP
1854 Also note that the \f(CW\*(C`sv_set*()\*(C'\fR and \f(CW\*(C`sv_cat*()\*(C'\fR functions described
1855 earlier do \fBnot\fR invoke \*(Aqset\*(Aq magic on their targets. This must
1856 be done by the user either by calling the \f(CWSvSETMAGIC()\fR macro after
1857 calling these functions, or by using one of the \f(CW\*(C`sv_set*_mg()\*(C'\fR or
1858 \&\f(CW\*(C`sv_cat*_mg()\*(C'\fR functions. Similarly, generic C code must call the
1859 \&\f(CWSvGETMAGIC()\fR macro to invoke any \*(Aqget\*(Aq magic if they use an SV
1860 obtained from external sources in functions that don\*(Aqt handle magic.
1861 See perlapi for a description of these functions.
1862 For example, calls to the \f(CW\*(C`sv_cat*()\*(C'\fR functions typically need to be
1863 followed by \f(CWSvSETMAGIC()\fR, but they don\*(Aqt need a prior \f(CWSvGETMAGIC()\fR
1864 since their implementation handles \*(Aqget\*(Aq magic.
1865 .SS "Finding Magic"
1866 .IX Subsection "Finding Magic"
1867 .Vb 2
1868 \& MAGIC *mg_find(SV *sv, int type); /* Finds the magic pointer of that
1869 \& * type */
1870 .Ve
1871 .PP
1872 This routine returns a pointer to a \f(CW\*(C`MAGIC\*(C'\fR structure stored in the SV.
1873 If the SV does not have that magical
1874 feature, \f(CW\*(C`NULL\*(C'\fR is returned. If the
1875 SV has multiple instances of that magical feature, the first one will be
1876 returned. \f(CW\*(C`mg_findext\*(C'\fR can be used
1877 to find a \f(CW\*(C`MAGIC\*(C'\fR structure of an SV
1878 based on both its magic type and its magic virtual table:
1879 .PP
1880 .Vb 1
1881 \& MAGIC *mg_findext(SV *sv, int type, MGVTBL *vtbl);
1882 .Ve
1883 .PP
1884 Also, if the SV passed to \f(CW\*(C`mg_find\*(C'\fR or \f(CW\*(C`mg_findext\*(C'\fR is not of type
1885 SVt_PVMG, Perl may core dump.
1886 .PP
1887 .Vb 1
1888 \& int mg_copy(SV* sv, SV* nsv, const char* key, STRLEN klen);
1889 .Ve
1890 .PP
1891 This routine checks to see what types of magic \f(CW\*(C`sv\*(C'\fR has. If the mg_type
1892 field is an uppercase letter, then the mg_obj is copied to \f(CW\*(C`nsv\*(C'\fR, but
1893 the mg_type field is changed to be the lowercase letter.
1894 .SS "Understanding the Magic of Tied Hashes and Arrays"
1895 .IX Subsection "Understanding the Magic of Tied Hashes and Arrays"
1896 Tied hashes and arrays are magical beasts of the \f(CW\*(C`PERL_MAGIC_tied\*(C'\fR
1897 magic type.
1898 .PP
1899 WARNING: As of the 5.004 release, proper usage of the array and hash
1900 access functions requires understanding a few caveats. Some
1901 of these caveats are actually considered bugs in the API, to be fixed
1902 in later releases, and are bracketed with [MAYCHANGE] below. If
1903 you find yourself actually applying such information in this section, be
1904 aware that the behavior may change in the future, umm, without warning.
1905 .PP
1906 The perl tie function associates a variable with an object that implements
1907 the various GET, SET, etc methods. To perform the equivalent of the perl
1908 tie function from an XSUB, you must mimic this behaviour. The code below
1909 carries out the necessary steps \-\- firstly it creates a new hash, and then
1910 creates a second hash which it blesses into the class which will implement
1911 the tie methods. Lastly it ties the two hashes together, and returns a
1912 reference to the new tied hash. Note that the code below does NOT call the
1913 TIEHASH method in the MyTie class \-
1914 see "Calling Perl Routines from within C Programs" for details on how
1915 to do this.
1916 .PP
1917 .Vb 10
1918 \& SV*
1919 \& mytie()
1920 \& PREINIT:
1921 \& HV *hash;
1922 \& HV *stash;
1923 \& SV *tie;
1924 \& CODE:
1925 \& hash = newHV();
1926 \& tie = newRV_noinc((SV*)newHV());
1927 \& stash = gv_stashpv("MyTie", GV_ADD);
1928 \& sv_bless(tie, stash);
1929 \& hv_magic(hash, (GV*)tie, PERL_MAGIC_tied);
1930 \& SvREFCNT_dec(tie); /* hv_magic() increases tie ref count */
1931 \& RETVAL = newRV_noinc(hash);
1932 \& OUTPUT:
1933 \& RETVAL
1934 .Ve
1935 .PP
1936 The \f(CW\*(C`av_store\*(C'\fR function, when given a tied array argument, merely
1937 copies the magic of the array onto the value to be "stored", using
1938 \&\f(CW\*(C`mg_copy\*(C'\fR. It may also return NULL, indicating that the value did not
1939 actually need to be stored in the array. [MAYCHANGE] After a call to
1940 \&\f(CW\*(C`av_store\*(C'\fR on a tied array, the caller will usually need to call
1941 \&\f(CWmg_set(val)\fR to actually invoke the perl level "STORE" method on the
1942 TIEARRAY object. If \f(CW\*(C`av_store\*(C'\fR did return NULL, a call to
1943 \&\f(CWSvREFCNT_dec(val)\fR will also be usually necessary to avoid a memory
1944 leak. [/MAYCHANGE]
1945 .PP
1946 The previous paragraph is applicable verbatim to tied hash access using the
1947 \&\f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_store_ent\*(C'\fR functions as well.
1948 .PP
1949 \&\f(CW\*(C`av_fetch\*(C'\fR and the corresponding hash functions \f(CW\*(C`hv_fetch\*(C'\fR and
1950 \&\f(CW\*(C`hv_fetch_ent\*(C'\fR actually return an undefined mortal value whose magic
1951 has been initialized using \f(CW\*(C`mg_copy\*(C'\fR. Note the value so returned does not
1952 need to be deallocated, as it is already mortal. [MAYCHANGE] But you will
1953 need to call \f(CWmg_get()\fR on the returned value in order to actually invoke
1954 the perl level "FETCH" method on the underlying TIE object. Similarly,
1955 you may also call \f(CWmg_set()\fR on the return value after possibly assigning
1956 a suitable value to it using \f(CW\*(C`sv_setsv\*(C'\fR, which will invoke the "STORE"
1957 method on the TIE object. [/MAYCHANGE]
1958 .PP
1959 [MAYCHANGE]
1960 In other words, the array or hash fetch/store functions don\*(Aqt really
1961 fetch and store actual values in the case of tied arrays and hashes. They
1962 merely call \f(CW\*(C`mg_copy\*(C'\fR to attach magic to the values that were meant to be
1963 "stored" or "fetched". Later calls to \f(CW\*(C`mg_get\*(C'\fR and \f(CW\*(C`mg_set\*(C'\fR actually
1964 do the job of invoking the TIE methods on the underlying objects. Thus
1965 the magic mechanism currently implements a kind of lazy access to arrays
1966 and hashes.
1967 .PP
1968 Currently (as of perl version 5.004), use of the hash and array access
1969 functions requires the user to be aware of whether they are operating on
1970 "normal" hashes and arrays, or on their tied variants. The API may be
1971 changed to provide more transparent access to both tied and normal data
1972 types in future versions.
1973 [/MAYCHANGE]
1974 .PP
1975 You would do well to understand that the TIEARRAY and TIEHASH interfaces
1976 are mere sugar to invoke some perl method calls while using the uniform hash
1977 and array syntax. The use of this sugar imposes some overhead (typically
1978 about two to four extra opcodes per FETCH/STORE operation, in addition to
1979 the creation of all the mortal variables required to invoke the methods).
1980 This overhead will be comparatively small if the TIE methods are themselves
1981 substantial, but if they are only a few statements long, the overhead
1982 will not be insignificant.
1983 .SS "Localizing changes"
1984 .IX Subsection "Localizing changes"
1985 Perl has a very handy construction
1986 .PP
1987 .Vb 4
1988 \& {
1989 \& local $var = 2;
1990 \& ...
1991 \& }
1992 .Ve
1993 .PP
1994 This construction is \fIapproximately\fR equivalent to
1995 .PP
1996 .Vb 6
1997 \& {
1998 \& my $oldvar = $var;
1999 \& $var = 2;
2000 \& ...
2001 \& $var = $oldvar;
2002 \& }
2003 .Ve
2004 .PP
2005 The biggest difference is that the first construction would
2006 reinstate the initial value of \f(CW$var\fR, irrespective of how control exits
2007 the block: \f(CW\*(C`goto\*(C'\fR, \f(CW\*(C`return\*(C'\fR, \f(CW\*(C`die\*(C'\fR/\f(CW\*(C`eval\*(C'\fR, etc. It is a little bit
2008 more efficient as well.
2009 .PP
2010 There is a way to achieve a similar task from C via Perl API: create a
2011 \&\fIpseudo\-block\fR, and arrange for some changes to be automatically
2012 undone at the end of it, either explicit, or via a non\-local exit (via
2013 \&\fBdie()\fR). A \fIblock\fR\-like construct is created by a pair of
2014 \&\f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR macros (see "Returning a Scalar" in perlcall).
2015 Such a construct may be created specially for some important localized
2016 task, or an existing one (like boundaries of enclosing Perl
2017 subroutine/block, or an existing pair for freeing TMPs) may be
2018 used. (In the second case the overhead of additional localization must
2019 be almost negligible.) Note that any XSUB is automatically enclosed in
2020 an \f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR pair.
2021 .PP
2022 Inside such a \fIpseudo\-block\fR the following service is available:
2023 .ie n .IP """SAVEINT(int i)""" 4
2024 .el .IP "\f(CWSAVEINT(int i)\fR" 4
2025 .IX Item "SAVEINT(int i)"
2026 .PD 0
2027 .ie n .IP """SAVEIV(IV i)""" 4
2028 .el .IP "\f(CWSAVEIV(IV i)\fR" 4
2029 .IX Item "SAVEIV(IV i)"
2030 .ie n .IP """SAVEI32(I32 i)""" 4
2031 .el .IP "\f(CWSAVEI32(I32 i)\fR" 4
2032 .IX Item "SAVEI32(I32 i)"
2033 .ie n .IP """SAVEI8(I8 i)""" 4
2034 .el .IP "\f(CWSAVEI8(I8 i)\fR" 4
2035 .IX Item "SAVEI8(I8 i)"
2036 .ie n .IP """SAVEI16(I16 i)""" 4
2037 .el .IP "\f(CWSAVEI16(I16 i)\fR" 4
2038 .IX Item "SAVEI16(I16 i)"
2039 .ie n .IP """SAVEBOOL(int i)""" 4
2040 .el .IP "\f(CWSAVEBOOL(int i)\fR" 4
2041 .IX Item "SAVEBOOL(int i)"
2042 .ie n .IP """SAVESTRLEN(STRLEN i)""" 4
2043 .el .IP "\f(CWSAVESTRLEN(STRLEN i)\fR" 4
2044 .IX Item "SAVESTRLEN(STRLEN i)"
2045 .PD
2046 These macros arrange things to restore the value of integer variable
2047 \&\f(CW\*(C`i\*(C'\fR at the end of the enclosing \fIpseudo\-block\fR.
2048 .ie n .IP SAVESPTR(s) 4
2049 .el .IP \f(CWSAVESPTR(s)\fR 4
2050 .IX Item "SAVESPTR(s)"
2051 .PD 0
2052 .ie n .IP SAVEPPTR(p) 4
2053 .el .IP \f(CWSAVEPPTR(p)\fR 4
2054 .IX Item "SAVEPPTR(p)"
2055 .PD
2056 These macros arrange things to restore the value of pointers \f(CW\*(C`s\*(C'\fR and
2057 \&\f(CW\*(C`p\*(C'\fR. \f(CW\*(C`s\*(C'\fR must be a pointer of a type which survives conversion to
2058 \&\f(CW\*(C`SV*\*(C'\fR and back, \f(CW\*(C`p\*(C'\fR should be able to survive conversion to \f(CW\*(C`char*\*(C'\fR
2059 and back.
2060 .ie n .IP """SAVERCPV(char **ppv)""" 4
2061 .el .IP "\f(CWSAVERCPV(char **ppv)\fR" 4
2062 .IX Item "SAVERCPV(char **ppv)"
2063 This macro arranges to restore the value of a \f(CW\*(C`char *\*(C'\fR variable which
2064 was allocated with a call to \f(CWrcpv_new()\fR to its previous state when
2065 the current pseudo block is completed. The pointer stored in \f(CW*ppv\fR at
2066 the time of the call will be refcount incremented and stored on the save
2067 stack. Later when the current \fIpseudo\-block\fR is completed the value
2068 stored in \f(CW*ppv\fR will be refcount decremented, and the previous value
2069 restored from the savestack which will also be refcount decremented.
2070 .Sp
2071 This is the \f(CW\*(C`RCPV\*(C'\fR equivalent of \f(CWSAVEGENERICSV()\fR.
2072 .ie n .IP """SAVEGENERICSV(SV **psv)""" 4
2073 .el .IP "\f(CWSAVEGENERICSV(SV **psv)\fR" 4
2074 .IX Item "SAVEGENERICSV(SV **psv)"
2075 This macro arranges to restore the value of a \f(CW\*(C`SV *\*(C'\fR variable to its
2076 previous state when the current pseudo block is completed. The pointer
2077 stored in \f(CW*psv\fR at the time of the call will be refcount incremented
2078 and stored on the save stack. Later when the current \fIpseudo\-block\fR is
2079 completed the value stored in \f(CW*ppv\fR will be refcount decremented, and
2080 the previous value restored from the savestack which will also be refcount
2081 decremented. This the C equivalent of \f(CW\*(C`local $sv\*(C'\fR.
2082 .ie n .IP """SAVEFREESV(SV *sv)""" 4
2083 .el .IP "\f(CWSAVEFREESV(SV *sv)\fR" 4
2084 .IX Item "SAVEFREESV(SV *sv)"
2085 The refcount of \f(CW\*(C`sv\*(C'\fR will be decremented at the end of
2086 \&\fIpseudo\-block\fR. This is similar to \f(CW\*(C`sv_2mortal\*(C'\fR in that it is also a
2087 mechanism for doing a delayed \f(CW\*(C`SvREFCNT_dec\*(C'\fR. However, while \f(CW\*(C`sv_2mortal\*(C'\fR
2088 extends the lifetime of \f(CW\*(C`sv\*(C'\fR until the beginning of the next statement,
2089 \&\f(CW\*(C`SAVEFREESV\*(C'\fR extends it until the end of the enclosing scope. These
2090 lifetimes can be wildly different.
2091 .Sp
2092 Also compare \f(CW\*(C`SAVEMORTALIZESV\*(C'\fR.
2093 .ie n .IP """SAVEMORTALIZESV(SV *sv)""" 4
2094 .el .IP "\f(CWSAVEMORTALIZESV(SV *sv)\fR" 4
2095 .IX Item "SAVEMORTALIZESV(SV *sv)"
2096 Just like \f(CW\*(C`SAVEFREESV\*(C'\fR, but mortalizes \f(CW\*(C`sv\*(C'\fR at the end of the current
2097 scope instead of decrementing its reference count. This usually has the
2098 effect of keeping \f(CW\*(C`sv\*(C'\fR alive until the statement that called the currently
2099 live scope has finished executing.
2100 .ie n .IP """SAVEFREEOP(OP *op)""" 4
2101 .el .IP "\f(CWSAVEFREEOP(OP *op)\fR" 4
2102 .IX Item "SAVEFREEOP(OP *op)"
2103 The \f(CW\*(C`OP *\*(C'\fR is \f(CWop_free()\fRed at the end of \fIpseudo\-block\fR.
2104 .ie n .IP SAVEFREEPV(p) 4
2105 .el .IP \f(CWSAVEFREEPV(p)\fR 4
2106 .IX Item "SAVEFREEPV(p)"
2107 The chunk of memory which is pointed to by \f(CW\*(C`p\*(C'\fR is \f(CWSafefree()\fRed at the
2108 end of the current \fIpseudo\-block\fR.
2109 .ie n .IP """SAVEFREERCPV(char *pv)""" 4
2110 .el .IP "\f(CWSAVEFREERCPV(char *pv)\fR" 4
2111 .IX Item "SAVEFREERCPV(char *pv)"
2112 Ensures that a \f(CW\*(C`char *\*(C'\fR which was created by a call to \f(CWrcpv_new()\fR is
2113 \&\f(CWrcpv_free()\fRed at the end of the current \fIpseudo\-block\fR.
2114 .Sp
2115 This is the RCPV equivalent of \f(CWSAVEFREESV()\fR.
2116 .ie n .IP """SAVECLEARSV(SV *sv)""" 4
2117 .el .IP "\f(CWSAVECLEARSV(SV *sv)\fR" 4
2118 .IX Item "SAVECLEARSV(SV *sv)"
2119 Clears a slot in the current scratchpad which corresponds to \f(CW\*(C`sv\*(C'\fR at
2120 the end of \fIpseudo\-block\fR.
2121 .ie n .IP """SAVEDELETE(HV *hv, char *key, I32 length)""" 4
2122 .el .IP "\f(CWSAVEDELETE(HV *hv, char *key, I32 length)\fR" 4
2123 .IX Item "SAVEDELETE(HV *hv, char *key, I32 length)"
2124 The key \f(CW\*(C`key\*(C'\fR of \f(CW\*(C`hv\*(C'\fR is deleted at the end of \fIpseudo\-block\fR. The
2125 string pointed to by \f(CW\*(C`key\*(C'\fR is \fBSafefree()\fRed. If one has a \fIkey\fR in
2126 short\-lived storage, the corresponding string may be reallocated like
2127 this:
2128 .Sp
2129 .Vb 1
2130 \& SAVEDELETE(PL_defstash, savepv(tmpbuf), strlen(tmpbuf));
2131 .Ve
2132 .ie n .IP """SAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)""" 4
2133 .el .IP "\f(CWSAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)\fR" 4
2134 .IX Item "SAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)"
2135 At the end of \fIpseudo\-block\fR the function \f(CW\*(C`f\*(C'\fR is called with the
2136 only argument \f(CW\*(C`p\*(C'\fR which may be NULL.
2137 .ie n .IP """SAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)""" 4
2138 .el .IP "\f(CWSAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)\fR" 4
2139 .IX Item "SAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)"
2140 At the end of \fIpseudo\-block\fR the function \f(CW\*(C`f\*(C'\fR is called with the
2141 implicit context argument (if any), and \f(CW\*(C`p\*(C'\fR which may be NULL.
2142 .Sp
2143 Note the \fIend of the current pseudo\-block\fR may occur much later than
2144 the \fIend of the current statement\fR. You may wish to look at the
2145 \&\f(CWMORTALSVFUNC_X()\fR macro instead.
2146 .ie n .IP """MORTALSVFUNC_X(SVFUNC_t f, SV *sv)""" 4
2147 .el .IP "\f(CWMORTALSVFUNC_X(SVFUNC_t f, SV *sv)\fR" 4
2148 .IX Item "MORTALSVFUNC_X(SVFUNC_t f, SV *sv)"
2149 At the end of \fIthe current statement\fR the function \f(CW\*(C`f\*(C'\fR is called with
2150 the implicit context argument (if any), and \f(CW\*(C`sv\*(C'\fR which may be NULL.
2151 .Sp
2152 Be aware that the parameter argument to the destructor function differs
2153 from the related \f(CWSAVEDESTRUCTOR_X()\fR in that it MUST be either NULL or
2154 an \f(CW\*(C`SV*\*(C'\fR.
2155 .Sp
2156 Note the \fIend of the current statement\fR may occur much before the
2157 \&\fIend of the current pseudo\-block\fR. You may wish to look at the
2158 \&\f(CWSAVEDESTRUCTOR_X()\fR macro instead.
2159 .ie n .IP """MORTALDESTRUCTOR_SV(SV *coderef, SV *args)""" 4
2160 .el .IP "\f(CWMORTALDESTRUCTOR_SV(SV *coderef, SV *args)\fR" 4
2161 .IX Item "MORTALDESTRUCTOR_SV(SV *coderef, SV *args)"
2162 At the end of \fIthe current statement\fR the Perl function contained in
2163 \&\f(CW\*(C`coderef\*(C'\fR is called with the arguments provided (if any) in \f(CW\*(C`args\*(C'\fR.
2164 See the documentation for \f(CWmortal_destructor_sv()\fR for details on
2165 the \f(CW\*(C`args\*(C'\fR parameter is handled.
2166 .Sp
2167 Note the \fIend of the current statement\fR may occur much before the
2168 \&\fIend of the current pseudo\-block\fR. If you wish to call a perl
2169 function at the end of the current pseudo block you should use the
2170 \&\f(CWSAVEDESTRUCTOR_X()\fR API instead, which will require you create a
2171 C wrapper to call the Perl function.
2172 .ie n .IP SAVESTACK_POS() 4
2173 .el .IP \f(CWSAVESTACK_POS()\fR 4
2174 .IX Item "SAVESTACK_POS()"
2175 The current offset on the Perl internal stack (cf. \f(CW\*(C`SP\*(C'\fR) is restored
2176 at the end of \fIpseudo\-block\fR.
2177 .PP
2178 The following API list contains functions, thus one needs to
2179 provide pointers to the modifiable data explicitly (either C pointers,
2180 or Perlish \f(CW\*(C`GV *\*(C'\fRs). Where the above macros take \f(CW\*(C`int\*(C'\fR, a similar
2181 function takes \f(CW\*(C`int *\*(C'\fR.
2182 .PP
2183 Other macros above have functions implementing them, but it\*(Aqs probably
2184 best to just use the macro, and not those or the ones below.
2185 .ie n .IP """SV* save_scalar(GV *gv)""" 4
2186 .el .IP "\f(CWSV* save_scalar(GV *gv)\fR" 4
2187 .IX Item "SV* save_scalar(GV *gv)"
2188 Equivalent to Perl code \f(CW\*(C`local $gv\*(C'\fR.
2189 .ie n .IP """AV* save_ary(GV *gv)""" 4
2190 .el .IP "\f(CWAV* save_ary(GV *gv)\fR" 4
2191 .IX Item "AV* save_ary(GV *gv)"
2192 .PD 0
2193 .ie n .IP """HV* save_hash(GV *gv)""" 4
2194 .el .IP "\f(CWHV* save_hash(GV *gv)\fR" 4
2195 .IX Item "HV* save_hash(GV *gv)"
2196 .PD
2197 Similar to \f(CW\*(C`save_scalar\*(C'\fR, but localize \f(CW@gv\fR and \f(CW%gv\fR.
2198 .ie n .IP """void save_item(SV *item)""" 4
2199 .el .IP "\f(CWvoid save_item(SV *item)\fR" 4
2200 .IX Item "void save_item(SV *item)"
2201 Duplicates the current value of \f(CW\*(C`SV\*(C'\fR. On the exit from the current
2202 \&\f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR \fIpseudo\-block\fR the value of \f(CW\*(C`SV\*(C'\fR will be restored
2203 using the stored value. It doesn\*(Aqt handle magic. Use \f(CW\*(C`save_scalar\*(C'\fR if
2204 magic is affected.
2205 .ie n .IP """SV* save_svref(SV **sptr)""" 4
2206 .el .IP "\f(CWSV* save_svref(SV **sptr)\fR" 4
2207 .IX Item "SV* save_svref(SV **sptr)"
2208 Similar to \f(CW\*(C`save_scalar\*(C'\fR, but will reinstate an \f(CW\*(C`SV *\*(C'\fR.
2209 .ie n .IP """void save_aptr(AV **aptr)""" 4
2210 .el .IP "\f(CWvoid save_aptr(AV **aptr)\fR" 4
2211 .IX Item "void save_aptr(AV **aptr)"
2212 .PD 0
2213 .ie n .IP """void save_hptr(HV **hptr)""" 4
2214 .el .IP "\f(CWvoid save_hptr(HV **hptr)\fR" 4
2215 .IX Item "void save_hptr(HV **hptr)"
2216 .PD
2217 Similar to \f(CW\*(C`save_svref\*(C'\fR, but localize \f(CW\*(C`AV *\*(C'\fR and \f(CW\*(C`HV *\*(C'\fR.
2218 .PP
2219 The \f(CW\*(C`Alias\*(C'\fR module implements localization of the basic types within the
2220 \&\fIcaller\*(Aqs scope\fR. People who are interested in how to localize things in
2221 the containing scope should take a look there too.
2222 .SH Subroutines
2223 .IX Header "Subroutines"
2224 .SS "XSUBs and the Argument Stack"
2225 .IX Subsection "XSUBs and the Argument Stack"
2226 The XSUB mechanism is a simple way for Perl programs to access C subroutines.
2227 An XSUB routine will have a stack that contains the arguments from the Perl
2228 program, and a way to map from the Perl data structures to a C equivalent.
2229 .PP
2230 The stack arguments are accessible through the \f(CWST(n)\fR macro, which returns
2231 the \f(CW\*(C`n\*(C'\fR\*(Aqth stack argument. Argument 0 is the first argument passed in the
2232 Perl subroutine call. These arguments are \f(CW\*(C`SV*\*(C'\fR, and can be used anywhere
2233 an \f(CW\*(C`SV*\*(C'\fR is used.
2234 .PP
2235 Most of the time, output from the C routine can be handled through use of
2236 the RETVAL and OUTPUT directives. However, there are some cases where the
2237 argument stack is not already long enough to handle all the return values.
2238 An example is the POSIX \fBtzname()\fR call, which takes no arguments, but returns
2239 two, the local time zone\*(Aqs standard and summer time abbreviations.
2240 .PP
2241 To handle this situation, the PPCODE directive is used and the stack is
2242 extended using the macro:
2243 .PP
2244 .Vb 1
2245 \& EXTEND(SP, num);
2246 .Ve
2247 .PP
2248 where \f(CW\*(C`SP\*(C'\fR is the macro that represents the local copy of the stack pointer,
2249 and \f(CW\*(C`num\*(C'\fR is the number of elements the stack should be extended by.
2250 .PP
2251 Now that there is room on the stack, values can be pushed on it using \f(CW\*(C`PUSHs\*(C'\fR
2252 macro. The pushed values will often need to be "mortal" (See
2253 "Reference Counts and Mortality"):
2254 .PP
2255 .Vb 7
2256 \& PUSHs(sv_2mortal(newSViv(an_integer)))
2257 \& PUSHs(sv_2mortal(newSVuv(an_unsigned_integer)))
2258 \& PUSHs(sv_2mortal(newSVnv(a_double)))
2259 \& PUSHs(sv_2mortal(newSVpv("Some String",0)))
2260 \& /* Although the last example is better written as the more
2261 \& * efficient: */
2262 \& PUSHs(newSVpvs_flags("Some String", SVs_TEMP))
2263 .Ve
2264 .PP
2265 And now the Perl program calling \f(CW\*(C`tzname\*(C'\fR, the two values will be assigned
2266 as in:
2267 .PP
2268 .Vb 1
2269 \& ($standard_abbrev, $summer_abbrev) = POSIX::tzname;
2270 .Ve
2271 .PP
2272 An alternate (and possibly simpler) method to pushing values on the stack is
2273 to use the macro:
2274 .PP
2275 .Vb 1
2276 \& XPUSHs(SV*)
2277 .Ve
2278 .PP
2279 This macro automatically adjusts the stack for you, if needed. Thus, you
2280 do not need to call \f(CW\*(C`EXTEND\*(C'\fR to extend the stack.
2281 .PP
2282 Despite their suggestions in earlier versions of this document the macros
2283 \&\f(CW\*(C`(X)PUSH[iunp]\*(C'\fR are \fInot\fR suited to XSUBs which return multiple results.
2284 For that, either stick to the \f(CW\*(C`(X)PUSHs\*(C'\fR macros shown above, or use the new
2285 \&\f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros instead; see "Putting a C value on Perl stack".
2286 .PP
2287 For more information, consult perlxs and perlxstut.
2288 .SS "Autoloading with XSUBs"
2289 .IX Subsection "Autoloading with XSUBs"
2290 If an AUTOLOAD routine is an XSUB, as with Perl subroutines, Perl puts the
2291 fully\-qualified name of the autoloaded subroutine in the \f(CW$AUTOLOAD\fR variable
2292 of the XSUB\*(Aqs package.
2293 .PP
2294 But it also puts the same information in certain fields of the XSUB itself:
2295 .PP
2296 .Vb 4
2297 \& HV *stash = CvSTASH(cv);
2298 \& const char *subname = SvPVX(cv);
2299 \& STRLEN name_length = SvCUR(cv); /* in bytes */
2300 \& U32 is_utf8 = SvUTF8(cv);
2301 .Ve
2302 .PP
2303 \&\f(CWSvPVX(cv)\fR contains just the sub name itself, not including the package.
2304 For an AUTOLOAD routine in UNIVERSAL or one of its superclasses,
2305 \&\f(CWCvSTASH(cv)\fR returns NULL during a method call on a nonexistent package.
2306 .PP
2307 \&\fBNote\fR: Setting \f(CW$AUTOLOAD\fR stopped working in 5.6.1, which did not support
2308 XS AUTOLOAD subs at all. Perl 5.8.0 introduced the use of fields in the
2309 XSUB itself. Perl 5.16.0 restored the setting of \f(CW$AUTOLOAD\fR. If you need
2310 to support 5.8\-5.14, use the XSUB\*(Aqs fields.
2311 .SS "Calling Perl Routines from within C Programs"
2312 .IX Subsection "Calling Perl Routines from within C Programs"
2313 There are four routines that can be used to call a Perl subroutine from
2314 within a C program. These four are:
2315 .PP
2316 .Vb 4
2317 \& I32 call_sv(SV*, I32);
2318 \& I32 call_pv(const char*, I32);
2319 \& I32 call_method(const char*, I32);
2320 \& I32 call_argv(const char*, I32, char**);
2321 .Ve
2322 .PP
2323 The routine most often used is \f(CW\*(C`call_sv\*(C'\fR. The \f(CW\*(C`SV*\*(C'\fR argument
2324 contains either the name of the Perl subroutine to be called, or a
2325 reference to the subroutine. The second argument consists of flags
2326 that control the context in which the subroutine is called, whether
2327 or not the subroutine is being passed arguments, how errors should be
2328 trapped, and how to treat return values.
2329 .PP
2330 All four routines return the number of arguments that the subroutine returned
2331 on the Perl stack.
2332 .PP
2333 These routines used to be called \f(CW\*(C`perl_call_sv\*(C'\fR, etc., before Perl v5.6.0,
2334 but those names are now deprecated; macros of the same name are provided for
2335 compatibility.
2336 .PP
2337 When using any of these routines (except \f(CW\*(C`call_argv\*(C'\fR), the programmer
2338 must manipulate the Perl stack. These include the following macros and
2339 functions:
2340 .PP
2341 .Vb 11
2342 \& dSP
2343 \& SP
2344 \& PUSHMARK()
2345 \& PUTBACK
2346 \& SPAGAIN
2347 \& ENTER
2348 \& SAVETMPS
2349 \& FREETMPS
2350 \& LEAVE
2351 \& XPUSH*()
2352 \& POP*()
2353 .Ve
2354 .PP
2355 For a detailed description of calling conventions from C to Perl,
2356 consult perlcall.
2357 .SS "Putting a C value on Perl stack"
2358 .IX Subsection "Putting a C value on Perl stack"
2359 A lot of opcodes (this is an elementary operation in the internal perl
2360 stack machine) put an SV* on the stack. However, as an optimization
2361 the corresponding SV is (usually) not recreated each time. The opcodes
2362 reuse specially assigned SVs (\fItarget\fRs) which are (as a corollary)
2363 not constantly freed/created.
2364 .PP
2365 Each of the targets is created only once (but see
2366 "Scratchpads and recursion" below), and when an opcode needs to put
2367 an integer, a double, or a string on the stack, it just sets the
2368 corresponding parts of its \fItarget\fR and puts the \fItarget\fR on stack.
2369 .PP
2370 The macro to put this target on stack is \f(CW\*(C`PUSHTARG\*(C'\fR, and it is
2371 directly used in some opcodes, as well as indirectly in zillions of
2372 others, which use it via \f(CW\*(C`(X)PUSH[iunp]\*(C'\fR.
2373 .PP
2374 Because the target is reused, you must be careful when pushing multiple
2375 values on the stack. The following code will not do what you think:
2376 .PP
2377 .Vb 2
2378 \& XPUSHi(10);
2379 \& XPUSHi(20);
2380 .Ve
2381 .PP
2382 This translates as "set \f(CW\*(C`TARG\*(C'\fR to 10, push a pointer to \f(CW\*(C`TARG\*(C'\fR onto
2383 the stack; set \f(CW\*(C`TARG\*(C'\fR to 20, push a pointer to \f(CW\*(C`TARG\*(C'\fR onto the stack".
2384 At the end of the operation, the stack does not contain the values 10
2385 and 20, but actually contains two pointers to \f(CW\*(C`TARG\*(C'\fR, which we have set
2386 to 20.
2387 .PP
2388 If you need to push multiple different values then you should either use
2389 the \f(CW\*(C`(X)PUSHs\*(C'\fR macros, or else use the new \f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros,
2390 none of which make use of \f(CW\*(C`TARG\*(C'\fR. The \f(CW\*(C`(X)PUSHs\*(C'\fR macros simply push an
2391 SV* on the stack, which, as noted under "XSUBs and the Argument Stack",
2392 will often need to be "mortal". The new \f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros make
2393 this a little easier to achieve by creating a new mortal for you (via
2394 \&\f(CW\*(C`(X)PUSHmortal\*(C'\fR), pushing that onto the stack (extending it if necessary
2395 in the case of the \f(CW\*(C`mXPUSH[iunp]\*(C'\fR macros), and then setting its value.
2396 Thus, instead of writing this to "fix" the example above:
2397 .PP
2398 .Vb 2
2399 \& XPUSHs(sv_2mortal(newSViv(10)))
2400 \& XPUSHs(sv_2mortal(newSViv(20)))
2401 .Ve
2402 .PP
2403 you can simply write:
2404 .PP
2405 .Vb 2
2406 \& mXPUSHi(10)
2407 \& mXPUSHi(20)
2408 .Ve
2409 .PP
2410 On a related note, if you do use \f(CW\*(C`(X)PUSH[iunp]\*(C'\fR, then you\*(Aqre going to
2411 need a \f(CW\*(C`dTARG\*(C'\fR in your variable declarations so that the \f(CW\*(C`*PUSH*\*(C'\fR
2412 macros can make use of the local variable \f(CW\*(C`TARG\*(C'\fR. See also
2413 \&\f(CW\*(C`dTARGET\*(C'\fR and \f(CW\*(C`dXSTARG\*(C'\fR.
2414 .PP
2415 Be careful with TARG: each non\-recursive call to your OP, or XSUB for
2416 dXSTARG, will use the same SV for TARG for each call, so you can\*(Aqt
2417 assume the SV is pristine since the results from the previous call to
2418 your OP or XSUB may still be in TARG.
2419 .PP
2420 Other state such as a cached numeric version of a PV or the cached
2421 UTF\-8 length may also be set. You can avoid problems with this by
2422 using the normal sv_set*() APIs instead of trying to optimize by
2423 directly manipulating the SV.
2424 .PP
2425 For OPs that implement the \f(CW\*(C`OA_TARGLEX\*(C'\fR optimization, TARG may be
2426 special in other ways, there may be references to it, it might be
2427 blessed, it might be tied, or magical in other ways.
2428 .PP
2429 You must not use TARG to return a reference, since that reference will
2430 be live until the next time the OP or XSUB executes non\-recursively,
2431 or until perl exits.
2432 .SS Scratchpads
2433 .IX Subsection "Scratchpads"
2434 The question remains on when the SVs which are \fItarget\fRs for opcodes
2435 are created. The answer is that they are created when the current
2436 unit\-\-a subroutine or a file (for opcodes for statements outside of
2437 subroutines)\-\-is compiled. During this time a special anonymous Perl
2438 array is created, which is called a scratchpad for the current unit.
2439 .PP
2440 A scratchpad keeps SVs which are lexicals for the current unit and are
2441 targets for opcodes. A previous version of this document
2442 stated that one can deduce that an SV lives on a scratchpad
2443 by looking on its flags: lexicals have \f(CW\*(C`SVs_PADMY\*(C'\fR set, and
2444 \&\fItarget\fRs have \f(CW\*(C`SVs_PADTMP\*(C'\fR set. But this has never been fully true.
2445 \&\f(CW\*(C`SVs_PADMY\*(C'\fR could be set on a variable that no longer resides in any pad.
2446 While \fItarget\fRs do have \f(CW\*(C`SVs_PADTMP\*(C'\fR set, it can also be set on variables
2447 that have never resided in a pad, but nonetheless act like \fItarget\fRs. As
2448 of perl 5.21.5, the \f(CW\*(C`SVs_PADMY\*(C'\fR flag is no longer used and is defined as
2449 0. \f(CWSvPADMY()\fR now returns true for anything without \f(CW\*(C`SVs_PADTMP\*(C'\fR.
2450 .PP
2451 The correspondence between OPs and \fItarget\fRs is not 1\-to\-1. Different
2452 OPs in the compile tree of the unit can use the same target, if this
2453 would not conflict with the expected life of the temporary.
2454 .SS "Scratchpads and recursion"
2455 .IX Subsection "Scratchpads and recursion"
2456 In fact it is not 100% true that a compiled unit contains a pointer to
2457 the scratchpad AV. In fact it contains a pointer to an AV of
2458 (initially) one element, and this element is the scratchpad AV. Why do
2459 we need an extra level of indirection?
2460 .PP
2461 The answer is \fBrecursion\fR, and maybe \fBthreads\fR. Both
2462 these can create several execution pointers going into the same
2463 subroutine. For the subroutine\-child not write over the temporaries
2464 for the subroutine\-parent (lifespan of which covers the call to the
2465 child), the parent and the child should have different
2466 scratchpads. (\fIAnd\fR the lexicals should be separate anyway!)
2467 .PP
2468 So each subroutine is born with an array of scratchpads (of length 1).
2469 On each entry to the subroutine it is checked that the current
2470 depth of the recursion is not more than the length of this array, and
2471 if it is, new scratchpad is created and pushed into the array.
2472 .PP
2473 The \fItarget\fRs on this scratchpad are \f(CW\*(C`undef\*(C'\fRs, but they are already
2474 marked with correct flags.
2475 .SH "Memory Allocation"
2476 .IX Header "Memory Allocation"
2477 .SS Allocation
2478 .IX Subsection "Allocation"
2479 All memory meant to be used with the Perl API functions should be manipulated
2480 using the macros described in this section. The macros provide the necessary
2481 transparency between differences in the actual malloc implementation that is
2482 used within perl. Never pass pointers between \f(CW\*(C`Newx\*(C'\fR, \f(CW\*(C`Renew\*(C'\fR, \f(CW\*(C`Safefree\*(C'\fR
2483 and \fIlibc\fR equivalents \f(CW\*(C`malloc\*(C'\fR, \f(CW\*(C`realloc\*(C'\fR, \f(CW\*(C`free\*(C'\fR. They are not from the
2484 same memory pool or allocator. Either an instant or delayed \fISEGV\fR will
2485 occur, or subtle memory leaks or subtle heap corruption.
2486 .PP
2487 The following three macros are used to initially allocate memory :
2488 .PP
2489 .Vb 3
2490 \& Newx(pointer, number, type);
2491 \& Newxc(pointer, number, type, cast);
2492 \& Newxz(pointer, number, type);
2493 .Ve
2494 .PP
2495 The first argument \f(CW\*(C`pointer\*(C'\fR should be the name of a variable that will
2496 point to the newly allocated memory.
2497 .PP
2498 The second and third arguments \f(CW\*(C`number\*(C'\fR and \f(CW\*(C`type\*(C'\fR specify how many of
2499 the specified type of data structure should be allocated. The argument
2500 \&\f(CW\*(C`type\*(C'\fR is passed to \f(CW\*(C`sizeof\*(C'\fR. The final argument to \f(CW\*(C`Newxc\*(C'\fR, \f(CW\*(C`cast\*(C'\fR,
2501 should be used if the \f(CW\*(C`pointer\*(C'\fR argument is different from the \f(CW\*(C`type\*(C'\fR
2502 argument.
2503 .PP
2504 Unlike the \f(CW\*(C`Newx\*(C'\fR and \f(CW\*(C`Newxc\*(C'\fR macros, the \f(CW\*(C`Newxz\*(C'\fR macro calls \f(CW\*(C`memzero\*(C'\fR
2505 to zero out all the newly allocated memory.
2506 .SS Reallocation
2507 .IX Subsection "Reallocation"
2508 .Vb 3
2509 \& Renew(pointer, number, type);
2510 \& Renewc(pointer, number, type, cast);
2511 \& Safefree(pointer)
2512 .Ve
2513 .PP
2514 These three macros are used to change a memory buffer size or to free a
2515 piece of memory no longer needed. The arguments to \f(CW\*(C`Renew\*(C'\fR and \f(CW\*(C`Renewc\*(C'\fR
2516 match those of \f(CW\*(C`New\*(C'\fR and \f(CW\*(C`Newc\*(C'\fR with the exception of not needing the
2517 "magic cookie" argument.
2518 .SS Moving
2519 .IX Subsection "Moving"
2520 .Vb 3
2521 \& Move(source, dest, number, type);
2522 \& Copy(source, dest, number, type);
2523 \& Zero(dest, number, type);
2524 .Ve
2525 .PP
2526 These three macros are used to move, copy, or zero out previously allocated
2527 memory. The \f(CW\*(C`source\*(C'\fR and \f(CW\*(C`dest\*(C'\fR arguments point to the source and
2528 destination starting points. Perl will move, copy, or zero out \f(CW\*(C`number\*(C'\fR
2529 instances of the size of the \f(CW\*(C`type\*(C'\fR data structure (using the \f(CW\*(C`sizeof\*(C'\fR
2530 function).
2531 .SH PerlIO
2532 .IX Header "PerlIO"
2533 The most recent development releases of Perl have been experimenting with
2534 removing Perl\*(Aqs dependency on the "normal" standard I/O suite and allowing
2535 other stdio implementations to be used. This involves creating a new
2536 abstraction layer that then calls whichever implementation of stdio Perl
2537 was compiled with. All XSUBs should now use the functions in the PerlIO
2538 abstraction layer and not make any assumptions about what kind of stdio
2539 is being used.
2540 .PP
2541 For a complete description of the PerlIO abstraction, consult perlapio.
2542 .SH "Compiled code"
2543 .IX Header "Compiled code"
2544 .SS "Code tree"
2545 .IX Subsection "Code tree"
2546 Here we describe the internal form your code is converted to by
2547 Perl. Start with a simple example:
2548 .PP
2549 .Vb 1
2550 \& $a = $b + $c;
2551 .Ve
2552 .PP
2553 This is converted to a tree similar to this one:
2554 .PP
2555 .Vb 5
2556 \& assign\-to
2557 \& / \e
2558 \& + $a
2559 \& / \e
2560 \& $b $c
2561 .Ve
2562 .PP
2563 (but slightly more complicated). This tree reflects the way Perl
2564 parsed your code, but has nothing to do with the execution order.
2565 There is an additional "thread" going through the nodes of the tree
2566 which shows the order of execution of the nodes. In our simplified
2567 example above it looks like:
2568 .PP
2569 .Vb 1
2570 \& $b \-\-\-> $c \-\-\-> + \-\-\-> $a \-\-\-> assign\-to
2571 .Ve
2572 .PP
2573 But with the actual compile tree for \f(CW\*(C`$a = $b + $c\*(C'\fR it is different:
2574 some nodes \fIoptimized away\fR. As a corollary, though the actual tree
2575 contains more nodes than our simplified example, the execution order
2576 is the same as in our example.
2577 .SS "Examining the tree"
2578 .IX Subsection "Examining the tree"
2579 If you have your perl compiled for debugging (usually done with
2580 \&\f(CW\*(C`\-DDEBUGGING\*(C'\fR on the \f(CW\*(C`Configure\*(C'\fR command line), you may examine the
2581 compiled tree by specifying \f(CW\*(C`\-Dx\*(C'\fR on the Perl command line. The
2582 output takes several lines per node, and for \f(CW\*(C`$b+$c\*(C'\fR it looks like
2583 this:
2584 .PP
2585 .Vb 10
2586 \& 5 TYPE = add ===> 6
2587 \& TARG = 1
2588 \& FLAGS = (SCALAR,KIDS)
2589 \& {
2590 \& TYPE = null ===> (4)
2591 \& (was rv2sv)
2592 \& FLAGS = (SCALAR,KIDS)
2593 \& {
2594 \& 3 TYPE = gvsv ===> 4
2595 \& FLAGS = (SCALAR)
2596 \& GV = main::b
2597 \& }
2598 \& }
2599 \& {
2600 \& TYPE = null ===> (5)
2601 \& (was rv2sv)
2602 \& FLAGS = (SCALAR,KIDS)
2603 \& {
2604 \& 4 TYPE = gvsv ===> 5
2605 \& FLAGS = (SCALAR)
2606 \& GV = main::c
2607 \& }
2608 \& }
2609 .Ve
2610 .PP
2611 This tree has 5 nodes (one per \f(CW\*(C`TYPE\*(C'\fR specifier), only 3 of them are
2612 not optimized away (one per number in the left column). The immediate
2613 children of the given node correspond to \f(CW\*(C`{}\*(C'\fR pairs on the same level
2614 of indentation, thus this listing corresponds to the tree:
2615 .PP
2616 .Vb 5
2617 \& add
2618 \& / \e
2619 \& null null
2620 \& | |
2621 \& gvsv gvsv
2622 .Ve
2623 .PP
2624 The execution order is indicated by \f(CW\*(C`===>\*(C'\fR marks, thus it is \f(CW\*(C`3
2625 4 5 6\*(C'\fR (node \f(CW6\fR is not included into above listing), i.e.,
2626 \&\f(CW\*(C`gvsv gvsv add whatever\*(C'\fR.
2627 .PP
2628 Each of these nodes represents an op, a fundamental operation inside the
2629 Perl core. The code which implements each operation can be found in the
2630 \&\fIpp*.c\fR files; the function which implements the op with type \f(CW\*(C`gvsv\*(C'\fR
2631 is \f(CW\*(C`pp_gvsv\*(C'\fR, and so on. As the tree above shows, different ops have
2632 different numbers of children: \f(CW\*(C`add\*(C'\fR is a binary operator, as one would
2633 expect, and so has two children. To accommodate the various different
2634 numbers of children, there are various types of op data structure, and
2635 they link together in different ways.
2636 .PP
2637 The simplest type of op structure is \f(CW\*(C`OP\*(C'\fR: this has no children. Unary
2638 operators, \f(CW\*(C`UNOP\*(C'\fRs, have one child, and this is pointed to by the
2639 \&\f(CW\*(C`op_first\*(C'\fR field. Binary operators (\f(CW\*(C`BINOP\*(C'\fRs) have not only an
2640 \&\f(CW\*(C`op_first\*(C'\fR field but also an \f(CW\*(C`op_last\*(C'\fR field. The most complex type of
2641 op is a \f(CW\*(C`LISTOP\*(C'\fR, which has any number of children. In this case, the
2642 first child is pointed to by \f(CW\*(C`op_first\*(C'\fR and the last child by
2643 \&\f(CW\*(C`op_last\*(C'\fR. The children in between can be found by iteratively
2644 following the \f(CW\*(C`OpSIBLING\*(C'\fR pointer from the first child to the last (but
2645 see below).
2646 .PP
2647 There are also some other op types: a \f(CW\*(C`PMOP\*(C'\fR holds a regular expression,
2648 and has no children, and a \f(CW\*(C`LOOP\*(C'\fR may or may not have children. If the
2649 \&\f(CW\*(C`op_children\*(C'\fR field is non\-zero, it behaves like a \f(CW\*(C`LISTOP\*(C'\fR. To
2650 complicate matters, if a \f(CW\*(C`UNOP\*(C'\fR is actually a \f(CW\*(C`null\*(C'\fR op after
2651 optimization (see "Compile pass 2: context propagation") it will still
2652 have children in accordance with its former type.
2653 .PP
2654 Finally, there is a \f(CW\*(C`LOGOP\*(C'\fR, or logic op. Like a \f(CW\*(C`LISTOP\*(C'\fR, this has one
2655 or more children, but it doesn\*(Aqt have an \f(CW\*(C`op_last\*(C'\fR field: so you have to
2656 follow \f(CW\*(C`op_first\*(C'\fR and then the \f(CW\*(C`OpSIBLING\*(C'\fR chain itself to find the
2657 last child. Instead it has an \f(CW\*(C`op_other\*(C'\fR field, which is comparable to
2658 the \f(CW\*(C`op_next\*(C'\fR field described below, and represents an alternate
2659 execution path. Operators like \f(CW\*(C`and\*(C'\fR, \f(CW\*(C`or\*(C'\fR and \f(CW\*(C`?\*(C'\fR are \f(CW\*(C`LOGOP\*(C'\fRs. Note
2660 that in general, \f(CW\*(C`op_other\*(C'\fR may not point to any of the direct children
2661 of the \f(CW\*(C`LOGOP\*(C'\fR.
2662 .PP
2663 Starting in version 5.21.2, perls built with the experimental
2664 define \f(CW\*(C`\-DPERL_OP_PARENT\*(C'\fR add an extra boolean flag for each op,
2665 \&\f(CW\*(C`op_moresib\*(C'\fR. When not set, this indicates that this is the last op in an
2666 \&\f(CW\*(C`OpSIBLING\*(C'\fR chain. This frees up the \f(CW\*(C`op_sibling\*(C'\fR field on the last
2667 sibling to point back to the parent op. Under this build, that field is
2668 also renamed \f(CW\*(C`op_sibparent\*(C'\fR to reflect its joint role. The macro
2669 \&\f(CWOpSIBLING(o)\fR wraps this special behaviour, and always returns NULL on
2670 the last sibling. With this build the \f(CWop_parent(o)\fR function can be
2671 used to find the parent of any op. Thus for forward compatibility, you
2672 should always use the \f(CWOpSIBLING(o)\fR macro rather than accessing
2673 \&\f(CW\*(C`op_sibling\*(C'\fR directly.
2674 .PP
2675 Another way to examine the tree is to use a compiler back\-end module, such
2676 as B::Concise.
2677 .SS "Compile pass 1: check routines"
2678 .IX Subsection "Compile pass 1: check routines"
2679 The tree is created by the compiler while \fIyacc\fR code feeds it
2680 the constructions it recognizes. Since \fIyacc\fR works bottom\-up, so does
2681 the first pass of perl compilation.
2682 .PP
2683 What makes this pass interesting for perl developers is that some
2684 optimization may be performed on this pass. This is optimization by
2685 so\-called "check routines". The correspondence between node names
2686 and corresponding check routines is described in \fIopcode.pl\fR (do not
2687 forget to run \f(CW\*(C`make regen_headers\*(C'\fR if you modify this file).
2688 .PP
2689 A check routine is called when the node is fully constructed except
2690 for the execution\-order thread. Since at this time there are no
2691 back\-links to the currently constructed node, one can do most any
2692 operation to the top\-level node, including freeing it and/or creating
2693 new nodes above/below it.
2694 .PP
2695 The check routine returns the node which should be inserted into the
2696 tree (if the top\-level node was not modified, check routine returns
2697 its argument).
2698 .PP
2699 By convention, check routines have names \f(CW\*(C`ck_*\*(C'\fR. They are usually
2700 called from \f(CW\*(C`new*OP\*(C'\fR subroutines (or \f(CW\*(C`convert\*(C'\fR) (which in turn are
2701 called from \fIperly.y\fR).
2702 .SS "Compile pass 1a: constant folding"
2703 .IX Subsection "Compile pass 1a: constant folding"
2704 Immediately after the check routine is called the returned node is
2705 checked for being compile\-time executable. If it is (the value is
2706 judged to be constant) it is immediately executed, and a \fIconstant\fR
2707 node with the "return value" of the corresponding subtree is
2708 substituted instead. The subtree is deleted.
2709 .PP
2710 If constant folding was not performed, the execution\-order thread is
2711 created.
2712 .SS "Compile pass 2: context propagation"
2713 .IX Subsection "Compile pass 2: context propagation"
2714 When a context for a part of compile tree is known, it is propagated
2715 down through the tree. At this time the context can have 5 values
2716 (instead of 2 for runtime context): void, boolean, scalar, list, and
2717 lvalue. In contrast with the pass 1 this pass is processed from top
2718 to bottom: a node\*(Aqs context determines the context for its children.
2719 .PP
2720 Additional context\-dependent optimizations are performed at this time.
2721 Since at this moment the compile tree contains back\-references (via
2722 "thread" pointers), nodes cannot be \fBfree()\fRd now. To allow
2723 optimized\-away nodes at this stage, such nodes are \fBnull()\fRified instead
2724 of \fBfree()\fRing (i.e. their type is changed to OP_NULL).
2725 .SS "Compile pass 3: peephole optimization"
2726 .IX Subsection "Compile pass 3: peephole optimization"
2727 After the compile tree for a subroutine (or for an \f(CW\*(C`eval\*(C'\fR or a file)
2728 is created, an additional pass over the code is performed. This pass
2729 is neither top\-down nor bottom\-up, but in the execution order (with
2730 additional complications for conditionals). Optimizations performed
2731 at this stage are subject to the same restrictions as in the pass 2.
2732 .PP
2733 Peephole optimizations are done by calling the function pointed to
2734 by the global variable \f(CW\*(C`PL_peepp\*(C'\fR. By default, \f(CW\*(C`PL_peepp\*(C'\fR just
2735 calls the function pointed to by the global variable \f(CW\*(C`PL_rpeepp\*(C'\fR.
2736 By default, that performs some basic op fixups and optimisations along
2737 the execution\-order op chain, and recursively calls \f(CW\*(C`PL_rpeepp\*(C'\fR for
2738 each side chain of ops (resulting from conditionals). Extensions may
2739 provide additional optimisations or fixups, hooking into either the
2740 per\-subroutine or recursive stage, like this:
2741 .PP
2742 .Vb 10
2743 \& static peep_t prev_peepp;
2744 \& static void my_peep(pTHX_ OP *o)
2745 \& {
2746 \& /* custom per\-subroutine optimisation goes here */
2747 \& prev_peepp(aTHX_ o);
2748 \& /* custom per\-subroutine optimisation may also go here */
2749 \& }
2750 \& BOOT:
2751 \& prev_peepp = PL_peepp;
2752 \& PL_peepp = my_peep;
2753 \&
2754 \& static peep_t prev_rpeepp;
2755 \& static void my_rpeep(pTHX_ OP *first)
2756 \& {
2757 \& OP *o = first, *t = first;
2758 \& for(; o = o\->op_next, t = t\->op_next) {
2759 \& /* custom per\-op optimisation goes here */
2760 \& o = o\->op_next;
2761 \& if (!o || o == t) break;
2762 \& /* custom per\-op optimisation goes AND here */
2763 \& }
2764 \& prev_rpeepp(aTHX_ orig_o);
2765 \& }
2766 \& BOOT:
2767 \& prev_rpeepp = PL_rpeepp;
2768 \& PL_rpeepp = my_rpeep;
2769 .Ve
2770 .SS "Pluggable runops"
2771 .IX Subsection "Pluggable runops"
2772 The compile tree is executed in a runops function. There are two runops
2773 functions, in \fIrun.c\fR and in \fIdump.c\fR. \f(CW\*(C`Perl_runops_debug\*(C'\fR is used
2774 with DEBUGGING and \f(CW\*(C`Perl_runops_standard\*(C'\fR is used otherwise. For fine
2775 control over the execution of the compile tree it is possible to provide
2776 your own runops function.
2777 .PP
2778 It\*(Aqs probably best to copy one of the existing runops functions and
2779 change it to suit your needs. Then, in the BOOT section of your XS
2780 file, add the line:
2781 .PP
2782 .Vb 1
2783 \& PL_runops = my_runops;
2784 .Ve
2785 .PP
2786 This function should be as efficient as possible to keep your programs
2787 running as fast as possible.
2788 .SS "Compile\-time scope hooks"
2789 .IX Subsection "Compile-time scope hooks"
2790 As of perl 5.14 it is possible to hook into the compile\-time lexical
2791 scope mechanism using \f(CW\*(C`Perl_blockhook_register\*(C'\fR. This is used like
2792 this:
2793 .PP
2794 .Vb 2
2795 \& STATIC void my_start_hook(pTHX_ int full);
2796 \& STATIC BHK my_hooks;
2797 \&
2798 \& BOOT:
2799 \& BhkENTRY_set(&my_hooks, bhk_start, my_start_hook);
2800 \& Perl_blockhook_register(aTHX_ &my_hooks);
2801 .Ve
2802 .PP
2803 This will arrange to have \f(CW\*(C`my_start_hook\*(C'\fR called at the start of
2804 compiling every lexical scope. The available hooks are:
2805 .ie n .IP """void bhk_start(pTHX_ int full)""" 4
2806 .el .IP "\f(CWvoid bhk_start(pTHX_ int full)\fR" 4
2807 .IX Item "void bhk_start(pTHX_ int full)"
2808 This is called just after starting a new lexical scope. Note that Perl
2809 code like
2810 .Sp
2811 .Vb 1
2812 \& if ($x) { ... }
2813 .Ve
2814 .Sp
2815 creates two scopes: the first starts at the \f(CW\*(C`(\*(C'\fR and has \f(CW\*(C`full == 1\*(C'\fR,
2816 the second starts at the \f(CW\*(C`{\*(C'\fR and has \f(CW\*(C`full == 0\*(C'\fR. Both end at the
2817 \&\f(CW\*(C`}\*(C'\fR, so calls to \f(CW\*(C`start\*(C'\fR and \f(CW\*(C`pre\*(C'\fR/\f(CW\*(C`post_end\*(C'\fR will match. Anything
2818 pushed onto the save stack by this hook will be popped just before the
2819 scope ends (between the \f(CW\*(C`pre_\*(C'\fR and \f(CW\*(C`post_end\*(C'\fR hooks, in fact).
2820 .ie n .IP """void bhk_pre_end(pTHX_ OP **o)""" 4
2821 .el .IP "\f(CWvoid bhk_pre_end(pTHX_ OP **o)\fR" 4
2822 .IX Item "void bhk_pre_end(pTHX_ OP **o)"
2823 This is called at the end of a lexical scope, just before unwinding the
2824 stack. \fIo\fR is the root of the optree representing the scope; it is a
2825 double pointer so you can replace the OP if you need to.
2826 .ie n .IP """void bhk_post_end(pTHX_ OP **o)""" 4
2827 .el .IP "\f(CWvoid bhk_post_end(pTHX_ OP **o)\fR" 4
2828 .IX Item "void bhk_post_end(pTHX_ OP **o)"
2829 This is called at the end of a lexical scope, just after unwinding the
2830 stack. \fIo\fR is as above. Note that it is possible for calls to \f(CW\*(C`pre_\*(C'\fR
2831 and \f(CW\*(C`post_end\*(C'\fR to nest, if there is something on the save stack that
2832 calls string eval.
2833 .ie n .IP """void bhk_eval(pTHX_ OP *const o)""" 4
2834 .el .IP "\f(CWvoid bhk_eval(pTHX_ OP *const o)\fR" 4
2835 .IX Item "void bhk_eval(pTHX_ OP *const o)"
2836 This is called just before starting to compile an \f(CW\*(C`eval STRING\*(C'\fR, \f(CW\*(C`do
2837 FILE\*(C'\fR, \f(CW\*(C`require\*(C'\fR or \f(CW\*(C`use\*(C'\fR, after the eval has been set up. \fIo\fR is the
2838 OP that requested the eval, and will normally be an \f(CW\*(C`OP_ENTEREVAL\*(C'\fR,
2839 \&\f(CW\*(C`OP_DOFILE\*(C'\fR or \f(CW\*(C`OP_REQUIRE\*(C'\fR.
2840 .PP
2841 Once you have your hook functions, you need a \f(CW\*(C`BHK\*(C'\fR structure to put
2842 them in. It\*(Aqs best to allocate it statically, since there is no way to
2843 free it once it\*(Aqs registered. The function pointers should be inserted
2844 into this structure using the \f(CW\*(C`BhkENTRY_set\*(C'\fR macro, which will also set
2845 flags indicating which entries are valid. If you do need to allocate
2846 your \f(CW\*(C`BHK\*(C'\fR dynamically for some reason, be sure to zero it before you
2847 start.
2848 .PP
2849 Once registered, there is no mechanism to switch these hooks off, so if
2850 that is necessary you will need to do this yourself. An entry in \f(CW\*(C`%^H\*(C'\fR
2851 is probably the best way, so the effect is lexically scoped; however it
2852 is also possible to use the \f(CW\*(C`BhkDISABLE\*(C'\fR and \f(CW\*(C`BhkENABLE\*(C'\fR macros to
2853 temporarily switch entries on and off. You should also be aware that
2854 generally speaking at least one scope will have opened before your
2855 extension is loaded, so you will see some \f(CW\*(C`pre\*(C'\fR/\f(CW\*(C`post_end\*(C'\fR pairs that
2856 didn\*(Aqt have a matching \f(CW\*(C`start\*(C'\fR.
2857 .ie n .SH "Examining internal data structures with the ""dump"" functions"
2858 .el .SH "Examining internal data structures with the \f(CWdump\fP functions"
2859 .IX Header "Examining internal data structures with the dump functions"
2860 To aid debugging, the source file \fIdump.c\fR contains a number of
2861 functions which produce formatted output of internal data structures.
2862 .PP
2863 The most commonly used of these functions is \f(CW\*(C`Perl_sv_dump\*(C'\fR; it\*(Aqs used
2864 for dumping SVs, AVs, HVs, and CVs. The \f(CW\*(C`Devel::Peek\*(C'\fR module calls
2865 \&\f(CW\*(C`sv_dump\*(C'\fR to produce debugging output from Perl\-space, so users of that
2866 module should already be familiar with its format.
2867 .PP
2868 \&\f(CW\*(C`Perl_op_dump\*(C'\fR can be used to dump an \f(CW\*(C`OP\*(C'\fR structure or any of its
2869 derivatives, and produces output similar to \f(CW\*(C`perl \-Dx\*(C'\fR; in fact,
2870 \&\f(CW\*(C`Perl_dump_eval\*(C'\fR will dump the main root of the code being evaluated,
2871 exactly like \f(CW\*(C`\-Dx\*(C'\fR.
2872 .PP
2873 Other useful functions are \f(CW\*(C`Perl_dump_sub\*(C'\fR, which turns a \f(CW\*(C`GV\*(C'\fR into an
2874 op tree, \f(CW\*(C`Perl_dump_packsubs\*(C'\fR which calls \f(CW\*(C`Perl_dump_sub\*(C'\fR on all the
2875 subroutines in a package like so: (Thankfully, these are all xsubs, so
2876 there is no op tree)
2877 .PP
2878 .Vb 1
2879 \& (gdb) print Perl_dump_packsubs(PL_defstash)
2880 \&
2881 \& SUB attributes::bootstrap = (xsub 0x811fedc 0)
2882 \&
2883 \& SUB UNIVERSAL::can = (xsub 0x811f50c 0)
2884 \&
2885 \& SUB UNIVERSAL::isa = (xsub 0x811f304 0)
2886 \&
2887 \& SUB UNIVERSAL::VERSION = (xsub 0x811f7ac 0)
2888 \&
2889 \& SUB DynaLoader::boot_DynaLoader = (xsub 0x805b188 0)
2890 .Ve
2891 .PP
2892 and \f(CW\*(C`Perl_dump_all\*(C'\fR, which dumps all the subroutines in the stash and
2893 the op tree of the main root.
2894 .SH "How multiple interpreters and concurrency are supported"
2895 .IX Header "How multiple interpreters and concurrency are supported"
2896 .SS "Background and MULTIPLICITY"
2897 .IX Subsection "Background and MULTIPLICITY"
2898 The Perl interpreter can be regarded as a closed box: it has an API
2899 for feeding it code or otherwise making it do things, but it also has
2900 functions for its own use. This smells a lot like an object, and
2901 there is a way for you to build Perl so that you can have multiple
2902 interpreters, with one interpreter represented either as a C structure,
2903 or inside a thread\-specific structure. These structures contain all
2904 the context, the state of that interpreter.
2905 .PP
2906 The macro that controls the major Perl build flavor is MULTIPLICITY. The
2907 MULTIPLICITY build has a C structure that packages all the interpreter
2908 state, which is being passed to various perl functions as a "hidden"
2909 first argument. MULTIPLICITY makes multi\-threaded perls possible (with the
2910 ithreads threading model, related to the macro USE_ITHREADS.)
2911 .PP
2912 PERL_IMPLICIT_CONTEXT is a legacy synonym for MULTIPLICITY.
2913 .PP
2914 To see whether you have non\-const data you can use a BSD (or GNU)
2915 compatible \f(CW\*(C`nm\*(C'\fR:
2916 .PP
2917 .Vb 1
2918 \& nm libperl.a | grep \-v \*(Aq [TURtr] \*(Aq
2919 .Ve
2920 .PP
2921 If this displays any \f(CW\*(C`D\*(C'\fR or \f(CW\*(C`d\*(C'\fR symbols (or possibly \f(CW\*(C`C\*(C'\fR or \f(CW\*(C`c\*(C'\fR),
2922 you have non\-const data. The symbols the \f(CW\*(C`grep\*(C'\fR removed are as follows:
2923 \&\f(CW\*(C`Tt\*(C'\fR are \fItext\fR, or code, the \f(CW\*(C`Rr\*(C'\fR are \fIread\-only\fR (const) data,
2924 and the \f(CW\*(C`U\*(C'\fR is <undefined>, external symbols referred to.
2925 .PP
2926 The test \fIt/porting/libperl.t\fR does this kind of symbol sanity
2927 checking on \f(CW\*(C`libperl.a\*(C'\fR.
2928 .PP
2929 All this obviously requires a way for the Perl internal functions to be
2930 either subroutines taking some kind of structure as the first
2931 argument, or subroutines taking nothing as the first argument. To
2932 enable these two very different ways of building the interpreter,
2933 the Perl source (as it does in so many other situations) makes heavy
2934 use of macros and subroutine naming conventions.
2935 .PP
2936 First problem: deciding which functions will be public API functions and
2937 which will be private. All functions whose names begin \f(CW\*(C`S_\*(C'\fR are private
2938 (think "S" for "secret" or "static"). All other functions begin with
2939 "Perl_", but just because a function begins with "Perl_" does not mean it is
2940 part of the API. (See "Internal
2941 Functions".) The easiest way to be \fBsure\fR a
2942 function is part of the API is to find its entry in perlapi.
2943 If it exists in perlapi, it\*(Aqs part of the API. If it doesn\*(Aqt, and you
2944 think it should be (i.e., you need it for your extension), submit an issue at
2945 <https://github.com/Perl/perl5/issues> explaining why you think it should be.
2946 .PP
2947 Second problem: there must be a syntax so that the same subroutine
2948 declarations and calls can pass a structure as their first argument,
2949 or pass nothing. To solve this, the subroutines are named and
2950 declared in a particular way. Here\*(Aqs a typical start of a static
2951 function used within the Perl guts:
2952 .PP
2953 .Vb 2
2954 \& STATIC void
2955 \& S_incline(pTHX_ char *s)
2956 .Ve
2957 .PP
2958 STATIC becomes "static" in C, and may be #define\*(Aqd to nothing in some
2959 configurations in the future.
2960 .PP
2961 A public function (i.e. part of the internal API, but not necessarily
2962 sanctioned for use in extensions) begins like this:
2963 .PP
2964 .Vb 2
2965 \& void
2966 \& Perl_sv_setiv(pTHX_ SV* dsv, IV num)
2967 .Ve
2968 .PP
2969 \&\f(CW\*(C`pTHX_\*(C'\fR is one of a number of macros (in \fIperl.h\fR) that hide the
2970 details of the interpreter\*(Aqs context. THX stands for "thread", "this",
2971 or "thingy", as the case may be. (And no, George Lucas is not involved. :\-)
2972 The first character could be \*(Aqp\*(Aq for a \fBp\fRrototype, \*(Aqa\*(Aq for \fBa\fRrgument,
2973 or \*(Aqd\*(Aq for \fBd\fReclaration, so we have \f(CW\*(C`pTHX\*(C'\fR, \f(CW\*(C`aTHX\*(C'\fR and \f(CW\*(C`dTHX\*(C'\fR, and
2974 their variants.
2975 .PP
2976 When Perl is built without options that set MULTIPLICITY, there is no
2977 first argument containing the interpreter\*(Aqs context. The trailing underscore
2978 in the pTHX_ macro indicates that the macro expansion needs a comma
2979 after the context argument because other arguments follow it. If
2980 MULTIPLICITY is not defined, pTHX_ will be ignored, and the
2981 subroutine is not prototyped to take the extra argument. The form of the
2982 macro without the trailing underscore is used when there are no additional
2983 explicit arguments.
2984 .PP
2985 When a core function calls another, it must pass the context. This
2986 is normally hidden via macros. Consider \f(CW\*(C`sv_setiv\*(C'\fR. It expands into
2987 something like this:
2988 .PP
2989 .Vb 6
2990 \& #ifdef MULTIPLICITY
2991 \& #define sv_setiv(a,b) Perl_sv_setiv(aTHX_ a, b)
2992 \& /* can\*(Aqt do this for vararg functions, see below */
2993 \& #else
2994 \& #define sv_setiv Perl_sv_setiv
2995 \& #endif
2996 .Ve
2997 .PP
2998 This works well, and means that XS authors can gleefully write:
2999 .PP
3000 .Vb 1
3001 \& sv_setiv(foo, bar);
3002 .Ve
3003 .PP
3004 and still have it work under all the modes Perl could have been
3005 compiled with.
3006 .PP
3007 This doesn\*(Aqt work so cleanly for varargs functions, though, as macros
3008 imply that the number of arguments is known in advance. Instead we
3009 either need to spell them out fully, passing \f(CW\*(C`aTHX_\*(C'\fR as the first
3010 argument (the Perl core tends to do this with functions like
3011 Perl_warner), or use a context\-free version.
3012 .PP
3013 The context\-free version of Perl_warner is called
3014 Perl_warner_nocontext, and does not take the extra argument. Instead
3015 it does \f(CW\*(C`dTHX;\*(C'\fR to get the context from thread\-local storage. We
3016 \&\f(CW\*(C`#define warner Perl_warner_nocontext\*(C'\fR so that extensions get source
3017 compatibility at the expense of performance. (Passing an arg is
3018 cheaper than grabbing it from thread\-local storage.)
3019 .PP
3020 You can ignore [pad]THXx when browsing the Perl headers/sources.
3021 Those are strictly for use within the core. Extensions and embedders
3022 need only be aware of [pad]THX.
3023 .SS "So what happened to dTHR?"
3024 .IX Subsection "So what happened to dTHR?"
3025 \&\f(CW\*(C`dTHR\*(C'\fR was introduced in perl 5.005 to support the older thread model.
3026 The older thread model now uses the \f(CW\*(C`THX\*(C'\fR mechanism to pass context
3027 pointers around, so \f(CW\*(C`dTHR\*(C'\fR is not useful any more. Perl 5.6.0 and
3028 later still have it for backward source compatibility, but it is defined
3029 to be a no\-op.
3030 .SS "How do I use all this in extensions?"
3031 .IX Subsection "How do I use all this in extensions?"
3032 See also "Dealing with embedded perls and threads" in perlclib.
3033 .PP
3034 When Perl is built with MULTIPLICITY, extensions that call
3035 any functions in the Perl API will need to pass the initial context
3036 argument somehow. The kicker is that you will need to write it in
3037 such a way that the extension still compiles when Perl hasn\*(Aqt been
3038 built with MULTIPLICITY enabled.
3039 .PP
3040 There are three ways to do this. First, the easy but inefficient way,
3041 which is also the default, in order to maintain source compatibility
3042 with extensions: whenever \fIXSUB.h\fR is #included, it redefines the aTHX
3043 and aTHX_ macros to call a function that will return the context.
3044 Thus, something like:
3045 .PP
3046 .Vb 1
3047 \& sv_setiv(sv, num);
3048 .Ve
3049 .PP
3050 in your extension will translate to this when MULTIPLICITY is
3051 in effect:
3052 .PP
3053 .Vb 1
3054 \& Perl_sv_setiv(Perl_get_context(), sv, num);
3055 .Ve
3056 .PP
3057 or to this otherwise:
3058 .PP
3059 .Vb 1
3060 \& Perl_sv_setiv(sv, num);
3061 .Ve
3062 .PP
3063 You don\*(Aqt have to do anything new in your extension to get this; since
3064 the Perl library provides \fBPerl_get_context()\fR, it will all just
3065 work.
3066 .PP
3067 The second, more efficient way is to use the following template for
3068 your Foo.xs:
3069 .PP
3070 .Vb 4
3071 \& #define PERL_NO_GET_CONTEXT /* we want efficiency */
3072 \& #include "EXTERN.h"
3073 \& #include "perl.h"
3074 \& #include "XSUB.h"
3075 \&
3076 \& STATIC void my_private_function(int arg1, int arg2);
3077 \&
3078 \& STATIC void
3079 \& my_private_function(int arg1, int arg2)
3080 \& {
3081 \& dTHX; /* fetch context */
3082 \& ... call many Perl API functions ...
3083 \& }
3084 \&
3085 \& [... etc ...]
3086 \&
3087 \& MODULE = Foo PACKAGE = Foo
3088 \&
3089 \& /* typical XSUB */
3090 \&
3091 \& void
3092 \& my_xsub(arg)
3093 \& int arg
3094 \& CODE:
3095 \& my_private_function(arg, 10);
3096 .Ve
3097 .PP
3098 Note that the only two changes from the normal way of writing an
3099 extension is the addition of a \f(CW\*(C`#define PERL_NO_GET_CONTEXT\*(C'\fR before
3100 including the Perl headers, followed by a \f(CW\*(C`dTHX;\*(C'\fR declaration at
3101 the start of every function that will call the Perl API. (You\*(Aqll
3102 know which functions need this, because the C compiler will complain
3103 that there\*(Aqs an undeclared identifier in those functions.) No changes
3104 are needed for the XSUBs themselves, because the \fBXS()\fR macro is
3105 correctly defined to pass in the implicit context if needed.
3106 .PP
3107 The third, even more efficient way is to ape how it is done within
3108 the Perl guts:
3109 .PP
3110 .Vb 4
3111 \& #define PERL_NO_GET_CONTEXT /* we want efficiency */
3112 \& #include "EXTERN.h"
3113 \& #include "perl.h"
3114 \& #include "XSUB.h"
3115 \&
3116 \& /* pTHX_ only needed for functions that call Perl API */
3117 \& STATIC void my_private_function(pTHX_ int arg1, int arg2);
3118 \&
3119 \& STATIC void
3120 \& my_private_function(pTHX_ int arg1, int arg2)
3121 \& {
3122 \& /* dTHX; not needed here, because THX is an argument */
3123 \& ... call Perl API functions ...
3124 \& }
3125 \&
3126 \& [... etc ...]
3127 \&
3128 \& MODULE = Foo PACKAGE = Foo
3129 \&
3130 \& /* typical XSUB */
3131 \&
3132 \& void
3133 \& my_xsub(arg)
3134 \& int arg
3135 \& CODE:
3136 \& my_private_function(aTHX_ arg, 10);
3137 .Ve
3138 .PP
3139 This implementation never has to fetch the context using a function
3140 call, since it is always passed as an extra argument. Depending on
3141 your needs for simplicity or efficiency, you may mix the previous
3142 two approaches freely.
3143 .PP
3144 Never add a comma after \f(CW\*(C`pTHX\*(C'\fR yourself\-\-always use the form of the
3145 macro with the underscore for functions that take explicit arguments,
3146 or the form without the argument for functions with no explicit arguments.
3147 .SS "Should I do anything special if I call perl from multiple threads?"
3148 .IX Subsection "Should I do anything special if I call perl from multiple threads?"
3149 If you create interpreters in one thread and then proceed to call them in
3150 another, you need to make sure perl\*(Aqs own Thread Local Storage (TLS) slot is
3151 initialized correctly in each of those threads.
3152 .PP
3153 The \f(CW\*(C`perl_alloc\*(C'\fR and \f(CW\*(C`perl_clone\*(C'\fR API functions will automatically set
3154 the TLS slot to the interpreter they created, so that there is no need to do
3155 anything special if the interpreter is always accessed in the same thread that
3156 created it, and that thread did not create or call any other interpreters
3157 afterwards. If that is not the case, you have to set the TLS slot of the
3158 thread before calling any functions in the Perl API on that particular
3159 interpreter. This is done by calling the \f(CW\*(C`PERL_SET_CONTEXT\*(C'\fR macro in that
3160 thread as the first thing you do:
3161 .PP
3162 .Vb 2
3163 \& /* do this before doing anything else with some_perl */
3164 \& PERL_SET_CONTEXT(some_perl);
3165 \&
3166 \& ... other Perl API calls on some_perl go here ...
3167 .Ve
3168 .PP
3169 (You can always get the current context via \f(CW\*(C`PERL_GET_CONTEXT\*(C'\fR.)
3170 .SS "Future Plans and PERL_IMPLICIT_SYS"
3171 .IX Subsection "Future Plans and PERL_IMPLICIT_SYS"
3172 Just as MULTIPLICITY provides a way to bundle up everything
3173 that the interpreter knows about itself and pass it around, so too are
3174 there plans to allow the interpreter to bundle up everything it knows
3175 about the environment it\*(Aqs running on. This is enabled with the
3176 PERL_IMPLICIT_SYS macro. Currently it only works with USE_ITHREADS on
3177 Windows.
3178 .PP
3179 This allows the ability to provide an extra pointer (called the "host"
3180 environment) for all the system calls. This makes it possible for
3181 all the system stuff to maintain their own state, broken down into
3182 seven C structures. These are thin wrappers around the usual system
3183 calls (see \fIwin32/perllib.c\fR) for the default perl executable, but for a
3184 more ambitious host (like the one that would do \fBfork()\fR emulation) all
3185 the extra work needed to pretend that different interpreters are
3186 actually different "processes", would be done here.
3187 .PP
3188 The Perl engine/interpreter and the host are orthogonal entities.
3189 There could be one or more interpreters in a process, and one or
3190 more "hosts", with free association between them.
3191 .SH "Internal Functions"
3192 .IX Header "Internal Functions"
3193 All of Perl\*(Aqs internal functions which will be exposed to the outside
3194 world are prefixed by \f(CW\*(C`Perl_\*(C'\fR so that they will not conflict with XS
3195 functions or functions used in a program in which Perl is embedded.
3196 Similarly, all global variables begin with \f(CW\*(C`PL_\*(C'\fR. (By convention,
3197 static functions start with \f(CW\*(C`S_\*(C'\fR.)
3198 .PP
3199 Inside the Perl core (\f(CW\*(C`PERL_CORE\*(C'\fR defined), you can get at the functions
3200 either with or without the \f(CW\*(C`Perl_\*(C'\fR prefix, thanks to a bunch of defines
3201 that live in \fIembed.h\fR. Note that extension code should \fInot\fR set
3202 \&\f(CW\*(C`PERL_CORE\*(C'\fR; this exposes the full perl internals, and is likely to cause
3203 breakage of the XS in each new perl release.
3204 .PP
3205 The file \fIembed.h\fR is generated automatically from
3206 \&\fIembed.pl\fR and \fIembed.fnc\fR. \fIembed.pl\fR also creates the prototyping
3207 header files for the internal functions, generates the documentation
3208 and a lot of other bits and pieces. It\*(Aqs important that when you add
3209 a new function to the core or change an existing one, you change the
3210 data in the table in \fIembed.fnc\fR as well. Here\*(Aqs a sample entry from
3211 that table:
3212 .PP
3213 .Vb 1
3214 \& Apd |SV** |av_fetch |AV* ar|I32 key|I32 lval
3215 .Ve
3216 .PP
3217 The first column is a set of flags, the second column the return type,
3218 the third column the name. Columns after that are the arguments.
3219 The flags are documented at the top of \fIembed.fnc\fR.
3220 .PP
3221 If you edit \fIembed.pl\fR or \fIembed.fnc\fR, you will need to run
3222 \&\f(CW\*(C`make regen_headers\*(C'\fR to force a rebuild of \fIembed.h\fR and other
3223 auto\-generated files.
3224 .SS "Formatted Printing of IVs, UVs, and NVs"
3225 .IX Subsection "Formatted Printing of IVs, UVs, and NVs"
3226 If you are printing IVs, UVs, or NVS instead of the \fBstdio\fR\|(3) style
3227 formatting codes like \f(CW%d\fR, \f(CW%ld\fR, \f(CW%f\fR, you should use the
3228 following macros for portability
3229 .PP
3230 .Vb 7
3231 \& IVdf IV in decimal
3232 \& UVuf UV in decimal
3233 \& UVof UV in octal
3234 \& UVxf UV in hexadecimal
3235 \& NVef NV %e\-like
3236 \& NVff NV %f\-like
3237 \& NVgf NV %g\-like
3238 .Ve
3239 .PP
3240 These will take care of 64\-bit integers and long doubles.
3241 For example:
3242 .PP
3243 .Vb 1
3244 \& printf("IV is %" IVdf "\en", iv);
3245 .Ve
3246 .PP
3247 The \f(CW\*(C`IVdf\*(C'\fR will expand to whatever is the correct format for the IVs.
3248 Note that the spaces are required around the format in case the code is
3249 compiled with C++, to maintain compliance with its standard.
3250 .PP
3251 Note that there are different "long doubles": Perl will use
3252 whatever the compiler has.
3253 .PP
3254 If you are printing addresses of pointers, use \f(CW%p\fR or UVxf combined
3255 with \fBPTR2UV()\fR.
3256 .SS "Formatted Printing of SVs"
3257 .IX Subsection "Formatted Printing of SVs"
3258 The contents of SVs may be printed using the \f(CW\*(C`SVf\*(C'\fR format, like so:
3259 .PP
3260 .Vb 1
3261 \& Perl_croak(aTHX_ "This croaked because: %" SVf "\en", SVfARG(err_msg))
3262 .Ve
3263 .PP
3264 where \f(CW\*(C`err_msg\*(C'\fR is an SV.
3265 .PP
3266 Not all scalar types are printable. Simple values certainly are: one of
3267 IV, UV, NV, or PV. Also, if the SV is a reference to some value,
3268 either it will be dereferenced and the value printed, or information
3269 about the type of that value and its address are displayed. The results
3270 of printing any other type of SV are undefined and likely to lead to an
3271 interpreter crash. NVs are printed using a \f(CW%g\fR\-ish format.
3272 .PP
3273 Note that the spaces are required around the \f(CW\*(C`SVf\*(C'\fR in case the code is
3274 compiled with C++, to maintain compliance with its standard.
3275 .PP
3276 Note that any filehandle being printed to under UTF\-8 must be expecting
3277 UTF\-8 in order to get good results and avoid Wide\-character warnings.
3278 One way to do this for typical filehandles is to invoke perl with the
3279 \&\f(CW\*(C`\-C\*(C'\fR parameter. (See "\-C [number/list]" in perlrun.
3280 .PP
3281 You can use this to concatenate two scalars:
3282 .PP
3283 .Vb 4
3284 \& SV *var1 = get_sv("var1", GV_ADD);
3285 \& SV *var2 = get_sv("var2", GV_ADD);
3286 \& SV *var3 = newSVpvf("var1=%" SVf " and var2=%" SVf,
3287 \& SVfARG(var1), SVfARG(var2));
3288 .Ve
3289 .PP
3290 \&\f(CW\*(C`SVf_QUOTEDPREFIX\*(C'\fR is similar to \f(CW\*(C`SVf\*(C'\fR except that it restricts the
3291 number of the characters printed, showing at most the first
3292 \&\f(CW\*(C`PERL_QUOTEDPREFIX_LEN\*(C'\fR characters of the argument, and rendering it with
3293 double quotes and with the contents escaped using double quoted string
3294 escaping rules. If the string is longer than this then ellipses "..."
3295 will be appended after the trailing quote. This is intended for error
3296 messages where the string is assumed to be a class name.
3297 .PP
3298 \&\f(CW\*(C`HvNAMEf\*(C'\fR and \f(CW\*(C`HvNAMEf_QUOTEDPREFIX\*(C'\fR are similar to \f(CW\*(C`SVf\*(C'\fR except they
3299 extract the string, length and utf8 flags from the argument using the
3300 \&\f(CWHvNAME()\fR, \f(CWHvNAMELEN()\fR, \f(CWHvNAMEUTF8()\fR macros. This is intended
3301 for stringifying a class name directly from an stash HV.
3302 .SS "Formatted Printing of Strings"
3303 .IX Subsection "Formatted Printing of Strings"
3304 If you just want the bytes printed in a 7bit NUL\-terminated string, you can
3305 just use \f(CW%s\fR (assuming they are all really only 7bit). But if there is a
3306 possibility the value will be encoded as UTF\-8 or contains bytes above
3307 \&\f(CW0x7F\fR (and therefore 8bit), you should instead use the \f(CW\*(C`UTF8f\*(C'\fR format.
3308 And as its parameter, use the \f(CWUTF8fARG()\fR macro:
3309 .PP
3310 .Vb 1
3311 \& chr * msg;
3312 \&
3313 \& /* U+2018: \exE2\ex80\ex98 LEFT SINGLE QUOTATION MARK
3314 \& U+2019: \exE2\ex80\ex99 RIGHT SINGLE QUOTATION MARK */
3315 \& if (can_utf8)
3316 \& msg = "\exE2\ex80\ex98Uses fancy quotes\exE2\ex80\ex99";
3317 \& else
3318 \& msg = "\*(AqUses simple quotes\*(Aq";
3319 \&
3320 \& Perl_croak(aTHX_ "The message is: %" UTF8f "\en",
3321 \& UTF8fARG(can_utf8, strlen(msg), msg));
3322 .Ve
3323 .PP
3324 The first parameter to \f(CW\*(C`UTF8fARG\*(C'\fR is a boolean: 1 if the string is in
3325 UTF\-8; 0 if string is in native byte encoding (Latin1).
3326 The second parameter is the number of bytes in the string to print.
3327 And the third and final parameter is a pointer to the first byte in the
3328 string.
3329 .PP
3330 Note that any filehandle being printed to under UTF\-8 must be expecting
3331 UTF\-8 in order to get good results and avoid Wide\-character warnings.
3332 One way to do this for typical filehandles is to invoke perl with the
3333 \&\f(CW\*(C`\-C\*(C'\fR parameter. (See "\-C [number/list]" in perlrun.
3334 .ie n .SS "Formatted Printing of ""Size_t"" and ""SSize_t"""
3335 .el .SS "Formatted Printing of \f(CWSize_t\fP and \f(CWSSize_t\fP"
3336 .IX Subsection "Formatted Printing of Size_t and SSize_t"
3337 The most general way to do this is to cast them to a UV or IV, and
3338 print as in the
3339 previous section.
3340 .PP
3341 But if you\*(Aqre using \f(CWPerlIO_printf()\fR, it\*(Aqs less typing and visual
3342 clutter to use the \f(CW%z\fR length modifier (for \fIsiZe\fR):
3343 .PP
3344 .Vb 1
3345 \& PerlIO_printf("STRLEN is %zu\en", len);
3346 .Ve
3347 .PP
3348 This modifier is not portable, so its use should be restricted to
3349 \&\f(CWPerlIO_printf()\fR.
3350 .ie n .SS "Formatted Printing of ""Ptrdiff_t"", ""intmax_t"", ""short"" and other special sizes"
3351 .el .SS "Formatted Printing of \f(CWPtrdiff_t\fP, \f(CWintmax_t\fP, \f(CWshort\fP and other special sizes"
3352 .IX Subsection "Formatted Printing of Ptrdiff_t, intmax_t, short and other special sizes"
3353 There are modifiers for these special situations if you are using
3354 \&\f(CWPerlIO_printf()\fR. See "size" in perlfunc.
3355 .SS "Pointer\-To\-Integer and Integer\-To\-Pointer"
3356 .IX Subsection "Pointer-To-Integer and Integer-To-Pointer"
3357 Because pointer size does not necessarily equal integer size,
3358 use the follow macros to do it right.
3359 .PP
3360 .Vb 4
3361 \& PTR2UV(pointer)
3362 \& PTR2IV(pointer)
3363 \& PTR2NV(pointer)
3364 \& INT2PTR(pointertotype, integer)
3365 .Ve
3366 .PP
3367 For example:
3368 .PP
3369 .Vb 2
3370 \& IV iv = ...;
3371 \& SV *sv = INT2PTR(SV*, iv);
3372 .Ve
3373 .PP
3374 and
3375 .PP
3376 .Vb 2
3377 \& AV *av = ...;
3378 \& UV uv = PTR2UV(av);
3379 .Ve
3380 .PP
3381 There are also
3382 .PP
3383 .Vb 2
3384 \& PTR2nat(pointer) /* pointer to integer of PTRSIZE */
3385 \& PTR2ul(pointer) /* pointer to unsigned long (unsafe) */
3386 .Ve
3387 .PP
3388 \&\fBPTR2ul()\fR is unsafe, since a pointer may be larger than unsigned long.
3389 .PP
3390 And \f(CW\*(C`PTRV\*(C'\fR which gives the native type for an integer the same size as
3391 pointers, such as \f(CW\*(C`unsigned\*(C'\fR or \f(CW\*(C`unsigned long\*(C'\fR.
3392 .SS "Exception Handling"
3393 .IX Subsection "Exception Handling"
3394 There are a couple of macros to do very basic exception handling in XS
3395 modules. You have to define \f(CW\*(C`NO_XSLOCKS\*(C'\fR before including \fIXSUB.h\fR to
3396 be able to use these macros:
3397 .PP
3398 .Vb 2
3399 \& #define NO_XSLOCKS
3400 \& #include "XSUB.h"
3401 .Ve
3402 .PP
3403 You can use these macros if you call code that may croak, but you need
3404 to do some cleanup before giving control back to Perl. For example:
3405 .PP
3406 .Vb 1
3407 \& dXCPT; /* set up necessary variables */
3408 \&
3409 \& XCPT_TRY_START {
3410 \& code_that_may_croak();
3411 \& } XCPT_TRY_END
3412 \&
3413 \& XCPT_CATCH
3414 \& {
3415 \& /* do cleanup here */
3416 \& XCPT_RETHROW;
3417 \& }
3418 .Ve
3419 .PP
3420 Note that you always have to rethrow an exception that has been
3421 caught. Using these macros, it is not possible to just catch the
3422 exception and ignore it. If you have to ignore the exception, you
3423 have to use the \f(CW\*(C`call_*\*(C'\fR function.
3424 .PP
3425 The advantage of using the above macros is that you don\*(Aqt have
3426 to setup an extra function for \f(CW\*(C`call_*\*(C'\fR, and that using these
3427 macros is faster than using \f(CW\*(C`call_*\*(C'\fR.
3428 .SS "Source Documentation"
3429 .IX Subsection "Source Documentation"
3430 There\*(Aqs an effort going on to document the internal functions and
3431 automatically produce reference manuals from them \-\- perlapi is one
3432 such manual which details all the functions which are available to XS
3433 writers. perlintern is the autogenerated manual for the functions
3434 which are not part of the API and are supposedly for internal use only.
3435 .PP
3436 Source documentation is created by putting POD comments into the C
3437 source, like this:
3438 .PP
3439 .Vb 2
3440 \& /*
3441 \& =for apidoc sv_setiv
3442 \&
3443 \& Copies an integer into the given SV. Does not handle \*(Aqset\*(Aq magic. See
3444 \& L<perlapi/sv_setiv_mg>.
3445 \&
3446 \& =cut
3447 \& */
3448 .Ve
3449 .PP
3450 Please try and supply some documentation if you add functions to the
3451 Perl core.
3452 .SS "Backwards compatibility"
3453 .IX Subsection "Backwards compatibility"
3454 The Perl API changes over time. New functions are
3455 added or the interfaces of existing functions are
3456 changed. The \f(CW\*(C`Devel::PPPort\*(C'\fR module tries to
3457 provide compatibility code for some of these changes, so XS writers don\*(Aqt
3458 have to code it themselves when supporting multiple versions of Perl.
3459 .PP
3460 \&\f(CW\*(C`Devel::PPPort\*(C'\fR generates a C header file \fIppport.h\fR that can also
3461 be run as a Perl script. To generate \fIppport.h\fR, run:
3462 .PP
3463 .Vb 1
3464 \& perl \-MDevel::PPPort \-eDevel::PPPort::WriteFile
3465 .Ve
3466 .PP
3467 Besides checking existing XS code, the script can also be used to retrieve
3468 compatibility information for various API calls using the \f(CW\*(C`\-\-api\-info\*(C'\fR
3469 command line switch. For example:
3470 .PP
3471 .Vb 1
3472 \& % perl ppport.h \-\-api\-info=sv_magicext
3473 .Ve
3474 .PP
3475 For details, see \f(CW\*(C`perldoc\ ppport.h\*(C'\fR.
3476 .SH "Unicode Support"
3477 .IX Header "Unicode Support"
3478 Perl 5.6.0 introduced Unicode support. It\*(Aqs important for porters and XS
3479 writers to understand this support and make sure that the code they
3480 write does not corrupt Unicode data.
3481 .SS "What \fBis\fP Unicode, anyway?"
3482 .IX Subsection "What is Unicode, anyway?"
3483 In the olden, less enlightened times, we all used to use ASCII. Most of
3484 us did, anyway. The big problem with ASCII is that it\*(Aqs American. Well,
3485 no, that\*(Aqs not actually the problem; the problem is that it\*(Aqs not
3486 particularly useful for people who don\*(Aqt use the Roman alphabet. What
3487 used to happen was that particular languages would stick their own
3488 alphabet in the upper range of the sequence, between 128 and 255. Of
3489 course, we then ended up with plenty of variants that weren\*(Aqt quite
3490 ASCII, and the whole point of it being a standard was lost.
3491 .PP
3492 Worse still, if you\*(Aqve got a language like Chinese or
3493 Japanese that has hundreds or thousands of characters, then you really
3494 can\*(Aqt fit them into a mere 256, so they had to forget about ASCII
3495 altogether, and build their own systems using pairs of numbers to refer
3496 to one character.
3497 .PP
3498 To fix this, some people formed Unicode, Inc. and
3499 produced a new character set containing all the characters you can
3500 possibly think of and more. There are several ways of representing these
3501 characters, and the one Perl uses is called UTF\-8. UTF\-8 uses
3502 a variable number of bytes to represent a character. You can learn more
3503 about Unicode and Perl\*(Aqs Unicode model in perlunicode.
3504 .PP
3505 (On EBCDIC platforms, Perl uses instead UTF\-EBCDIC, which is a form of
3506 UTF\-8 adapted for EBCDIC platforms. Below, we just talk about UTF\-8.
3507 UTF\-EBCDIC is like UTF\-8, but the details are different. The macros
3508 hide the differences from you, just remember that the particular numbers
3509 and bit patterns presented below will differ in UTF\-EBCDIC.)
3510 .SS "How can I recognise a UTF\-8 string?"
3511 .IX Subsection "How can I recognise a UTF-8 string?"
3512 You can\*(Aqt. This is because UTF\-8 data is stored in bytes just like
3513 non\-UTF\-8 data. The Unicode character 200, (\f(CW0xC8\fR for you hex types)
3514 capital E with a grave accent, is represented by the two bytes
3515 \&\f(CW\*(C`v196.172\*(C'\fR. Unfortunately, the non\-Unicode string \f(CW\*(C`chr(196).chr(172)\*(C'\fR
3516 has that byte sequence as well. So you can\*(Aqt tell just by looking \-\- this
3517 is what makes Unicode input an interesting problem.
3518 .PP
3519 In general, you either have to know what you\*(Aqre dealing with, or you
3520 have to guess. The API function \f(CW\*(C`is_utf8_string\*(C'\fR can help; it\*(Aqll tell
3521 you if a string contains only valid UTF\-8 characters, and the chances
3522 of a non\-UTF\-8 string looking like valid UTF\-8 become very small very
3523 quickly with increasing string length. On a character\-by\-character
3524 basis, \f(CW\*(C`isUTF8_CHAR\*(C'\fR
3525 will tell you whether the current character in a string is valid UTF\-8.
3526 .SS "How does UTF\-8 represent Unicode characters?"
3527 .IX Subsection "How does UTF-8 represent Unicode characters?"
3528 As mentioned above, UTF\-8 uses a variable number of bytes to store a
3529 character. Characters with values 0...127 are stored in one
3530 byte, just like good ol\*(Aq ASCII. Character 128 is stored as
3531 \&\f(CW\*(C`v194.128\*(C'\fR; this continues up to character 191, which is
3532 \&\f(CW\*(C`v194.191\*(C'\fR. Now we\*(Aqve run out of bits (191 is binary
3533 \&\f(CW10111111\fR) so we move on; character 192 is \f(CW\*(C`v195.128\*(C'\fR. And
3534 so it goes on, moving to three bytes at character 2048.
3535 "Unicode Encodings" in perlunicode has pictures of how this works.
3536 .PP
3537 Assuming you know you\*(Aqre dealing with a UTF\-8 string, you can find out
3538 how long the first character in it is with the \f(CW\*(C`UTF8SKIP\*(C'\fR macro:
3539 .PP
3540 .Vb 2
3541 \& char *utf = "\e305\e233\e340\e240\e201";
3542 \& I32 len;
3543 \&
3544 \& len = UTF8SKIP(utf); /* len is 2 here */
3545 \& utf += len;
3546 \& len = UTF8SKIP(utf); /* len is 3 here */
3547 .Ve
3548 .PP
3549 Another way to skip over characters in a UTF\-8 string is to use
3550 \&\f(CW\*(C`utf8_hop\*(C'\fR, which takes a string and a number of characters to skip
3551 over. You\*(Aqre on your own about bounds checking, though, so don\*(Aqt use it
3552 lightly.
3553 .PP
3554 All bytes in a multi\-byte UTF\-8 character will have the high bit set,
3555 so you can test if you need to do something special with this
3556 character like this (the \f(CWUTF8_IS_INVARIANT()\fR is a macro that tests
3557 whether the byte is encoded as a single byte even in UTF\-8):
3558 .PP
3559 .Vb 7
3560 \& U8 *utf; /* Initialize this to point to the beginning of the
3561 \& sequence to convert */
3562 \& U8 *utf_end; /* Initialize this to 1 beyond the end of the sequence
3563 \& pointed to by \*(Aqutf\*(Aq */
3564 \& UV uv; /* Returned code point; note: a UV, not a U8, not a
3565 \& char */
3566 \& STRLEN len; /* Returned length of character in bytes */
3567 \&
3568 \& if (!UTF8_IS_INVARIANT(*utf))
3569 \& /* Must treat this as UTF\-8 */
3570 \& uv = utf8_to_uvchr_buf(utf, utf_end, &len);
3571 \& else
3572 \& /* OK to treat this character as a byte */
3573 \& uv = *utf;
3574 .Ve
3575 .PP
3576 You can also see in that example that we use \f(CW\*(C`utf8_to_uvchr_buf\*(C'\fR to get the
3577 value of the character; the inverse function \f(CW\*(C`uvchr_to_utf8\*(C'\fR is available
3578 for putting a UV into UTF\-8:
3579 .PP
3580 .Vb 6
3581 \& if (!UVCHR_IS_INVARIANT(uv))
3582 \& /* Must treat this as UTF8 */
3583 \& utf8 = uvchr_to_utf8(utf8, uv);
3584 \& else
3585 \& /* OK to treat this character as a byte */
3586 \& *utf8++ = uv;
3587 .Ve
3588 .PP
3589 You \fBmust\fR convert characters to UVs using the above functions if
3590 you\*(Aqre ever in a situation where you have to match UTF\-8 and non\-UTF\-8
3591 characters. You may not skip over UTF\-8 characters in this case. If you
3592 do this, you\*(Aqll lose the ability to match hi\-bit non\-UTF\-8 characters;
3593 for instance, if your UTF\-8 string contains \f(CW\*(C`v196.172\*(C'\fR, and you skip
3594 that character, you can never match a \f(CWchr(200)\fR in a non\-UTF\-8 string.
3595 So don\*(Aqt do that!
3596 .PP
3597 (Note that we don\*(Aqt have to test for invariant characters in the
3598 examples above. The functions work on any well\-formed UTF\-8 input.
3599 It\*(Aqs just that it\*(Aqs faster to avoid the function overhead when it\*(Aqs not
3600 needed.)
3601 .SS "How does Perl store UTF\-8 strings?"
3602 .IX Subsection "How does Perl store UTF-8 strings?"
3603 Currently, Perl deals with UTF\-8 strings and non\-UTF\-8 strings
3604 slightly differently. A flag in the SV, \f(CW\*(C`SVf_UTF8\*(C'\fR, indicates that the
3605 string is internally encoded as UTF\-8. Without it, the byte value is the
3606 codepoint number and vice versa. This flag is only meaningful if the SV
3607 is \f(CW\*(C`SvPOK\*(C'\fR or immediately after stringification via \f(CW\*(C`SvPV\*(C'\fR or a
3608 similar macro. You can check and manipulate this flag with the
3609 following macros:
3610 .PP
3611 .Vb 3
3612 \& SvUTF8(sv)
3613 \& SvUTF8_on(sv)
3614 \& SvUTF8_off(sv)
3615 .Ve
3616 .PP
3617 This flag has an important effect on Perl\*(Aqs treatment of the string: if
3618 UTF\-8 data is not properly distinguished, regular expressions,
3619 \&\f(CW\*(C`length\*(C'\fR, \f(CW\*(C`substr\*(C'\fR and other string handling operations will have
3620 undesirable (wrong) results.
3621 .PP
3622 The problem comes when you have, for instance, a string that isn\*(Aqt
3623 flagged as UTF\-8, and contains a byte sequence that could be UTF\-8 \-\-
3624 especially when combining non\-UTF\-8 and UTF\-8 strings.
3625 .PP
3626 Never forget that the \f(CW\*(C`SVf_UTF8\*(C'\fR flag is separate from the PV value; you
3627 need to be sure you don\*(Aqt accidentally knock it off while you\*(Aqre
3628 manipulating SVs. More specifically, you cannot expect to do this:
3629 .PP
3630 .Vb 4
3631 \& SV *sv;
3632 \& SV *nsv;
3633 \& STRLEN len;
3634 \& char *p;
3635 \&
3636 \& p = SvPV(sv, len);
3637 \& frobnicate(p);
3638 \& nsv = newSVpvn(p, len);
3639 .Ve
3640 .PP
3641 The \f(CW\*(C`char*\*(C'\fR string does not tell you the whole story, and you can\*(Aqt
3642 copy or reconstruct an SV just by copying the string value. Check if the
3643 old SV has the UTF8 flag set (\fIafter\fR the \f(CW\*(C`SvPV\*(C'\fR call), and act
3644 accordingly:
3645 .PP
3646 .Vb 6
3647 \& p = SvPV(sv, len);
3648 \& is_utf8 = SvUTF8(sv);
3649 \& frobnicate(p, is_utf8);
3650 \& nsv = newSVpvn(p, len);
3651 \& if (is_utf8)
3652 \& SvUTF8_on(nsv);
3653 .Ve
3654 .PP
3655 In the above, your \f(CW\*(C`frobnicate\*(C'\fR function has been changed to be made
3656 aware of whether or not it\*(Aqs dealing with UTF\-8 data, so that it can
3657 handle the string appropriately.
3658 .PP
3659 Since just passing an SV to an XS function and copying the data of
3660 the SV is not enough to copy the UTF8 flags, even less right is just
3661 passing a \f(CW\*(C`char\ *\*(C'\fR to an XS function.
3662 .PP
3663 For full generality, use the \f(CW\*(C`DO_UTF8\*(C'\fR macro to see if the
3664 string in an SV is to be \fItreated\fR as UTF\-8. This takes into account
3665 if the call to the XS function is being made from within the scope of
3666 \&\f(CW\*(C`use\ bytes\*(C'\fR. If so, the underlying bytes that comprise the
3667 UTF\-8 string are to be exposed, rather than the character they
3668 represent. But this pragma should only really be used for debugging and
3669 perhaps low\-level testing at the byte level. Hence most XS code need
3670 not concern itself with this, but various areas of the perl core do need
3671 to support it.
3672 .PP
3673 And this isn\*(Aqt the whole story. Starting in Perl v5.12, strings that
3674 aren\*(Aqt encoded in UTF\-8 may also be treated as Unicode under various
3675 conditions (see "ASCII Rules versus Unicode Rules" in perlunicode).
3676 This is only really a problem for characters whose ordinals are between
3677 128 and 255, and their behavior varies under ASCII versus Unicode rules
3678 in ways that your code cares about (see "The "Unicode Bug"" in perlunicode).
3679 There is no published API for dealing with this, as it is subject to
3680 change, but you can look at the code for \f(CW\*(C`pp_lc\*(C'\fR in \fIpp.c\fR for an
3681 example as to how it\*(Aqs currently done.
3682 .SS "How do I pass a Perl string to a C library?"
3683 .IX Subsection "How do I pass a Perl string to a C library?"
3684 A Perl string, conceptually, is an opaque sequence of code points.
3685 Many C libraries expect their inputs to be "classical" C strings, which are
3686 arrays of octets 1\-255, terminated with a NUL byte. Your job when writing
3687 an interface between Perl and a C library is to define the mapping between
3688 Perl and that library.
3689 .PP
3690 Generally speaking, \f(CW\*(C`SvPVbyte\*(C'\fR and related macros suit this task well.
3691 These assume that your Perl string is a "byte string", i.e., is either
3692 raw, undecoded input into Perl or is pre\-encoded to, e.g., UTF\-8.
3693 .PP
3694 Alternatively, if your C library expects UTF\-8 text, you can use
3695 \&\f(CW\*(C`SvPVutf8\*(C'\fR and related macros. This has the same effect as encoding
3696 to UTF\-8 then calling the corresponding \f(CW\*(C`SvPVbyte\*(C'\fR\-related macro.
3697 .PP
3698 Some C libraries may expect other encodings (e.g., UTF\-16LE). To give
3699 Perl strings to such libraries
3700 you must either do that encoding in Perl then use \f(CW\*(C`SvPVbyte\*(C'\fR, or
3701 use an intermediary C library to convert from however Perl stores the
3702 string to the desired encoding.
3703 .PP
3704 Take care also that NULs in your Perl string don\*(Aqt confuse the C
3705 library. If possible, give the string\*(Aqs length to the C library; if that\*(Aqs
3706 not possible, consider rejecting strings that contain NUL bytes.
3707 .PP
3708 \fIWhat about \fR\f(CI\*(C`SvPV\*(C'\fR\fI, \fR\f(CI\*(C`SvPV_nolen\*(C'\fR\fI, etc.?\fR
3709 .IX Subsection "What about SvPV, SvPV_nolen, etc.?"
3710 .PP
3711 Consider a 3\-character Perl string \f(CW\*(C`$foo = "\ex64\ex78\ex8c"\*(C'\fR.
3712 Perl can store these 3 characters either of two ways:
3713 .IP \(bu 4
3714 bytes: 0x64 0x78 0x8c
3715 .IP \(bu 4
3716 UTF\-8: 0x64 0x78 0xc2 0x8c
3717 .PP
3718 Now let\*(Aqs say you convert \f(CW$foo\fR to a C string thus:
3719 .PP
3720 .Vb 2
3721 \& STRLEN strlen;
3722 \& char *str = SvPV(foo_sv, strlen);
3723 .Ve
3724 .PP
3725 At this point \f(CW\*(C`str\*(C'\fR could point to a 3\-byte C string or a 4\-byte one.
3726 .PP
3727 Generally speaking, we want \f(CW\*(C`str\*(C'\fR to be the same regardless of how
3728 Perl stores \f(CW$foo\fR, so the ambiguity here is undesirable. \f(CW\*(C`SvPVbyte\*(C'\fR
3729 and \f(CW\*(C`SvPVutf8\*(C'\fR solve that by giving predictable output: use
3730 \&\f(CW\*(C`SvPVbyte\*(C'\fR if your C library expects byte strings, or \f(CW\*(C`SvPVutf8\*(C'\fR
3731 if it expects UTF\-8.
3732 .PP
3733 If your C library happens to support both encodings, then \f(CW\*(C`SvPV\*(C'\fR\-\-always
3734 in tandem with lookups to \f(CW\*(C`SvUTF8\*(C'\fR!\-\-may be safe and (slightly) more
3735 efficient.
3736 .PP
3737 \&\fBTESTING\fR \fBTIP:\fR Use utf8\*(Aqs \f(CW\*(C`upgrade\*(C'\fR and \f(CW\*(C`downgrade\*(C'\fR functions
3738 in your tests to ensure consistent handling regardless of Perl\*(Aqs
3739 internal encoding.
3740 .SS "How do I convert a string to UTF\-8?"
3741 .IX Subsection "How do I convert a string to UTF-8?"
3742 If you\*(Aqre mixing UTF\-8 and non\-UTF\-8 strings, it is necessary to upgrade
3743 the non\-UTF\-8 strings to UTF\-8. If you\*(Aqve got an SV, the easiest way to do
3744 this is:
3745 .PP
3746 .Vb 1
3747 \& sv_utf8_upgrade(sv);
3748 .Ve
3749 .PP
3750 However, you must not do this, for example:
3751 .PP
3752 .Vb 2
3753 \& if (!SvUTF8(left))
3754 \& sv_utf8_upgrade(left);
3755 .Ve
3756 .PP
3757 If you do this in a binary operator, you will actually change one of the
3758 strings that came into the operator, and, while it shouldn\*(Aqt be noticeable
3759 by the end user, it can cause problems in deficient code.
3760 .PP
3761 Instead, \f(CW\*(C`bytes_to_utf8\*(C'\fR will give you a UTF\-8\-encoded \fBcopy\fR of its
3762 string argument. This is useful for having the data available for
3763 comparisons and so on, without harming the original SV. There\*(Aqs also
3764 \&\f(CW\*(C`utf8_to_bytes\*(C'\fR to go the other way, but naturally, this will fail if
3765 the string contains any characters above 255 that can\*(Aqt be represented
3766 in a single byte.
3767 .SS "How do I compare strings?"
3768 .IX Subsection "How do I compare strings?"
3769 "sv_cmp" in perlapi and "sv_cmp_flags" in perlapi do a lexicographic
3770 comparison of two SVs, and handle UTF\-8ness properly. Note, however,
3771 that Unicode specifies a much fancier mechanism for collation, available
3772 via the Unicode::Collate module.
3773 .PP
3774 To just compare two strings for equality/non\-equality, you can just use
3775 \&\f(CWmemEQ()\fR and \f(CWmemNE()\fR as usual,
3776 except the strings must be both UTF\-8 or not UTF\-8 encoded.
3777 .PP
3778 To compare two strings case\-insensitively, use
3779 \&\f(CWfoldEQ_utf8()\fR (the strings don\*(Aqt have to have
3780 the same UTF\-8ness).
3781 .SS "Is there anything else I need to know?"
3782 .IX Subsection "Is there anything else I need to know?"
3783 Not really. Just remember these things:
3784 .IP \(bu 3
3785 There\*(Aqs no way to tell if a \f(CW\*(C`char\ *\*(C'\fR or \f(CW\*(C`U8\ *\*(C'\fR string is UTF\-8
3786 or not. But you can tell if an SV is to be treated as UTF\-8 by calling
3787 \&\f(CW\*(C`DO_UTF8\*(C'\fR on it, after stringifying it with \f(CW\*(C`SvPV\*(C'\fR or a similar
3788 macro. And, you can tell if SV is actually UTF\-8 (even if it is not to
3789 be treated as such) by looking at its \f(CW\*(C`SvUTF8\*(C'\fR flag (again after
3790 stringifying it). Don\*(Aqt forget to set the flag if something should be
3791 UTF\-8.
3792 Treat the flag as part of the PV, even though it\*(Aqs not \-\- if you pass on
3793 the PV to somewhere, pass on the flag too.
3794 .IP \(bu 3
3795 If a string is UTF\-8, \fBalways\fR use \f(CW\*(C`utf8_to_uvchr_buf\*(C'\fR to get at the value,
3796 unless \f(CWUTF8_IS_INVARIANT(*s)\fR in which case you can use \f(CW*s\fR.
3797 .IP \(bu 3
3798 When writing a character UV to a UTF\-8 string, \fBalways\fR use
3799 \&\f(CW\*(C`uvchr_to_utf8\*(C'\fR, unless \f(CW\*(C`UVCHR_IS_INVARIANT(uv))\*(C'\fR in which case
3800 you can use \f(CW\*(C`*s = uv\*(C'\fR.
3801 .IP \(bu 3
3802 Mixing UTF\-8 and non\-UTF\-8 strings is
3803 tricky. Use \f(CW\*(C`bytes_to_utf8\*(C'\fR to get
3804 a new string which is UTF\-8 encoded, and then combine them.
3805 .SH "Custom Operators"
3806 .IX Header "Custom Operators"
3807 Custom operator support is an experimental feature that allows you to
3808 define your own ops. This is primarily to allow the building of
3809 interpreters for other languages in the Perl core, but it also allows
3810 optimizations through the creation of "macro\-ops" (ops which perform the
3811 functions of multiple ops which are usually executed together, such as
3812 \&\f(CW\*(C`gvsv, gvsv, add\*(C'\fR.)
3813 .PP
3814 This feature is implemented as a new op type, \f(CW\*(C`OP_CUSTOM\*(C'\fR. The Perl
3815 core does not "know" anything special about this op type, and so it will
3816 not be involved in any optimizations. This also means that you can
3817 define your custom ops to be any op structure \-\- unary, binary, list and
3818 so on \-\- you like.
3819 .PP
3820 It\*(Aqs important to know what custom operators won\*(Aqt do for you. They
3821 won\*(Aqt let you add new syntax to Perl, directly. They won\*(Aqt even let you
3822 add new keywords, directly. In fact, they won\*(Aqt change the way Perl
3823 compiles a program at all. You have to do those changes yourself, after
3824 Perl has compiled the program. You do this either by manipulating the op
3825 tree using a \f(CW\*(C`CHECK\*(C'\fR block and the \f(CW\*(C`B::Generate\*(C'\fR module, or by adding
3826 a custom peephole optimizer with the \f(CW\*(C`optimize\*(C'\fR module.
3827 .PP
3828 When you do this, you replace ordinary Perl ops with custom ops by
3829 creating ops with the type \f(CW\*(C`OP_CUSTOM\*(C'\fR and the \f(CW\*(C`op_ppaddr\*(C'\fR of your own
3830 PP function. This should be defined in XS code, and should look like
3831 the PP ops in \f(CW\*(C`pp_*.c\*(C'\fR. You are responsible for ensuring that your op
3832 takes the appropriate number of values from the stack, and you are
3833 responsible for adding stack marks if necessary.
3834 .PP
3835 You should also "register" your op with the Perl interpreter so that it
3836 can produce sensible error and warning messages. Since it is possible to
3837 have multiple custom ops within the one "logical" op type \f(CW\*(C`OP_CUSTOM\*(C'\fR,
3838 Perl uses the value of \f(CW\*(C`o\->op_ppaddr\*(C'\fR to determine which custom op
3839 it is dealing with. You should create an \f(CW\*(C`XOP\*(C'\fR structure for each
3840 ppaddr you use, set the properties of the custom op with
3841 \&\f(CW\*(C`XopENTRY_set\*(C'\fR, and register the structure against the ppaddr using
3842 \&\f(CW\*(C`Perl_custom_op_register\*(C'\fR. A trivial example might look like:
3843 .PP
3844 .Vb 2
3845 \& static XOP my_xop;
3846 \& static OP *my_pp(pTHX);
3847 \&
3848 \& BOOT:
3849 \& XopENTRY_set(&my_xop, xop_name, "myxop");
3850 \& XopENTRY_set(&my_xop, xop_desc, "Useless custom op");
3851 \& Perl_custom_op_register(aTHX_ my_pp, &my_xop);
3852 .Ve
3853 .PP
3854 The available fields in the structure are:
3855 .IP xop_name 4
3856 .IX Item "xop_name"
3857 A short name for your op. This will be included in some error messages,
3858 and will also be returned as \f(CW\*(C`$op\->name\*(C'\fR by the B module, so
3859 it will appear in the output of module like B::Concise.
3860 .IP xop_desc 4
3861 .IX Item "xop_desc"
3862 A short description of the function of the op.
3863 .IP xop_class 4
3864 .IX Item "xop_class"
3865 Which of the various \f(CW*OP\fR structures this op uses. This should be one of
3866 the \f(CW\*(C`OA_*\*(C'\fR constants from \fIop.h\fR, namely
3867 .RS 4
3868 .IP OA_BASEOP 4
3869 .IX Item "OA_BASEOP"
3870 .PD 0
3871 .IP OA_UNOP 4
3872 .IX Item "OA_UNOP"
3873 .IP OA_BINOP 4
3874 .IX Item "OA_BINOP"
3875 .IP OA_LOGOP 4
3876 .IX Item "OA_LOGOP"
3877 .IP OA_LISTOP 4
3878 .IX Item "OA_LISTOP"
3879 .IP OA_PMOP 4
3880 .IX Item "OA_PMOP"
3881 .IP OA_SVOP 4
3882 .IX Item "OA_SVOP"
3883 .IP OA_PADOP 4
3884 .IX Item "OA_PADOP"
3885 .IP OA_PVOP_OR_SVOP 4
3886 .IX Item "OA_PVOP_OR_SVOP"
3887 .PD
3888 This should be interpreted as \*(Aq\f(CW\*(C`PVOP\*(C'\fR\*(Aq only. The \f(CW\*(C`_OR_SVOP\*(C'\fR is because
3889 the only core \f(CW\*(C`PVOP\*(C'\fR, \f(CW\*(C`OP_TRANS\*(C'\fR, can sometimes be a \f(CW\*(C`SVOP\*(C'\fR instead.
3890 .IP OA_LOOP 4
3891 .IX Item "OA_LOOP"
3892 .PD 0
3893 .IP OA_COP 4
3894 .IX Item "OA_COP"
3895 .PD
3896 .RE
3897 .RS 4
3898 .Sp
3899 The other \f(CW\*(C`OA_*\*(C'\fR constants should not be used.
3900 .RE
3901 .IP xop_peep 4
3902 .IX Item "xop_peep"
3903 This member is of type \f(CW\*(C`Perl_cpeep_t\*(C'\fR, which expands to \f(CW\*(C`void
3904 (*Perl_cpeep_t)(aTHX_ OP *o, OP *oldop)\*(C'\fR. If it is set, this function
3905 will be called from \f(CW\*(C`Perl_rpeep\*(C'\fR when ops of this type are encountered
3906 by the peephole optimizer. \fIo\fR is the OP that needs optimizing;
3907 \&\fIoldop\fR is the previous OP optimized, whose \f(CW\*(C`op_next\*(C'\fR points to \fIo\fR.
3908 .IP xop_dump 4
3909 .IX Item "xop_dump"
3910 This member is a pointer to a function of type
3911 \&\f(CW\*(C`void (pTHX_ OP *, struct OpDumpContext *)\*(C'\fR. If set, this function is called
3912 by \f(CWop_dump()\fR when dumping a custom operator of this type, after the op\*(Aqs
3913 basic fields have been printed. This function may make use of
3914 \&\f(CWopdump_printf()\fR to emit additional output that may be useful for debugging.
3915 .Sp
3916 The opaque structure pointer passed in as its final argument should be passed
3917 directly into \f(CWopdump_printf()\fR.
3918 .PP
3919 \&\f(CW\*(C`B::Generate\*(C'\fR directly supports the creation of custom ops by name.
3920 .SH Stacks
3921 .IX Header "Stacks"
3922 Descriptions above occasionally refer to "the stack", but there are in fact
3923 many stack\-like data structures within the perl interpreter. When otherwise
3924 unqualified, "the stack" usually refers to the value stack.
3925 .PP
3926 The various stacks have different purposes, and operate in slightly different
3927 ways. Their differences are noted below.
3928 .SS "Value Stack"
3929 .IX Subsection "Value Stack"
3930 This stack stores the values that regular perl code is operating on, usually
3931 intermediate values of expressions within a statement. The stack itself is
3932 formed of an array of SV pointers.
3933 .PP
3934 The base of this stack is pointed to by the interpreter variable
3935 \&\f(CW\*(C`PL_stack_base\*(C'\fR, of type \f(CW\*(C`SV **\*(C'\fR.
3936 .PP
3937 The head of the stack is \f(CW\*(C`PL_stack_sp\*(C'\fR, and points to the most
3938 recently\-pushed item.
3939 .PP
3940 Items are pushed to the stack by using the \f(CWPUSHs()\fR macro or its variants
3941 described above; \f(CWXPUSHs()\fR, \f(CWmPUSHs()\fR, \f(CWmXPUSHs()\fR and the typed
3942 versions. Note carefully that the non\-\f(CW\*(C`X\*(C'\fR versions of these macros do not
3943 check the size of the stack and assume it to be big enough. These must be
3944 paired with a suitable check of the stack\*(Aqs size, such as the \f(CW\*(C`EXTEND\*(C'\fR macro
3945 to ensure it is large enough. For example
3946 .PP
3947 .Vb 5
3948 \& EXTEND(SP, 4);
3949 \& mPUSHi(10);
3950 \& mPUSHi(20);
3951 \& mPUSHi(30);
3952 \& mPUSHi(40);
3953 .Ve
3954 .PP
3955 This is slightly more performant than making four separate checks in four
3956 separate \f(CWmXPUSHi()\fR calls.
3957 .PP
3958 As a further performance optimisation, the various \f(CW\*(C`PUSH\*(C'\fR macros all operate
3959 using a local variable \f(CW\*(C`SP\*(C'\fR, rather than the interpreter\-global variable
3960 \&\f(CW\*(C`PL_stack_sp\*(C'\fR. This variable is declared by the \f(CW\*(C`dSP\*(C'\fR macro \- though it is
3961 normally implied by XSUBs and similar so it is rare you have to consider it
3962 directly. Once declared, the \f(CW\*(C`PUSH\*(C'\fR macros will operate only on this local
3963 variable, so before invoking any other perl core functions you must use the
3964 \&\f(CW\*(C`PUTBACK\*(C'\fR macro to return the value from the local \f(CW\*(C`SP\*(C'\fR variable back to
3965 the interpreter variable. Similarly, after calling a perl core function which
3966 may have had reason to move the stack or push/pop values to it, you must use
3967 the \f(CW\*(C`SPAGAIN\*(C'\fR macro which refreshes the local \f(CW\*(C`SP\*(C'\fR value back from the
3968 interpreter one.
3969 .PP
3970 Items are popped from the stack by using the \f(CW\*(C`POPs\*(C'\fR macro or its typed
3971 versions, There is also a macro \f(CW\*(C`TOPs\*(C'\fR that inspects the topmost item without
3972 removing it.
3973 .PP
3974 Note specifically that SV pointers on the value stack do not contribute to the
3975 overall reference count of the xVs being referred to. If newly\-created xVs are
3976 being pushed to the stack you must arrange for them to be destroyed at a
3977 suitable time; usually by using one of the \f(CW\*(C`mPUSH*\*(C'\fR macros or \f(CWsv_2mortal()\fR
3978 to mortalise the xV.
3979 .SS "Mark Stack"
3980 .IX Subsection "Mark Stack"
3981 The value stack stores individual perl scalar values as temporaries between
3982 expressions. Some perl expressions operate on entire lists; for that purpose
3983 we need to know where on the stack each list begins. This is the purpose of the
3984 mark stack.
3985 .PP
3986 The mark stack stores integers as I32 values, which are the height of the
3987 value stack at the time before the list began; thus the mark itself actually
3988 points to the value stack entry one before the list. The list itself starts at
3989 \&\f(CW\*(C`mark + 1\*(C'\fR.
3990 .PP
3991 The base of this stack is pointed to by the interpreter variable
3992 \&\f(CW\*(C`PL_markstack\*(C'\fR, of type \f(CW\*(C`I32 *\*(C'\fR.
3993 .PP
3994 The head of the stack is \f(CW\*(C`PL_markstack_ptr\*(C'\fR, and points to the most
3995 recently\-pushed item.
3996 .PP
3997 Items are pushed to the stack by using the \f(CWPUSHMARK()\fR macro. Even though
3998 the stack itself stores (value) stack indices as integers, the \f(CW\*(C`PUSHMARK\*(C'\fR
3999 macro should be given a stack pointer directly; it will calculate the index
4000 offset by comparing to the \f(CW\*(C`PL_stack_sp\*(C'\fR variable. Thus almost always the
4001 code to perform this is
4002 .PP
4003 .Vb 1
4004 \& PUSHMARK(SP);
4005 .Ve
4006 .PP
4007 Items are popped from the stack by the \f(CW\*(C`POPMARK\*(C'\fR macro. There is also a macro
4008 \&\f(CW\*(C`TOPMARK\*(C'\fR that inspects the topmost item without removing it. These macros
4009 return I32 index values directly. There is also the \f(CW\*(C`dMARK\*(C'\fR macro which
4010 declares a new SV double\-pointer variable, called \f(CW\*(C`mark\*(C'\fR, which points at the
4011 marked stack slot; this is the usual macro that C code will use when operating
4012 on lists given on the stack.
4013 .PP
4014 As noted above, the \f(CW\*(C`mark\*(C'\fR variable itself will point at the most recently
4015 pushed value on the value stack before the list begins, and so the list itself
4016 starts at \f(CW\*(C`mark + 1\*(C'\fR. The values of the list may be iterated by code such as
4017 .PP
4018 .Vb 4
4019 \& for(SV **svp = mark + 1; svp <= PL_stack_sp; svp++) {
4020 \& SV *item = *svp;
4021 \& ...
4022 \& }
4023 .Ve
4024 .PP
4025 Note specifically in the case that the list is already empty, \f(CW\*(C`mark\*(C'\fR will
4026 equal \f(CW\*(C`PL_stack_sp\*(C'\fR.
4027 .PP
4028 Because the \f(CW\*(C`mark\*(C'\fR variable is converted to a pointer on the value stack,
4029 extra care must be taken if \f(CW\*(C`EXTEND\*(C'\fR or any of the \f(CW\*(C`XPUSH\*(C'\fR macros are
4030 invoked within the function, because the stack may need to be moved to
4031 extend it and so the existing pointer will now be invalid. If this may be a
4032 problem, a possible solution is to track the mark offset as an integer and
4033 track the mark itself later on after the stack had been moved.
4034 .PP
4035 .Vb 1
4036 \& I32 markoff = POPMARK;
4037 \&
4038 \& ...
4039 \&
4040 \& SP **mark = PL_stack_base + markoff;
4041 .Ve
4042 .SS "Temporaries Stack"
4043 .IX Subsection "Temporaries Stack"
4044 As noted above, xV references on the main value stack do not contribute to the
4045 reference count of an xV, and so another mechanism is used to track when
4046 temporary values which live on the stack must be released. This is the job of
4047 the temporaries stack.
4048 .PP
4049 The temporaries stack stores pointers to xVs whose reference counts will be
4050 decremented soon.
4051 .PP
4052 The base of this stack is pointed to by the interpreter variable
4053 \&\f(CW\*(C`PL_tmps_stack\*(C'\fR, of type \f(CW\*(C`SV **\*(C'\fR.
4054 .PP
4055 The head of the stack is indexed by \f(CW\*(C`PL_tmps_ix\*(C'\fR, an integer which stores the
4056 index in the array of the most recently\-pushed item.
4057 .PP
4058 There is no public API to directly push items to the temporaries stack. Instead,
4059 the API function \f(CWsv_2mortal()\fR is used to mortalize an xV, adding its
4060 address to the temporaries stack.
4061 .PP
4062 Likewise, there is no public API to read values from the temporaries stack.
4063 Instead, the macros \f(CW\*(C`SAVETMPS\*(C'\fR and \f(CW\*(C`FREETMPS\*(C'\fR are used. The \f(CW\*(C`SAVETMPS\*(C'\fR
4064 macro establishes the base levels of the temporaries stack, by capturing the
4065 current value of \f(CW\*(C`PL_tmps_ix\*(C'\fR into \f(CW\*(C`PL_tmps_floor\*(C'\fR and saving the previous
4066 value to the save stack. Thereafter, whenever \f(CW\*(C`FREETMPS\*(C'\fR is invoked all of
4067 the temporaries that have been pushed since that level are reclaimed.
4068 .PP
4069 While it is common to see these two macros in pairs within an \f(CW\*(C`ENTER\*(C'\fR/
4070 \&\f(CW\*(C`LEAVE\*(C'\fR pair, it is not necessary to match them. It is permitted to invoke
4071 \&\f(CW\*(C`FREETMPS\*(C'\fR multiple times since the most recent \f(CW\*(C`SAVETMPS\*(C'\fR; for example in a
4072 loop iterating over elements of a list. While you can invoke \f(CW\*(C`SAVETMPS\*(C'\fR
4073 multiple times within a scope pair, it is unlikely to be useful. Subsequent
4074 invocations will move the temporaries floor further up, thus effectively
4075 trapping the existing temporaries to only be released at the end of the scope.
4076 .SS "Save Stack"
4077 .IX Subsection "Save Stack"
4078 The save stack is used by perl to implement the \f(CW\*(C`local\*(C'\fR keyword and other
4079 similar behaviours; any cleanup operations that need to be performed when
4080 leaving the current scope. Items pushed to this stack generally capture the
4081 current value of some internal variable or state, which will be restored when
4082 the scope is unwound due to leaving, \f(CW\*(C`return\*(C'\fR, \f(CW\*(C`die\*(C'\fR, \f(CW\*(C`goto\*(C'\fR or other
4083 reasons.
4084 .PP
4085 Whereas other perl internal stacks store individual items all of the same type
4086 (usually SV pointers or integers), the items pushed to the save stack are
4087 formed of many different types, having multiple fields to them. For example,
4088 the \f(CW\*(C`SAVEt_INT\*(C'\fR type needs to store both the address of the \f(CW\*(C`int\*(C'\fR variable
4089 to restore, and the value to restore it to. This information could have been
4090 stored using fields of a \f(CW\*(C`struct\*(C'\fR, but would have to be large enough to store
4091 three pointers in the largest case, which would waste a lot of space in most
4092 of the smaller cases.
4093 .PP
4094 Instead, the stack stores information in a variable\-length encoding of \f(CW\*(C`ANY\*(C'\fR
4095 structures. The final value pushed is stored in the \f(CW\*(C`UV\*(C'\fR field which encodes
4096 the kind of item held by the preceding items; the count and types of which
4097 will depend on what kind of item is being stored. The kind field is pushed
4098 last because that will be the first field to be popped when unwinding items
4099 from the stack.
4100 .PP
4101 The base of this stack is pointed to by the interpreter variable
4102 \&\f(CW\*(C`PL_savestack\*(C'\fR, of type \f(CW\*(C`ANY *\*(C'\fR.
4103 .PP
4104 The head of the stack is indexed by \f(CW\*(C`PL_savestack_ix\*(C'\fR, an integer which
4105 stores the index in the array at which the next item should be pushed. (Note
4106 that this is different to most other stacks, which reference the most
4107 recently\-pushed item).
4108 .PP
4109 Items are pushed to the save stack by using the various \f(CW\*(C`SAVE...()\*(C'\fR macros.
4110 Many of these macros take a variable and store both its address and current
4111 value on the save stack, ensuring that value gets restored on scope exit.
4112 .PP
4113 .Vb 5
4114 \& SAVEI8(i8)
4115 \& SAVEI16(i16)
4116 \& SAVEI32(i32)
4117 \& SAVEINT(i)
4118 \& ...
4119 .Ve
4120 .PP
4121 There are also a variety of other special\-purpose macros which save particular
4122 types or values of interest. \f(CW\*(C`SAVETMPS\*(C'\fR has already been mentioned above.
4123 Others include \f(CW\*(C`SAVEFREEPV\*(C'\fR which arranges for a PV (i.e. a string buffer) to
4124 be freed, or \f(CW\*(C`SAVEDESTRUCTOR\*(C'\fR which arranges for a given function pointer to
4125 be invoked on scope exit. A full list of such macros can be found in
4126 \&\fIscope.h\fR.
4127 .PP
4128 There is no public API for popping individual values or items from the save
4129 stack. Instead, via the scope stack, the \f(CW\*(C`ENTER\*(C'\fR and \f(CW\*(C`LEAVE\*(C'\fR pair forms a way
4130 to start and stop nested scopes. Leaving a nested scope via \f(CW\*(C`LEAVE\*(C'\fR will
4131 restore all of the saved values that had been pushed since the most recent
4132 \&\f(CW\*(C`ENTER\*(C'\fR.
4133 .SS "Scope Stack"
4134 .IX Subsection "Scope Stack"
4135 As with the mark stack to the value stack, the scope stack forms a pair with
4136 the save stack. The scope stack stores the height of the save stack at which
4137 nested scopes begin, and allows the save stack to be unwound back to that
4138 point when the scope is left.
4139 .PP
4140 When perl is built with debugging enabled, there is a second part to this
4141 stack storing human\-readable string names describing the type of stack
4142 context. Each push operation saves the name as well as the height of the save
4143 stack, and each pop operation checks the topmost name with what is expected,
4144 causing an assertion failure if the name does not match.
4145 .PP
4146 The base of this stack is pointed to by the interpreter variable
4147 \&\f(CW\*(C`PL_scopestack\*(C'\fR, of type \f(CW\*(C`I32 *\*(C'\fR. If enabled, the scope stack names are
4148 stored in a separate array pointed to by \f(CW\*(C`PL_scopestack_name\*(C'\fR, of type
4149 \&\f(CW\*(C`const char **\*(C'\fR.
4150 .PP
4151 The head of the stack is indexed by \f(CW\*(C`PL_scopestack_ix\*(C'\fR, an integer which
4152 stores the index of the array or arrays at which the next item should be
4153 pushed. (Note that this is different to most other stacks, which reference the
4154 most recently\-pushed item).
4155 .PP
4156 Values are pushed to the scope stack using the \f(CW\*(C`ENTER\*(C'\fR macro, which begins a
4157 new nested scope. Any items pushed to the save stack are then restored at the
4158 next nested invocation of the \f(CW\*(C`LEAVE\*(C'\fR macro.
4159 .SH "Dynamic Scope and the Context Stack"
4160 .IX Header "Dynamic Scope and the Context Stack"
4161 \&\fBNote:\fR this section describes a non\-public internal API that is subject
4162 to change without notice.
4163 .SS "Introduction to the context stack"
4164 .IX Subsection "Introduction to the context stack"
4165 In Perl, dynamic scoping refers to the runtime nesting of things like
4166 subroutine calls, evals etc, as well as the entering and exiting of block
4167 scopes. For example, the restoring of a \f(CW\*(C`local\*(C'\fRised variable is
4168 determined by the dynamic scope.
4169 .PP
4170 Perl tracks the dynamic scope by a data structure called the context
4171 stack, which is an array of \f(CW\*(C`PERL_CONTEXT\*(C'\fR structures, and which is
4172 itself a big union for all the types of context. Whenever a new scope is
4173 entered (such as a block, a \f(CW\*(C`for\*(C'\fR loop, or a subroutine call), a new
4174 context entry is pushed onto the stack. Similarly when leaving a block or
4175 returning from a subroutine call etc. a context is popped. Since the
4176 context stack represents the current dynamic scope, it can be searched.
4177 For example, \f(CW\*(C`next LABEL\*(C'\fR searches back through the stack looking for a
4178 loop context that matches the label; \f(CW\*(C`return\*(C'\fR pops contexts until it
4179 finds a sub or eval context or similar; \f(CW\*(C`caller\*(C'\fR examines sub contexts on
4180 the stack.
4181 .PP
4182 Each context entry is labelled with a context type, \f(CW\*(C`cx_type\*(C'\fR. Typical
4183 context types are \f(CW\*(C`CXt_SUB\*(C'\fR, \f(CW\*(C`CXt_EVAL\*(C'\fR etc., as well as \f(CW\*(C`CXt_BLOCK\*(C'\fR
4184 and \f(CW\*(C`CXt_NULL\*(C'\fR which represent a basic scope (as pushed by \f(CW\*(C`pp_enter\*(C'\fR)
4185 and a sort block. The type determines which part of the context union are
4186 valid.
4187 .PP
4188 The main division in the context struct is between a substitution scope
4189 (\f(CW\*(C`CXt_SUBST\*(C'\fR) and block scopes, which are everything else. The former is
4190 just used while executing \f(CW\*(C`s///e\*(C'\fR, and won\*(Aqt be discussed further
4191 here.
4192 .PP
4193 All the block scope types share a common base, which corresponds to
4194 \&\f(CW\*(C`CXt_BLOCK\*(C'\fR. This stores the old values of various scope\-related
4195 variables like \f(CW\*(C`PL_curpm\*(C'\fR, as well as information about the current
4196 scope, such as \f(CW\*(C`gimme\*(C'\fR. On scope exit, the old variables are restored.
4197 .PP
4198 Particular block scope types store extra per\-type information. For
4199 example, \f(CW\*(C`CXt_SUB\*(C'\fR stores the currently executing CV, while the various
4200 for loop types might hold the original loop variable SV. On scope exit,
4201 the per\-type data is processed; for example the CV has its reference count
4202 decremented, and the original loop variable is restored.
4203 .PP
4204 The macro \f(CW\*(C`cxstack\*(C'\fR returns the base of the current context stack, while
4205 \&\f(CW\*(C`cxstack_ix\*(C'\fR is the index of the current frame within that stack.
4206 .PP
4207 In fact, the context stack is actually part of a stack\-of\-stacks system;
4208 whenever something unusual is done such as calling a \f(CW\*(C`DESTROY\*(C'\fR or tie
4209 handler, a new stack is pushed, then popped at the end.
4210 .PP
4211 Note that the API described here changed considerably in perl 5.24; prior
4212 to that, big macros like \f(CW\*(C`PUSHBLOCK\*(C'\fR and \f(CW\*(C`POPSUB\*(C'\fR were used; in 5.24
4213 they were replaced by the inline static functions described below. In
4214 addition, the ordering and detail of how these macros/function work
4215 changed in many ways, often subtly. In particular they didn\*(Aqt handle
4216 saving the savestack and temps stack positions, and required additional
4217 \&\f(CW\*(C`ENTER\*(C'\fR, \f(CW\*(C`SAVETMPS\*(C'\fR and \f(CW\*(C`LEAVE\*(C'\fR compared to the new functions. The
4218 old\-style macros will not be described further.
4219 .SS "Pushing contexts"
4220 .IX Subsection "Pushing contexts"
4221 For pushing a new context, the two basic functions are
4222 \&\f(CW\*(C`cx = cx_pushblock()\*(C'\fR, which pushes a new basic context block and returns
4223 its address, and a family of similar functions with names like
4224 \&\f(CWcx_pushsub(cx)\fR which populate the additional type\-dependent fields in
4225 the \f(CW\*(C`cx\*(C'\fR struct. Note that \f(CW\*(C`CXt_NULL\*(C'\fR and \f(CW\*(C`CXt_BLOCK\*(C'\fR don\*(Aqt have their
4226 own push functions, as they don\*(Aqt store any data beyond that pushed by
4227 \&\f(CW\*(C`cx_pushblock\*(C'\fR.
4228 .PP
4229 The fields of the context struct and the arguments to the \f(CW\*(C`cx_*\*(C'\fR
4230 functions are subject to change between perl releases, representing
4231 whatever is convenient or efficient for that release.
4232 .PP
4233 A typical context stack pushing can be found in \f(CW\*(C`pp_entersub\*(C'\fR; the
4234 following shows a simplified and stripped\-down example of a non\-XS call,
4235 along with comments showing roughly what each function does.
4236 .PP
4237 .Vb 6
4238 \& dMARK;
4239 \& U8 gimme = GIMME_V;
4240 \& bool hasargs = cBOOL(PL_op\->op_flags & OPf_STACKED);
4241 \& OP *retop = PL_op\->op_next;
4242 \& I32 old_ss_ix = PL_savestack_ix;
4243 \& CV *cv = ....;
4244 \&
4245 \& /* ... make mortal copies of stack args which are PADTMPs here ... */
4246 \&
4247 \& /* ... do any additional savestack pushes here ... */
4248 \&
4249 \& /* Now push a new context entry of type \*(AqCXt_SUB\*(Aq; initially just
4250 \& * doing the actions common to all block types: */
4251 \&
4252 \& cx = cx_pushblock(CXt_SUB, gimme, MARK, old_ss_ix);
4253 \&
4254 \& /* this does (approximately):
4255 \& CXINC; /* cxstack_ix++ (grow if necessary) */
4256 \& cx = CX_CUR(); /* and get the address of new frame */
4257 \& cx\->cx_type = CXt_SUB;
4258 \& cx\->blk_gimme = gimme;
4259 \& cx\->blk_oldsp = MARK \- PL_stack_base;
4260 \& cx\->blk_oldsaveix = old_ss_ix;
4261 \& cx\->blk_oldcop = PL_curcop;
4262 \& cx\->blk_oldmarksp = PL_markstack_ptr \- PL_markstack;
4263 \& cx\->blk_oldscopesp = PL_scopestack_ix;
4264 \& cx\->blk_oldpm = PL_curpm;
4265 \& cx\->blk_old_tmpsfloor = PL_tmps_floor;
4266 \&
4267 \& PL_tmps_floor = PL_tmps_ix;
4268 \& */
4269 \&
4270 \&
4271 \& /* then update the new context frame with subroutine\-specific info,
4272 \& * such as the CV about to be executed: */
4273 \&
4274 \& cx_pushsub(cx, cv, retop, hasargs);
4275 \&
4276 \& /* this does (approximately):
4277 \& cx\->blk_sub.cv = cv;
4278 \& cx\->blk_sub.olddepth = CvDEPTH(cv);
4279 \& cx\->blk_sub.prevcomppad = PL_comppad;
4280 \& cx\->cx_type |= (hasargs) ? CXp_HASARGS : 0;
4281 \& cx\->blk_sub.retop = retop;
4282 \& SvREFCNT_inc_simple_void_NN(cv);
4283 \& */
4284 .Ve
4285 .PP
4286 Note that \f(CWcx_pushblock()\fR sets two new floors: for the args stack (to
4287 \&\f(CW\*(C`MARK\*(C'\fR) and the temps stack (to \f(CW\*(C`PL_tmps_ix\*(C'\fR). While executing at this
4288 scope level, every \f(CW\*(C`nextstate\*(C'\fR (amongst others) will reset the args and
4289 tmps stack levels to these floors. Note that since \f(CW\*(C`cx_pushblock\*(C'\fR uses
4290 the current value of \f(CW\*(C`PL_tmps_ix\*(C'\fR rather than it being passed as an arg,
4291 this dictates at what point \f(CW\*(C`cx_pushblock\*(C'\fR should be called. In
4292 particular, any new mortals which should be freed only on scope exit
4293 (rather than at the next \f(CW\*(C`nextstate\*(C'\fR) should be created first.
4294 .PP
4295 Most callers of \f(CW\*(C`cx_pushblock\*(C'\fR simply set the new args stack floor to the
4296 top of the previous stack frame, but for \f(CW\*(C`CXt_LOOP_LIST\*(C'\fR it stores the
4297 items being iterated over on the stack, and so sets \f(CW\*(C`blk_oldsp\*(C'\fR to the
4298 top of these items instead. Note that, contrary to its name, \f(CW\*(C`blk_oldsp\*(C'\fR
4299 doesn\*(Aqt always represent the value to restore \f(CW\*(C`PL_stack_sp\*(C'\fR to on scope
4300 exit.
4301 .PP
4302 Note the early capture of \f(CW\*(C`PL_savestack_ix\*(C'\fR to \f(CW\*(C`old_ss_ix\*(C'\fR, which is
4303 later passed as an arg to \f(CW\*(C`cx_pushblock\*(C'\fR. In the case of \f(CW\*(C`pp_entersub\*(C'\fR,
4304 this is because, although most values needing saving are stored in fields
4305 of the context struct, an extra value needs saving only when the debugger
4306 is running, and it doesn\*(Aqt make sense to bloat the struct for this rare
4307 case. So instead it is saved on the savestack. Since this value gets
4308 calculated and saved before the context is pushed, it is necessary to pass
4309 the old value of \f(CW\*(C`PL_savestack_ix\*(C'\fR to \f(CW\*(C`cx_pushblock\*(C'\fR, to ensure that the
4310 saved value gets freed during scope exit. For most users of
4311 \&\f(CW\*(C`cx_pushblock\*(C'\fR, where nothing needs pushing on the save stack,
4312 \&\f(CW\*(C`PL_savestack_ix\*(C'\fR is just passed directly as an arg to \f(CW\*(C`cx_pushblock\*(C'\fR.
4313 .PP
4314 Note that where possible, values should be saved in the context struct
4315 rather than on the save stack; it\*(Aqs much faster that way.
4316 .PP
4317 Normally \f(CW\*(C`cx_pushblock\*(C'\fR should be immediately followed by the appropriate
4318 \&\f(CW\*(C`cx_pushfoo\*(C'\fR, with nothing between them; this is because if code
4319 in\-between could die (e.g. a warning upgraded to fatal), then the context
4320 stack unwinding code in \f(CW\*(C`dounwind\*(C'\fR would see (in the example above) a
4321 \&\f(CW\*(C`CXt_SUB\*(C'\fR context frame, but without all the subroutine\-specific fields
4322 set, and crashes would soon ensue.
4323 .PP
4324 Where the two must be separate, initially set the type to \f(CW\*(C`CXt_NULL\*(C'\fR or
4325 \&\f(CW\*(C`CXt_BLOCK\*(C'\fR, and later change it to \f(CW\*(C`CXt_foo\*(C'\fR when doing the
4326 \&\f(CW\*(C`cx_pushfoo\*(C'\fR. This is exactly what \f(CW\*(C`pp_enteriter\*(C'\fR does, once it\*(Aqs
4327 determined which type of loop it\*(Aqs pushing.
4328 .SS "Popping contexts"
4329 .IX Subsection "Popping contexts"
4330 Contexts are popped using \f(CWcx_popsub()\fR etc. and \f(CWcx_popblock()\fR. Note
4331 however, that unlike \f(CW\*(C`cx_pushblock\*(C'\fR, neither of these functions actually
4332 decrement the current context stack index; this is done separately using
4333 \&\f(CWCX_POP()\fR.
4334 .PP
4335 There are two main ways that contexts are popped. During normal execution
4336 as scopes are exited, functions like \f(CW\*(C`pp_leave\*(C'\fR, \f(CW\*(C`pp_leaveloop\*(C'\fR and
4337 \&\f(CW\*(C`pp_leavesub\*(C'\fR process and pop just one context using \f(CW\*(C`cx_popfoo\*(C'\fR and
4338 \&\f(CW\*(C`cx_popblock\*(C'\fR. On the other hand, things like \f(CW\*(C`pp_return\*(C'\fR and \f(CW\*(C`next\*(C'\fR
4339 may have to pop back several scopes until a sub or loop context is found,
4340 and exceptions (such as \f(CW\*(C`die\*(C'\fR) need to pop back contexts until an eval
4341 context is found. Both of these are accomplished by \f(CWdounwind()\fR, which
4342 is capable of processing and popping all contexts above the target one.
4343 .PP
4344 Here is a typical example of context popping, as found in \f(CW\*(C`pp_leavesub\*(C'\fR
4345 (simplified slightly):
4346 .PP
4347 .Vb 4
4348 \& U8 gimme;
4349 \& PERL_CONTEXT *cx;
4350 \& SV **oldsp;
4351 \& OP *retop;
4352 \&
4353 \& cx = CX_CUR();
4354 \&
4355 \& gimme = cx\->blk_gimme;
4356 \& oldsp = PL_stack_base + cx\->blk_oldsp; /* last arg of previous frame */
4357 \&
4358 \& if (gimme == G_VOID)
4359 \& PL_stack_sp = oldsp;
4360 \& else
4361 \& leave_adjust_stacks(oldsp, oldsp, gimme, 0);
4362 \&
4363 \& CX_LEAVE_SCOPE(cx);
4364 \& cx_popsub(cx);
4365 \& cx_popblock(cx);
4366 \& retop = cx\->blk_sub.retop;
4367 \& CX_POP(cx);
4368 \&
4369 \& return retop;
4370 .Ve
4371 .PP
4372 The steps above are in a very specific order, designed to be the reverse
4373 order of when the context was pushed. The first thing to do is to copy
4374 and/or protect any return arguments and free any temps in the current
4375 scope. Scope exits like an rvalue sub normally return a mortal copy of
4376 their return args (as opposed to lvalue subs). It is important to make
4377 this copy before the save stack is popped or variables are restored, or
4378 bad things like the following can happen:
4379 .PP
4380 .Vb 2
4381 \& sub f { my $x =...; $x } # $x freed before we get to copy it
4382 \& sub f { /(...)/; $1 } # PL_curpm restored before $1 copied
4383 .Ve
4384 .PP
4385 Although we wish to free any temps at the same time, we have to be careful
4386 not to free any temps which are keeping return args alive; nor to free the
4387 temps we have just created while mortal copying return args. Fortunately,
4388 \&\f(CWleave_adjust_stacks()\fR is capable of making mortal copies of return args,
4389 shifting args down the stack, and only processing those entries on the
4390 temps stack that are safe to do so.
4391 .PP
4392 In void context no args are returned, so it\*(Aqs more efficient to skip
4393 calling \f(CWleave_adjust_stacks()\fR. Also in void context, a \f(CW\*(C`nextstate\*(C'\fR op
4394 is likely to be imminently called which will do a \f(CW\*(C`FREETMPS\*(C'\fR, so there\*(Aqs
4395 no need to do that either.
4396 .PP
4397 The next step is to pop savestack entries: \f(CWCX_LEAVE_SCOPE(cx)\fR is just
4398 defined as \f(CWLEAVE_SCOPE(cx\->blk_oldsaveix)\fR. Note that during the
4399 popping, it\*(Aqs possible for perl to call destructors, call \f(CW\*(C`STORE\*(C'\fR to undo
4400 localisations of tied vars, and so on. Any of these can die or call
4401 \&\f(CWexit()\fR. In this case, \f(CWdounwind()\fR will be called, and the current
4402 context stack frame will be re\-processed. Thus it is vital that all steps
4403 in popping a context are done in such a way to support reentrancy. The
4404 other alternative, of decrementing \f(CW\*(C`cxstack_ix\*(C'\fR \fIbefore\fR processing the
4405 frame, would lead to leaks and the like if something died halfway through,
4406 or overwriting of the current frame.
4407 .PP
4408 \&\f(CW\*(C`CX_LEAVE_SCOPE\*(C'\fR itself is safely re\-entrant: if only half the savestack
4409 items have been popped before dying and getting trapped by eval, then the
4410 \&\f(CW\*(C`CX_LEAVE_SCOPE\*(C'\fRs in \f(CW\*(C`dounwind\*(C'\fR or \f(CW\*(C`pp_leaveeval\*(C'\fR will continue where
4411 the first one left off.
4412 .PP
4413 The next step is the type\-specific context processing; in this case
4414 \&\f(CW\*(C`cx_popsub\*(C'\fR. In part, this looks like:
4415 .PP
4416 .Vb 4
4417 \& cv = cx\->blk_sub.cv;
4418 \& CvDEPTH(cv) = cx\->blk_sub.olddepth;
4419 \& cx\->blk_sub.cv = NULL;
4420 \& SvREFCNT_dec(cv);
4421 .Ve
4422 .PP
4423 where it\*(Aqs processing the just\-executed CV. Note that before it decrements
4424 the CV\*(Aqs reference count, it nulls the \f(CW\*(C`blk_sub.cv\*(C'\fR. This means that if
4425 it re\-enters, the CV won\*(Aqt be freed twice. It also means that you can\*(Aqt
4426 rely on such type\-specific fields having useful values after the return
4427 from \f(CW\*(C`cx_popfoo\*(C'\fR.
4428 .PP
4429 Next, \f(CW\*(C`cx_popblock\*(C'\fR restores all the various interpreter vars to their
4430 previous values or previous high water marks; it expands to:
4431 .PP
4432 .Vb 5
4433 \& PL_markstack_ptr = PL_markstack + cx\->blk_oldmarksp;
4434 \& PL_scopestack_ix = cx\->blk_oldscopesp;
4435 \& PL_curpm = cx\->blk_oldpm;
4436 \& PL_curcop = cx\->blk_oldcop;
4437 \& PL_tmps_floor = cx\->blk_old_tmpsfloor;
4438 .Ve
4439 .PP
4440 Note that it \fIdoesn\*(Aqt\fR restore \f(CW\*(C`PL_stack_sp\*(C'\fR; as mentioned earlier,
4441 which value to restore it to depends on the context type (specifically
4442 \&\f(CW\*(C`for (list) {}\*(C'\fR), and what args (if any) it returns; and that will
4443 already have been sorted out earlier by \f(CWleave_adjust_stacks()\fR.
4444 .PP
4445 Finally, the context stack pointer is actually decremented by \f(CWCX_POP(cx)\fR.
4446 After this point, it\*(Aqs possible that the current context frame could
4447 be overwritten by other contexts being pushed. Although things like ties
4448 and \f(CW\*(C`DESTROY\*(C'\fR are supposed to work within a new context stack, it\*(Aqs best
4449 not to assume this. Indeed on debugging builds, \f(CWCX_POP(cx)\fR deliberately
4450 sets \f(CW\*(C`cx\*(C'\fR to null to detect code that is still relying on the field
4451 values in that context frame. Note in the \f(CWpp_leavesub()\fR example above,
4452 we grab \f(CW\*(C`blk_sub.retop\*(C'\fR \fIbefore\fR calling \f(CW\*(C`CX_POP\*(C'\fR.
4453 .SS "Redoing contexts"
4454 .IX Subsection "Redoing contexts"
4455 Finally, there is \f(CWcx_topblock(cx)\fR, which acts like a super\-\f(CW\*(C`nextstate\*(C'\fR
4456 as regards to resetting various vars to their base values. It is used in
4457 places like \f(CW\*(C`pp_next\*(C'\fR, \f(CW\*(C`pp_redo\*(C'\fR and \f(CW\*(C`pp_goto\*(C'\fR where rather than
4458 exiting a scope, we want to re\-initialise the scope. As well as resetting
4459 \&\f(CW\*(C`PL_stack_sp\*(C'\fR like \f(CW\*(C`nextstate\*(C'\fR, it also resets \f(CW\*(C`PL_markstack_ptr\*(C'\fR,
4460 \&\f(CW\*(C`PL_scopestack_ix\*(C'\fR and \f(CW\*(C`PL_curpm\*(C'\fR. Note that it doesn\*(Aqt do a
4461 \&\f(CW\*(C`FREETMPS\*(C'\fR.
4462 .SH "Reference\-counted argument stack"
4463 .IX Header "Reference-counted argument stack"
4464 .SS Introduction
4465 .IX Subsection "Introduction"
4466 As of perl 5.40, there is a build option, \f(CW\*(C`PERL_RC_STACK\*(C'\fR, not enabled by
4467 default, which requires that items pushed onto, or popped off the argument
4468 stack have their reference counts adjusted. It is intended that eventually
4469 this will be the default way (and finally the only way) to configure perl.
4470 .PP
4471 The macros which manipulate the stack such as \fBPUSHs()\fR and \fBPOPs()\fR don\*(Aqt
4472 adjust the reference count of the SV. Most of the time this is fine, since
4473 something else is keeping the SV alive while on the argument stack, such
4474 a pointer from the TEMPs stack, or from the pad (e.g. a lexical variable
4475 or a \f(CW\*(C`PADTMP\*(C'\fR). Occasionally this can go horribly wrong. For example,
4476 this code:
4477 .PP
4478 .Vb 3
4479 \& my @a = (1,2,3);
4480 \& sub f { @a = (); print "(@_)\en" };
4481 \& f(@a, 4);
4482 .Ve
4483 .PP
4484 may print undefined or random freed values, since some of the elements of
4485 \&\f(CW@_\fR, which have been aliased to the elements of \f(CW@a\fR, have been freed.
4486 \&\f(CW\*(C`PERL_RC_STACK\*(C'\fR is intended to fix this by making each SV pointer on the
4487 argument stack increment the reference count (RC) of the SV by one.
4488 .PP
4489 In this new environment, unmodified existing PP and XS functions, which
4490 have been written assuming a non reference\-counted stack (non\-RC for
4491 short), are called via special wrapper functions which adjust the stack
4492 before and after. At the moment there is no API to write an RC XS
4493 function, so all XS code will continue to be called via a wrapper (which
4494 makes them slightly slower), but means that in general, CPAN distributions
4495 containing XS code continue to work without modification.
4496 .PP
4497 However, PP functions, either in perl core, or those in XS functions used
4498 to implement custom ops or to override the PP functions for built\-in ops,
4499 need dealing with specially. For the latter, they can just be wrapped;
4500 this involves the least work, but has a performance impact. In the longer
4501 term, and for core PP functions, they need unwrapping and rewriting using
4502 a new API. With this, the old macros such as \fBPUSHs()\fR have been replaced
4503 with a new set of (mostly inline) functions with a common prefix, such as
4504 \&\fBrpp_push_1()\fR. "RPP" stands for "reference\-counted push and pop functions".
4505 The new functions modify the reference count on \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds,
4506 while leaving them unadjusted otherwise. Thus in core they generally work
4507 in both cases, while in XS code they are portable to older perl versions
4508 via \f(CW\*(C`PPPort\*(C'\fR (XXX assuming that they get been added to \f(CW\*(C`PPPort\*(C'\fR).
4509 .PP
4510 The rest of this section is mainly concerned with how to convert existing
4511 PP functions, and how to write new PP functions to use the new \f(CW\*(C`rpp_\*(C'\fR
4512 API.
4513 .PP
4514 A reference\-counted perl can be built using the PERL_RC_STACK define.
4515 For development and debugging purposes, it is best to enable leaking
4516 scalar debugging too, as that displays extra information about scalars
4517 that have leaked or been prematurely freed.
4518 .PP
4519 .Vb 2
4520 \& Configure \-DDEBUGGING \e
4521 \& \-Accflags=\*(Aq\-DPERL_RC_STACK \-DDEBUG_LEAKING_SCALARS\*(Aq
4522 .Ve
4523 .SS "Reference counted stack states"
4524 .IX Subsection "Reference counted stack states"
4525 In the new regime, the current argument stack can be in one of three
4526 states, which can be determined by the shown expression.
4527 .IP \(bu 4
4528 not reference\-counted
4529 .Sp
4530 .Vb 1
4531 \& !AvREAL(PL_curstack)
4532 .Ve
4533 .Sp
4534 In this case, perl will assume when emptying the stack (such as during a
4535 \&\fBcroak()\fR) that the items on it don\*(Aqt need freeing. This is the traditional
4536 perl behaviour. On \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds, such stacks will be rarely
4537 encountered.
4538 .IP \(bu 4
4539 fully reference\-counted
4540 .Sp
4541 .Vb 1
4542 \& AvREAL(PL_curstack) && !PL_curstackinfo\->si_stack_nonrc_base
4543 .Ve
4544 .Sp
4545 All the items on the stack are reference counted, and will be freed by
4546 functions like \fBrpp_popfree_1()\fR or if perl \fBcroak()\fRs. This is the normal
4547 state of the stack in \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds.
4548 .IP \(bu 4
4549 partially reference\-counted (split)
4550 .Sp
4551 .Vb 1
4552 \& AvREAL(PL_curstack) && PL_curstackinfo\->si_stack_nonrc_base > 0
4553 .Ve
4554 .Sp
4555 In this case, items on the stack from the index \f(CW\*(C`si_stack_nonrc_base\*(C'\fR
4556 upwards are non\-RC; those below are RC. This state occurs when a PP or XS
4557 function has been wrapped. In this case, the wrapper function pushes a
4558 non\-RC copy of the arg pointers above the cut then calls the real
4559 function. When that returns, the wrapper function bumps up the RC of any
4560 returned args. See below for more details.
4561 .PP
4562 Note that perl uses a stack\-of\-stacks, and the \fBAvREAL()\fR and
4563 \&\f(CW\*(C`si_stack_nonrc_base\*(C'\fR states are per stack. When perl starts up, the main
4564 stack is RC, but by default, new stacks pushed in XS code via \fBPUSHSTACKi()\fR
4565 are non\-RC, so it is quite possible to get a mixture. The perl core itself
4566 uses the new \fBpush_stackinfo()\fR function which replaces \fBPUSHSTACKi()\fR and
4567 allows you to specify that the new stack should be RC by default.
4568 (XXX core mostly hasn\*(Aqt actually been updated yet to use \fBpush_stackinfo()\fR)
4569 .PP
4570 Most places in the core assume a particular RC environment. In particular,
4571 it is assumed that within a runops loop, all the PP functions are
4572 RC\-aware, either because they have been (re)written to be aware, or
4573 because they have been wrapped. Whenever a runops loop is entered via
4574 \&\fBCALLRUNOPS()\fR, it will check the current state of the stack, and if it\*(Aqs
4575 not fully RC, will temporarily update its contents to be fully RC before
4576 entering the main runops loop. Then if necessary it will restore the stack
4577 to its old state on return. This means that functions like \fBcall_sv()\fR,
4578 which can be called from any environment (e.g. RC core or wrapped and
4579 temporarily non\-RC XS code) will always do the Right Thing when invoking
4580 the runops loop, no matter what the current stack state is.
4581 .PP
4582 Similarly, croaks and the like (which can occur anywhere) have to be able
4583 to handle both stack types. So there are a few places in core \- \fBcall_sv()\fR,
4584 \&\fBeval_sv()\fR etc, \fBPerl_die_unwind()\fR and \fBS_my_exit_jump()\fR \- which have been
4585 specially crafted to handle both cases; everything else can assume a fixed
4586 environment.
4587 .SS Wrapping
4588 .IX Subsection "Wrapping"
4589 Normally a core PP function is declared like this:
4590 .PP
4591 .Vb 4
4592 \& PP(pp_foo)
4593 \& {
4594 \& ...
4595 \& }
4596 .Ve
4597 .PP
4598 This expands to something like:
4599 .PP
4600 .Vb 4
4601 \& OP* Perl_pp_foo(pTHX)
4602 \& {
4603 \& ...
4604 \& }
4605 .Ve
4606 .PP
4607 When such a function needs to be wrapped, it is instead declared as:
4608 .PP
4609 .Vb 4
4610 \& PP_wrapped(pp_foo, nargs, nlists)
4611 \& {
4612 \& ...
4613 \& }
4614 .Ve
4615 .PP
4616 which on non\-RC builds, expands to the same as \fBPP()\fR (the extra args are
4617 ignored). On RC builds it expands to something like
4618 .PP
4619 .Vb 4
4620 \& OP* Perl_pp_foo(pTHX)
4621 \& {
4622 \& return Perl_pp_wrap(aTHX_ S_Perl_pp_foo_norc, nargs, nlists);
4623 \& }
4624 \&
4625 \& STATIC OP* S_Perl_pp_foo_norc(pTHX)
4626 \& {
4627 \& ...
4628 \& }
4629 .Ve
4630 .PP
4631 Here the externally visible PP function calls \fBpp_wrap()\fR, which adjusts
4632 the stack contents, then calls the hidden real body of the PP function,
4633 then on return, adjusts the stack back.
4634 .PP
4635 There is an API macro, \fBXSPP_wrapped()\fR, intended for use on PP functions
4636 declared in XS code, It is identical to \fBPP_wrapped()\fR, except that it
4637 doesn\*(Aqt prepend a \f(CW\*(C`Perl_\*(C'\fR prefix to the function name.
4638 .PP
4639 The \f(CW\*(C`nargs\*(C'\fR and \f(CW\*(C`nlists\*(C'\fR parameters to the macro are numeric constants
4640 or simple expressions which specify how many arguments the PP function
4641 expects, or how many lists it expects. For example,
4642 .PP
4643 .Vb 1
4644 \& PP_wrapped(pp_add, 2, 0); /* consumes two args off the stack */
4645 \&
4646 \& PP_wrapped(pp_readline, /* consumes one or two args */
4647 \& ((PL_op\->op_flags & OPf_STACKED) ? 2 : 1), 0);
4648 \&
4649 \& PP_wrapped(pp_push, 0, 1); /* consumes one list */
4650 \&
4651 \& PP_wrapped(pp_aassign, 0, 2); /* consumes two lists */
4652 .Ve
4653 .PP
4654 To understand what \fBpp_wrap()\fR does, consider calling \fBPerl_pp_foo()\fR which
4655 expects three arguments. On entry the stack may look like:
4656 .PP
4657 .Vb 1
4658 \& ... A+ B+ C+
4659 .Ve
4660 .PP
4661 (where the \f(CW\*(C`+\*(C'\fR indicates that the pointers to A, B and C are each
4662 reference counted). The wrapper function \fBpp_wrap()\fR marks a cut at the
4663 current stack position using \f(CW\*(C`si_stack_nonrc_base\*(C'\fR, then, based on the
4664 value of \f(CW\*(C`nargs\*(C'\fR, pushes a copy of those three pointers above the cut:
4665 .PP
4666 .Vb 1
4667 \& ... A+ B+ C+ | A0 B0 C0
4668 .Ve
4669 .PP
4670 (where the \f(CW0\fR indicates that the pointers aren\*(Aqt RC), then calls the
4671 real PP function, \fBS_Perl_pp_foo_norc()\fR. That function processes A, B and C,
4672 pops them off the stack, and pushes some result SVs. None of this
4673 manipulation adjusts any RCs. On return to \fBpp_wrap()\fR, the stack may look
4674 something like:
4675 .PP
4676 .Vb 1
4677 \& ... A+ B+ C+ | X0 Y0
4678 .Ve
4679 .PP
4680 The wrapper function bumps up the RCs of X and Y, decrements A B C,
4681 shifts the results down and sets \f(CW\*(C`si_stack_nonrc_base\*(C'\fR to zero, leaving
4682 the stack as:
4683 .PP
4684 .Vb 1
4685 \& ... X+ Y+
4686 .Ve
4687 .PP
4688 In places like \fBpp_entersub()\fR, a similar wrapping (via the functions
4689 \&\fBrpp_invoke_xs()\fR and then \fBxs_wrap()\fR) is done when calling XS subs.
4690 .PP
4691 When \f(CW\*(C`nlists\*(C'\fR is positive, a similar action takes place, except that the
4692 mark stack is examined (and adjusted) in order to determine the number of
4693 args that need copying.
4694 .PP
4695 A complex calling environment might have multiple nested stacks with
4696 different RC states. Perl starts off with an RC stack. Then for example,
4697 \&\fBpp_entersub()\fR is called, which (via \fBxs_wrap()\fR) splits the stack and
4698 executes the XS function in a non\-RC environment. That function may call
4699 \&\fBPUSHSTACKi()\fR, which creates a new non\-RC stack, then calls \fBcall_sv()\fR, which
4700 does \fBCALLRUNOPS()\fR, which causes the new stack to temporarily become RC.
4701 Then a tied method is called, which pushes a new RC stack, and so on. (XXX
4702 currently tied methods actually push a non\-RC stack. To be fixed soon).
4703 .SS "(Re)writing a PP function using the \fBrpp_()\fP API"
4704 .IX Subsection "(Re)writing a PP function using the rpp_() API"
4705 Wrapping a PP function has a performance overhead, and is there mainly as
4706 a temporary crutch. Eventually, PP functions should be updated to use
4707 \&\fBrpp_()\fR functions, and any new PP functions should be written this way from
4708 scratch and thus not ever need wrapping.
4709 .PP
4710 A couple examples of core PP functions being converted can be seen in the
4711 commits \f(CW\*(C`v5.39.1\-304\-g205fcd8410\*(C'\fR and \f(CW\*(C`v5.39.1\-303\-g2fe263a83a\*(C'\fR, which
4712 demonstrate a unary and a binary op being converted (\fBpp_not()\fR and
4713 \&\fBpp_and()\fR).
4714 .PP
4715 The traditional PP stack API consisted of a \f(CW\*(C`dSP\*(C'\fR declaration, plus a
4716 number of macros to push, pop and extend the stack. A \fIvery simplified\fR
4717 \&\fBpp_add()\fR function might look something like:
4718 .PP
4719 .Vb 11
4720 \& PP(pp_add)
4721 \& {
4722 \& dSP;
4723 \& dTARGET;
4724 \& IV right = SvIV(POPs);
4725 \& IV left = SvIV(POPs);
4726 \& TARGi(left + right, 1);
4727 \& PUSHs(TARG);
4728 \& PUTBACK;
4729 \& return NORMAL;
4730 \& }
4731 .Ve
4732 .PP
4733 which expands to something like:
4734 .PP
4735 .Vb 10
4736 \& {
4737 \& SV **sp = PL_stack_sp;
4738 \& SV *targ = PAD_SV(PL_op\->op_targ);
4739 \& IV right = SvIV(*sp\-\-);
4740 \& IV left = SvIV(*sp\-\-);
4741 \& sv_setiv(targ, left + right);
4742 \& *++sp = targ;
4743 \& PL_stack_sp = sp;
4744 \& return PL_op\->op_next;
4745 \& }
4746 .Ve
4747 .PP
4748 The whole \f(CW\*(C`dSP\*(C'\fR thing harks back to the days before decent optimising
4749 compilers. It was always error\-prone, e.g. if you forgot a \f(CW\*(C`PUTBACK\*(C'\fR or
4750 \&\f(CW\*(C`SPAGAIN\*(C'\fR. The new API always just accesses \f(CW\*(C`PL_stack_sp\*(C'\fR directly. In
4751 fact the first step of upgrading a PP function is always to remove the
4752 \&\f(CW\*(C`dSP\*(C'\fR declaration. This has the happy side effect that any old\-style
4753 macros left in the pp function which implicitly use \f(CW\*(C`sp\*(C'\fR will become
4754 compile errors. The existence of a \f(CW\*(C`dSP\*(C'\fR somewhere in core is a good sign
4755 that that function still needs updating.
4756 .PP
4757 An obvious question is: why not just modify the definitions of the \fBPUSHs()\fR
4758 etc macros to modify reference counts on RC builds? The basic problem is
4759 that an SV may now be kept alive only by a single reference count from
4760 the stack (formerly, they tended to be on the TEMPs stack too). So in code
4761 like:
4762 .PP
4763 .Vb 2
4764 \& SV *sv = POPs;
4765 \& IV i = SvIV(sv);
4766 .Ve
4767 .PP
4768 including an \fBSvREFCNT_dec()\fR in the \f(CW\*(C`POPs\*(C'\fR macro definition would cause
4769 \&\f(CW\*(C`sv\*(C'\fR to be freed immediately, before its integer value can be read.
4770 .PP
4771 A potential issue with the new regime is that perl can croak at basically
4772 any point in execution (e.g. the \fBSvIV()\fR above might call \fBFETCH()\fR on a tied
4773 variable which then croaks). Thus at all times, the RC of each SV must be
4774 properly accounted for. In the example above, a naive approach to avoiding
4775 a premature free of \f(CW\*(C`sv\*(C'\fR might be:
4776 .PP
4777 .Vb 3
4778 \& SV *sv = *PL_stack_sp\-\-;
4779 \& IV i = SvIV(sv);
4780 \& SvREFCNT_dec(sv); // got i, so ok to free sv now
4781 .Ve
4782 .PP
4783 but that means that \f(CW\*(C`sv\*(C'\fR leaks if \fBSvIV()\fR triggers a croak.
4784 .PP
4785 To avoid that, the new regime has the general outline that arguments are
4786 left on the stack \fIuntil they are finished with\fR, then removed and their
4787 reference count adjusted at that point. With the new API, the \fBpp_add()\fR
4788 function looks something like:
4789 .PP
4790 .Vb 8
4791 \& {
4792 \& dTARGET;
4793 \& IV right = SvIV(PL_stack_sp[ 0]); // NB: arguments left on stack
4794 \& IV left = SvIV(PL_stack_sp[\-1]);
4795 \& TARGi(left + right, 1);
4796 \& rpp_replace_2_1(targ);
4797 \& return NORMAL;
4798 \& }
4799 .Ve
4800 .PP
4801 The \fBrpp_replace_2_1()\fR function pops two values off the stack and pushes
4802 one new value on, while adjusting reference counts as appropriate
4803 (depending on whether built with \f(CW\*(C`PERL_RC_STACK\*(C'\fR or not).
4804 .PP
4805 The \fBrpp_()\fR functions in the new API will be described in detail below, but
4806 in summary:
4807 .PP
4808 .Vb 2
4809 \& new function approximate old equivant
4810 \& \-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
4811 \&
4812 \& rpp_extend(n) EXTEND(SP, n)
4813 \&
4814 \& rpp_push_1(sv) PUSHs(sv)
4815 \& rpp_push_2(sv1, sv2)) PUSHs(sv1); PUSHs(sv2)
4816 \& rpp_xpush_1(sv) XPUSHs(sv)
4817 \& rpp_xpush_2(sv1, sv2)) EXTEND(SP,2); PUSHs(sv1); PUSHs(sv2);
4818 \&
4819 \& rpp_push_1_norc(sv) mPUSHs(sv) // on RC builds, skips RC++;
4820 \& // on non\-RC builds, mortalises
4821 \& rpp_popfree_1() (void)POPs;
4822 \& rpp_popfree_2() (void)POPs; (void)POPs;
4823 \& rpp_popfree_to(svp) PL_stack_sp = svp;
4824 \& rpp_obliterate_stack_to(ix) // see description below
4825 \&
4826 \& sv = rpp_pop_1_norc() sv = SvREFCNT_inc(POPs)
4827 \&
4828 \& rpp_replace_1_1(sv) (void)POPs; PUSHs(sv);
4829 \& rpp_replace_2_1(sv) (void)POPs; (void)POPs; PUSHs(sv);
4830 \& rpp_replace_at(sp, sv) *sp = sv;
4831 \& rpp_replace_at_norc(sp, sv) *sp = sv_2mortal(sv);
4832 \&
4833 \& rpp_context(mark, gimme,
4834 \& extra) SP \-= extra;
4835 \& // impose void/scalar/list context on return args
4836 \& SP = (gimme == G_VOID) ? mark : ....
4837 \&
4838 \& rpp_try_AMAGIC_1() tryAMAGICun_MG()
4839 \& rpp_try_AMAGIC_2() tryAMAGICbin_MG()
4840 \&
4841 \& rpp_is_lone(sv) SvTEMP(sv) && SvREFCNT(sv) == 1
4842 \& rpp_stack_is_rc() no equivalent
4843 \&
4844 \& rpp_invoke_xs(cv) CvXSUB(cv)(aTHX_ cv);
4845 \&
4846 \&
4847 \& (no replacement) dATARGET // just write the macro body in full
4848 .Ve
4849 .PP
4850 There are also some \f(CW\*(C`_NN\*(C'\fR variants which assume that any items being
4851 removed from the stack are non\-NULL, and so are slightly more efficient:
4852 .PP
4853 .Vb 3
4854 \& rpp_popfree_1_NN()
4855 \& rpp_popfree_2_NN()
4856 \& rpp_popfree_to_NN(svp)
4857 \&
4858 \& rpp_replace_1_1_NN(sv)
4859 \& rpp_replace_2_1_NN(sv)
4860 \& rpp_replace_at_NN(sp, sv)
4861 \& rpp_replace_at_norc_NN(sp, sv)
4862 .Ve
4863 .PP
4864 There are also a few \f(CW\*(C`_IMM\*(C'\fR variants, which expect the single pushed or
4865 replacement value to be an immortal, such as \f(CW&PL_sv_undef\fR \- this skips
4866 incrementing the ref count of the immortal SV. It doesn\*(Aqt matter if the
4867 ref count of the SV prematurely reaches zero, as \fBsv_free2()\fR will just
4868 resurrect it. Not every variant is provided; if a suitable one
4869 doesn\*(Aqt exist, just using a standard \f(CW\*(C`_1\*(C'\fR version is fine, albeit
4870 slightly slower.
4871 .PP
4872 .Vb 4
4873 \& rpp_push_IMM(&PL_sv_undef)
4874 \& rpp_xpush_IMM(&PL_sv_zero)
4875 \& rpp_replace_1_IMM_NN(&PL_sv_yes)
4876 \& rpp_replace_2_IMM_NN(&PL_sv_no)
4877 .Ve
4878 .PP
4879 Other new C and perl functions related to reference\-counted stacks are:
4880 .PP
4881 .Vb 3
4882 \& push_stackinfo(type,rc) PUSHSTACKi(type)
4883 \& pop_stackinfo() POPSTACK()
4884 \& switch_argstack(to) SWITCHSTACK(from,to)
4885 \&
4886 \& (Internals::stack_refcounted() & 1) # perl built with PERL_RC_STACK
4887 .Ve
4888 .PP
4889 Some of these new functions are trivial, but should be used in preference
4890 to writing direct code because they will work on both RC and non\-RC
4891 builds, and may do extra checks and assertions on \f(CW\*(C`DEBUGGING\*(C'\fR builds.
4892 .PP
4893 Note that \fBrpp_popfree_1()\fR etc aren\*(Aqt direct replacements for \f(CW\*(C`POPs\*(C'\fR. The
4894 \&\fBrpp_()\fR variants don\*(Aqt return a value and are intended to be called when
4895 the SV is finished with. So
4896 .PP
4897 .Vb 2
4898 \& SV *sv = POPs;
4899 \& ... do stuff with sv ...
4900 .Ve
4901 .PP
4902 becomes
4903 .PP
4904 .Vb 3
4905 \& SV *sv = *PL_stack_sp;
4906 \& ... do stuff with sv ...
4907 \& rpp_popfree_1(); /* does SvREFCNT_dec(*PL_stack_sp\-\-) */
4908 .Ve
4909 .PP
4910 The \fBrpp_replace_M_N()\fR functions are shortcuts for popping and freeing \f(CW\*(C`M\*(C'\fR
4911 items then pushing and bumping up the RCs of \f(CW\*(C`N\*(C'\fR items. Note that they
4912 handle edge cases such as an old and new SV being the same.
4913 .PP
4914 rpp_replace_at(sp, sv) is similar to \fBrpp_replace_1_1()\fR, except that
4915 it replaces an SV at an address in the stack rather than at the top.
4916 .PP
4917 rpp_replace_at_norc(sp, sv) is similar to \fBrpp_replace_at()\fR, except that
4918 it assumes that \f(CW\*(C`sv\*(C'\fR already has a bumped reference count. So, a bit
4919 like \fBrpp_push_1_norc()\fR (see below), it doesn\*(Aqt bother increasing \f(CW\*(C`sv\*(C'\fR\*(Aqs
4920 reference count, or on non\-RC builds it mortalises it instead.
4921 .PP
4922 rpp_popfree_to(svp) is designed to replace code like
4923 .PP
4924 .Vb 1
4925 \& PL_stack_sp = PL_stack_base + cx\->blk_oldsp;
4926 .Ve
4927 .PP
4928 which typically appears in list ops or scope exits when the arguments are
4929 finished with. Left unaltered, all the SVs above \f(CW\*(C`oldsp\*(C'\fR would leak. The
4930 new approach is
4931 .PP
4932 .Vb 1
4933 \& rpp_popfree_to(PL_stack_base + cx\->blk_oldsp);
4934 .Ve
4935 .PP
4936 There is a rarely\-used variant of this, \fBrpp_obliterate_stack_to()\fR, which
4937 pops the stack back to the specified index regardless of the current RC
4938 state of the stack. So for example if the stack is split, it will only
4939 adjust the RCs of any SVs which are below the split point, while
4940 \&\fBrpp_popfree_to()\fR would mindlessly free \fIall\fR SVs (on RC builds anyway).
4941 For normal PP functions you should only ever use \fBrpp_popfree_to()\fR, which
4942 is faster.
4943 .PP
4944 There are no new equivalents for all the convenience macros like \fBPOPi()\fR
4945 and (shudder) \fBdPOPPOPiirl()\fR. These should be replaced with the \fBrpp_()\fR
4946 functions above and with the conversions and variable declarations being
4947 made explicit, e.g. \fBdPOPPOPiirl()\fR becomes:
4948 .PP
4949 .Vb 3
4950 \& IV right = SvIV(PL_stack_sp[ 0]);
4951 \& IV left = SvIV(PL_stack_sp[\-1]);
4952 \& rpp_popfree_2();
4953 .Ve
4954 .PP
4955 A couple of the \fBrpp_()\fR functions with \f(CW\*(C`norc\*(C'\fR in their names don\*(Aqt adjust
4956 the reference count on RC builds (but, conversely, do on non\-RC builds).
4957 .PP
4958 rpp_push_1_norc(sv) does a simple \f(CW\*(C`*++PL_stack_sp = sv\*(C'\fR on RC builds. It
4959 is typically used to "root" a newly\-created SV, which already has an RC of
4960 1. On non\-RC builds it mortalises the SV instead. So for example, code
4961 which used to look like
4962 .PP
4963 .Vb 1
4964 \& mPUSHs(newSViv(i));
4965 .Ve
4966 .PP
4967 and which expanded to the equivalent of:
4968 .PP
4969 .Vb 1
4970 \& PUSHs(sv_2mortal(newSViv(i));
4971 .Ve
4972 .PP
4973 should be rewritten as:
4974 .PP
4975 .Vb 1
4976 \& rpp_push_1_norc(newSViv(i));
4977 .Ve
4978 .PP
4979 This is because \fBnewSViv()\fR and similar create a new SV with a reference
4980 count one too high (1 rather than 0). This count is then "donated" to the
4981 stack by pushing it. Conversely on non\-RC builds, the count is donated to
4982 the TEMPs stack.
4983 .PP
4984 Similarly, on RC builds, \f(CW\*(C`sv = rpp_pop_1_norc()\*(C'\fR does a simple
4985 \&\f(CW\*(C`sv = *PL_stack_sv\-\-\*(C'\fR without adjusting the reference count, while on
4986 non\-RC builds it actually increments the SV\*(Aqs reference count. It is
4987 intended for cases where you immediately want to increment the reference
4988 count again after popping, e.g. where the SV is to be immediately embedded
4989 somewhere. For example this code:
4990 .PP
4991 .Vb 4
4992 \& SV *sv = PL_stack_sp[0];
4993 \& SvREFCNT_inc(sv);
4994 \& av_store(av, i, sv); /* in real life should check return value */
4995 \& rpp_popfree_1();
4996 .Ve
4997 .PP
4998 can be more efficiently written as
4999 .PP
5000 .Vb 1
5001 \& av_store(av, i, rpp_pop_1_norc());
5002 .Ve
5003 .PP
5004 By using this function, the code works correctly on both RC and non\-RC
5005 builds.
5006 .PP
5007 A common operation on list ops is to impose void, scalar or list context
5008 on the return arguments, possibly discarding all, or all except one, of
5009 them. rpp_context(mark, gimme, extra) does this. As a first step (for
5010 convenience and efficiency) it notionally pops \f(CW\*(C`extra\*(C'\fR args off the
5011 stack. Then for list context, leaves things as is. For void context, the
5012 stack pointer is reset to mark, and everything above is popped. For
5013 scalar, the top argument (or &PL_sv_undef) is moved from the top to
5014 mark+1 and everything above is discarded.
5015 .PP
5016 The macros which appear at the start of many PP functions to check for
5017 unary or binary op overloading (among other things) have been replaced
5018 with \fBrpp_try_AMAGIC_1()\fR and \fB_2()\fR inline functions, which now rely on the
5019 calling PP function to choose whether to return immediately rather than
5020 the return being hidden away in the macro.
5021 .PP
5022 The \fBrpp_invoke_xs()\fR function calls the XS function associated with the CV,
5023 but may do so via a wrapper function to adjust the stack as necessary.
5024 .PP
5025 In the spirit of hiding away less in macros, \f(CW\*(C`dATARGET\*(C'\fR hasn\*(Aqt been given
5026 a replacement; where its effect is needed, it is now written out in full;
5027 see \fBpp_add()\fR for an example.
5028 .PP
5029 Finally, a couple of \fBrpp()\fR functions provide information rather than
5030 manipulate the stack.
5031 .PP
5032 rpp_is_lone(sv) indicates whether \f(CW\*(C`sv\*(C'\fR, assumed to be still on the stack,
5033 it kept alive only by a single reference\-counted pointer from the argument
5034 and/or temps stacks, and thus is a candidate for some optimisations (like
5035 skipping the copying of return arguments from a subroutine call).
5036 .PP
5037 \&\fBrpp_stack_is_rc()\fR indicates whether the current stack is currently
5038 reference\-counted. It\*(Aqs used mainly in a few places like \fBcall_sv()\fR which
5039 can be called from anywhere, and thus have to deal with both cases.
5040 .PP
5041 So for example, rather than using \fBrpp_xpush_1()\fR, \fBcall_sv()\fR has lines like:
5042 .PP
5043 .Vb 6
5044 \& rpp_extend(1);
5045 \& *++PL_stack_sp = sv;
5046 \& #ifdef PERL_RC_STACK
5047 \& if (rpp_stack_is_rc())
5048 \& SvREFCNT_inc_simple_void_NN(sv);
5049 \& #endif
5050 .Ve
5051 .PP
5052 which works on both standard builds and RC builds, and works whether
5053 \&\fBcall_sv()\fR is called from a standard PP function (\fBrpp_stack_is_rc()\fR is
5054 true) or from a wrapped PP or XS function (\fBrpp_stack_is_rc()\fR is false).
5055 Note that you\*(Aqre unlikely to need to use this function, as in most places,
5056 such as PP or XS functions, it is always RC or non\-RC respectively. In
5057 fact on debugging builds under \f(CW\*(C`PERL_RC_STACK\*(C'\fR, \fBPUSHs()\fR and similar
5058 macros include an \f(CW\*(C`assert(!rpp_stack_is_rc())\*(C'\fR, while \fBrpp_push_1()\fR and
5059 similar functions have \f(CW\*(C`assert(rpp_stack_is_rc())\*(C'\fR.
5060 .PP
5061 The macros for pushing new stackinfos have been replaced with inline
5062 functions which don\*(Aqt rely on \f(CW\*(C`dSP\*(C'\fR being in scope, and which have less
5063 ambiguous names: they make it clear that a new \fIstackinfo\fR is being
5064 pushed, rather than just some sort of \fIstack\fR. \fBpush_stackinfo()\fR also has
5065 a boolean argument indicating whether the new argument stack should be
5066 reference\-counted or not. For backwards compatibility, PUSHSTACKi(type) is
5067 defined to be push_stackinfo(type, 0).
5068 .PP
5069 Some test scripts check for things like leaks by testing that the
5070 reference count of a particular variable has an expected value. If this
5071 is different on a perl built with \f(CW\*(C`PERL_RC_STACK\*(C'\fR, then the perl
5072 function \fBInternals::stack_refcounted()\fR can be used. This returns an
5073 integer, the lowest bit of which indicates that perl was built with
5074 \&\f(CW\*(C`PERL_RC_STACK\*(C'\fR. Other bits are reserved for future use and should be
5075 masked out.
5076 .SH "Slab\-based operator allocation"
5077 .IX Header "Slab-based operator allocation"
5078 \&\fBNote:\fR this section describes a non\-public internal API that is subject
5079 to change without notice.
5080 .PP
5081 Perl\*(Aqs internal error\-handling mechanisms implement \f(CW\*(C`die\*(C'\fR (and its internal
5082 equivalents) using longjmp. If this occurs during lexing, parsing or
5083 compilation, we must ensure that any ops allocated as part of the compilation
5084 process are freed. (Older Perl versions did not adequately handle this
5085 situation: when failing a parse, they would leak ops that were stored in
5086 C \f(CW\*(C`auto\*(C'\fR variables and not linked anywhere else.)
5087 .PP
5088 To handle this situation, Perl uses \fIop slabs\fR that are attached to the
5089 currently\-compiling CV. A slab is a chunk of allocated memory. New ops are
5090 allocated as regions of the slab. If the slab fills up, a new one is created
5091 (and linked from the previous one). When an error occurs and the CV is freed,
5092 any ops remaining are freed.
5093 .PP
5094 Each op is preceded by two pointers: one points to the next op in the slab, and
5095 the other points to the slab that owns it. The next\-op pointer is needed so
5096 that Perl can iterate over a slab and free all its ops. (Op structures are of
5097 different sizes, so the slab\*(Aqs ops can\*(Aqt merely be treated as a dense array.)
5098 The slab pointer is needed for accessing a reference count on the slab: when
5099 the last op on a slab is freed, the slab itself is freed.
5100 .PP
5101 The slab allocator puts the ops at the end of the slab first. This will tend to
5102 allocate the leaves of the op tree first, and the layout will therefore
5103 hopefully be cache\-friendly. In addition, this means that there\*(Aqs no need to
5104 store the size of the slab (see below on why slabs vary in size), because Perl
5105 can follow pointers to find the last op.
5106 .PP
5107 It might seem possible to eliminate slab reference counts altogether, by having
5108 all ops implicitly attached to \f(CW\*(C`PL_compcv\*(C'\fR when allocated and freed when the
5109 CV is freed. That would also allow \f(CW\*(C`op_free\*(C'\fR to skip \f(CW\*(C`FreeOp\*(C'\fR altogether, and
5110 thus free ops faster. But that doesn\*(Aqt work in those cases where ops need to
5111 survive beyond their CVs, such as re\-evals.
5112 .PP
5113 The CV also has to have a reference count on the slab. Sometimes the first op
5114 created is immediately freed. If the reference count of the slab reaches 0,
5115 then it will be freed with the CV still pointing to it.
5116 .PP
5117 CVs use the \f(CW\*(C`CVf_SLABBED\*(C'\fR flag to indicate that the CV has a reference count
5118 on the slab. When this flag is set, the slab is accessible via \f(CW\*(C`CvSTART\*(C'\fR when
5119 \&\f(CW\*(C`CvROOT\*(C'\fR is not set, or by subtracting two pointers \f(CW\*(C`(2*sizeof(I32 *))\*(C'\fR from
5120 \&\f(CW\*(C`CvROOT\*(C'\fR when it is set. The alternative to this approach of sneaking the slab
5121 into \f(CW\*(C`CvSTART\*(C'\fR during compilation would be to enlarge the \f(CW\*(C`xpvcv\*(C'\fR struct by
5122 another pointer. But that would make all CVs larger, even though slab\-based op
5123 freeing is typically of benefit only for programs that make significant use of
5124 string eval.
5125 .PP
5126 When the \f(CW\*(C`CVf_SLABBED\*(C'\fR flag is set, the CV takes responsibility for freeing
5127 the slab. If \f(CW\*(C`CvROOT\*(C'\fR is not set when the CV is freed or undeffed, it is
5128 assumed that a compilation error has occurred, so the op slab is traversed and
5129 all the ops are freed.
5130 .PP
5131 Under normal circumstances, the CV forgets about its slab (decrementing the
5132 reference count) when the root is attached. So the slab reference counting that
5133 happens when ops are freed takes care of freeing the slab. In some cases, the
5134 CV is told to forget about the slab (\f(CW\*(C`cv_forget_slab\*(C'\fR) precisely so that the
5135 ops can survive after the CV is done away with.
5136 .PP
5137 Forgetting the slab when the root is attached is not strictly necessary, but
5138 avoids potential problems with \f(CW\*(C`CvROOT\*(C'\fR being written over. There is code all
5139 over the place, both in core and on CPAN, that does things with \f(CW\*(C`CvROOT\*(C'\fR, so
5140 forgetting the slab makes things more robust and avoids potential problems.
5141 .PP
5142 Since the CV takes ownership of its slab when flagged, that flag is never
5143 copied when a CV is cloned, as one CV could free a slab that another CV still
5144 points to, since forced freeing of ops ignores the reference count (but asserts
5145 that it looks right).
5146 .PP
5147 To avoid slab fragmentation, freed ops are marked as freed and attached to the
5148 slab\*(Aqs freed chain (an idea stolen from DBM::Deep). Those freed ops are reused
5149 when possible. Not reusing freed ops would be simpler, but it would result in
5150 significantly higher memory usage for programs with large \f(CW\*(C`if (DEBUG) {...}\*(C'\fR
5151 blocks.
5152 .PP
5153 \&\f(CW\*(C`SAVEFREEOP\*(C'\fR is slightly problematic under this scheme. Sometimes it can cause
5154 an op to be freed after its CV. If the CV has forcibly freed the ops on its
5155 slab and the slab itself, then we will be fiddling with a freed slab. Making
5156 \&\f(CW\*(C`SAVEFREEOP\*(C'\fR a no\-op doesn\*(Aqt help, as sometimes an op can be savefreed when
5157 there is no compilation error, so the op would never be freed. It holds
5158 a reference count on the slab, so the whole slab would leak. So \f(CW\*(C`SAVEFREEOP\*(C'\fR
5159 now sets a special flag on the op (\f(CW\*(C`\->op_savefree\*(C'\fR). The forced freeing of
5160 ops after a compilation error won\*(Aqt free any ops thus marked.
5161 .PP
5162 Since many pieces of code create tiny subroutines consisting of only a few ops,
5163 and since a huge slab would be quite a bit of baggage for those to carry
5164 around, the first slab is always very small. To avoid allocating too many
5165 slabs for a single CV, each subsequent slab is twice the size of the previous.
5166 .PP
5167 Smartmatch expects to be able to allocate an op at run time, run it, and then
5168 throw it away. For that to work the op is simply malloced when \f(CW\*(C`PL_compcv\*(C'\fR hasn\*(Aqt
5169 been set up. So all slab\-allocated ops are marked as such (\f(CW\*(C`\->op_slabbed\*(C'\fR),
5170 to distinguish them from malloced ops.
5171 .SH AUTHORS
5172 .IX Header "AUTHORS"
5173 Until May 1997, this document was maintained by Jeff Okamoto
5174 <[email protected]>. It is now maintained as part of Perl
5175 itself by the Perl 5 Porters <perl5\[email protected]>.
5176 .PP
5177 With lots of help and suggestions from Dean Roehrich, Malcolm Beattie,
5178 Andreas Koenig, Paul Hudson, Ilya Zakharevich, Paul Marquess, Neil
5179 Bowers, Matthew Green, Tim Bunce, Spider Boardman, Ulrich Pfeifer,
5180 Stephen McCamant, and Gurusamy Sarathy.
5181 .SH "SEE ALSO"
5182 .IX Header "SEE ALSO"
5183 perlapi, perlintern, perlxs, perlembed