Debian Codegraph @

upstream/debian-unstable/man1/perlguts.1

raw
1 .\" -*- mode: troff; coding: utf-8 -*-
2 .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
3 .\"
4 .\" Standard preamble:
5 .\" ========================================================================
6 .de Sp \" Vertical space (when we can't use .PP)
7 .if t .sp .5v
8 .if n .sp
9 ..
10 .de Vb \" Begin verbatim text
11 .ft CW
12 .nf
13 .ne \\$1
14 ..
15 .de Ve \" End verbatim text
16 .ft R
17 .fi
18 ..
19 .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
20 .ie n \{\
21 . ds C` ""
22 . ds C' ""
23 'br\}
24 .el\{\
25 . ds C`
26 . ds C'
27 'br\}
28 .\"
29 .\" Escape single quotes in literal strings from groff's Unicode transform.
30 .ie \n(.g .ds Aq \(aq
31 .el .ds Aq '
32 .\"
33 .\" If the F register is >0, we'll generate index entries on stderr for
34 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
35 .\" entries marked with X<> in POD. Of course, you'll have to process the
36 .\" output yourself in some meaningful fashion.
37 .\"
38 .\" Avoid warning from groff about undefined register 'F'.
39 .de IX
40 ..
41 .nr rF 0
42 .if \n(.g .if rF .nr rF 1
43 .if (\n(rF:(\n(.g==0)) \{\
44 . if \nF \{\
45 . de IX
46 . tm Index:\\$1\t\\n%\t"\\$2"
47 ..
48 . if !\nF==2 \{\
49 . nr % 0
50 . nr F 2
51 . \}
52 . \}
53 .\}
54 .rr rF
55 .\" ========================================================================
56 .\"
57 .IX Title "PERLGUTS 1"
58 .TH PERLGUTS 1 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide"
59 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
60 .\" way too many mistakes in technical documents.
61 .if n .ad l
62 .nh
63 .SH NAME
64 perlguts \- Introduction to the Perl API
65 .SH DESCRIPTION
66 .IX Header "DESCRIPTION"
67 This document attempts to describe how to use the Perl API, as well as
68 to provide some info on the basic workings of the Perl core. It is far
69 from complete and probably contains many errors. Please refer any
70 questions or comments to the author below.
71 .SH Variables
72 .IX Header "Variables"
73 .SS Datatypes
74 .IX Subsection "Datatypes"
75 Perl has three typedefs that handle Perl's three main data types:
76 .PP
77 .Vb 3
78 \& SV Scalar Value
79 \& AV Array Value
80 \& HV Hash Value
81 .Ve
82 .PP
83 Each typedef has specific routines that manipulate the various data types.
84 .SS "What is an ""IV""?"
85 .IX Subsection "What is an ""IV""?"
86 Perl uses a special typedef IV which is a simple signed integer type that is
87 guaranteed to be large enough to hold a pointer (as well as an integer).
88 Additionally, there is the UV, which is simply an unsigned IV.
89 .PP
90 Perl also uses several special typedefs to declare variables to hold
91 integers of (at least) a given size.
92 Use I8, I16, I32, and I64 to declare a signed integer variable which has
93 at least as many bits as the number in its name. These all evaluate to
94 the native C type that is closest to the given number of bits, but no
95 smaller than that number. For example, on many platforms, a \f(CW\*(C`short\*(C'\fR is
96 16 bits long, and if so, I16 will evaluate to a \f(CW\*(C`short\*(C'\fR. But on
97 platforms where a \f(CW\*(C`short\*(C'\fR isn't exactly 16 bits, Perl will use the
98 smallest type that contains 16 bits or more.
99 .PP
100 U8, U16, U32, and U64 are to declare the corresponding unsigned integer
101 types.
102 .PP
103 If the platform doesn't support 64\-bit integers, both I64 and U64 will
104 be undefined. Use IV and UV to declare the largest practicable, and
105 \&\f(CW\*(C`"WIDEST_UTYPE" in perlapi\*(C'\fR for the absolute maximum unsigned, but which
106 may not be usable in all circumstances.
107 .PP
108 A numeric constant can be specified with "\f(CW\*(C`INT16_C\*(C'\fR" in perlapi,
109 "\f(CW\*(C`UINTMAX_C\*(C'\fR" in perlapi, and similar.
110 .SS "Working with SVs"
111 .IX Subsection "Working with SVs"
112 An SV can be created and loaded with one command. There are five types of
113 values that can be loaded: an integer value (IV), an unsigned integer
114 value (UV), a double (NV), a string (PV), and another scalar (SV).
115 ("PV" stands for "Pointer Value". You might think that it is misnamed
116 because it is described as pointing only to strings. However, it is
117 possible to have it point to other things. For example, it could point
118 to an array of UVs. But,
119 using it for non-strings requires care, as the underlying assumption of
120 much of the internals is that PVs are just for strings. Often, for
121 example, a trailing \f(CW\*(C`NUL\*(C'\fR is tacked on automatically. The non-string use
122 is documented only in this paragraph.)
123 .PP
124 The seven routines are:
125 .PP
126 .Vb 7
127 \& SV* newSViv(IV);
128 \& SV* newSVuv(UV);
129 \& SV* newSVnv(double);
130 \& SV* newSVpv(const char*, STRLEN);
131 \& SV* newSVpvn(const char*, STRLEN);
132 \& SV* newSVpvf(const char*, ...);
133 \& SV* newSVsv(SV*);
134 .Ve
135 .PP
136 \&\f(CW\*(C`STRLEN\*(C'\fR is an integer type (\f(CW\*(C`Size_t\*(C'\fR, usually defined as \f(CW\*(C`size_t\*(C'\fR in
137 \&\fIconfig.h\fR) guaranteed to be large enough to represent the size of
138 any string that perl can handle.
139 .PP
140 In the unlikely case of a SV requiring more complex initialization, you
141 can create an empty SV with newSV(len). If \f(CW\*(C`len\*(C'\fR is 0 an empty SV of
142 type NULL is returned, else an SV of type PV is returned with len + 1 (for
143 the \f(CW\*(C`NUL\*(C'\fR) bytes of storage allocated, accessible via SvPVX. In both cases
144 the SV has the undef value.
145 .PP
146 .Vb 3
147 \& SV *sv = newSV(0); /* no storage allocated */
148 \& SV *sv = newSV(10); /* 10 (+1) bytes of uninitialised storage
149 \& * allocated */
150 .Ve
151 .PP
152 To change the value of an \fIalready-existing\fR SV, there are eight routines:
153 .PP
154 .Vb 9
155 \& void sv_setiv(SV*, IV);
156 \& void sv_setuv(SV*, UV);
157 \& void sv_setnv(SV*, double);
158 \& void sv_setpv(SV*, const char*);
159 \& void sv_setpvn(SV*, const char*, STRLEN)
160 \& void sv_setpvf(SV*, const char*, ...);
161 \& void sv_vsetpvfn(SV*, const char*, STRLEN, va_list *,
162 \& SV **, Size_t, bool *);
163 \& void sv_setsv(SV*, SV*);
164 .Ve
165 .PP
166 Notice that you can choose to specify the length of the string to be
167 assigned by using \f(CW\*(C`sv_setpvn\*(C'\fR, \f(CW\*(C`newSVpvn\*(C'\fR, or \f(CW\*(C`newSVpv\*(C'\fR, or you may
168 allow Perl to calculate the length by using \f(CW\*(C`sv_setpv\*(C'\fR or by specifying
169 0 as the second argument to \f(CW\*(C`newSVpv\*(C'\fR. Be warned, though, that Perl will
170 determine the string's length by using \f(CW\*(C`strlen\*(C'\fR, which depends on the
171 string terminating with a \f(CW\*(C`NUL\*(C'\fR character, and not otherwise containing
172 NULs.
173 .PP
174 The arguments of \f(CW\*(C`sv_setpvf\*(C'\fR are processed like \f(CW\*(C`sprintf\*(C'\fR, and the
175 formatted output becomes the value.
176 .PP
177 \&\f(CW\*(C`sv_vsetpvfn\*(C'\fR is an analogue of \f(CW\*(C`vsprintf\*(C'\fR, but it allows you to specify
178 either a pointer to a variable argument list or the address and length of
179 an array of SVs. The last argument points to a boolean; on return, if that
180 boolean is true, then locale-specific information has been used to format
181 the string, and the string's contents are therefore untrustworthy (see
182 perlsec). This pointer may be NULL if that information is not
183 important. Note that this function requires you to specify the length of
184 the format.
185 .PP
186 The \f(CW\*(C`sv_set*()\*(C'\fR functions are not generic enough to operate on values
187 that have "magic". See "Magic Virtual Tables" later in this document.
188 .PP
189 All SVs that contain strings should be terminated with a \f(CW\*(C`NUL\*(C'\fR character.
190 If it is not \f(CW\*(C`NUL\*(C'\fR\-terminated there is a risk of
191 core dumps and corruptions from code which passes the string to C
192 functions or system calls which expect a \f(CW\*(C`NUL\*(C'\fR\-terminated string.
193 Perl's own functions typically add a trailing \f(CW\*(C`NUL\*(C'\fR for this reason.
194 Nevertheless, you should be very careful when you pass a string stored
195 in an SV to a C function or system call.
196 .PP
197 To access the actual value that an SV points to, Perl's API exposes
198 several macros that coerce the actual scalar type into an IV, UV, double,
199 or string:
200 .IP \(bu 4
201 \&\f(CWSvIV(SV*)\fR (\f(CW\*(C`IV\*(C'\fR) and \f(CWSvUV(SV*)\fR (\f(CW\*(C`UV\*(C'\fR)
202 .IP \(bu 4
203 \&\f(CWSvNV(SV*)\fR (\f(CW\*(C`double\*(C'\fR)
204 .IP \(bu 4
205 Strings are a bit complicated:
206 .RS 4
207 .IP \(bu 4
208 Byte string: \f(CW\*(C`SvPVbyte(SV*, STRLEN len)\*(C'\fR or \f(CWSvPVbyte_nolen(SV*)\fR
209 .Sp
210 If the Perl string is \f(CW"\exff\exff"\fR, then this returns a 2\-byte \f(CW\*(C`char*\*(C'\fR.
211 .Sp
212 This is suitable for Perl strings that represent bytes.
213 .IP \(bu 4
214 UTF\-8 string: \f(CW\*(C`SvPVutf8(SV*, STRLEN len)\*(C'\fR or \f(CWSvPVutf8_nolen(SV*)\fR
215 .Sp
216 If the Perl string is \f(CW"\exff\exff"\fR, then this returns a 4\-byte \f(CW\*(C`char*\*(C'\fR.
217 .Sp
218 This is suitable for Perl strings that represent characters.
219 .Sp
220 \&\fBCAVEAT\fR: That \f(CW\*(C`char*\*(C'\fR will be encoded via Perl's internal UTF\-8 variant,
221 which means that if the SV contains non-Unicode code points (e.g.,
222 0x110000), then the result may contain extensions over valid UTF\-8.
223 See "is_strict_utf8_string" in perlapi for some methods Perl gives
224 you to check the UTF\-8 validity of these macros' returns.
225 .IP \(bu 4
226 You can also use \f(CW\*(C`SvPV(SV*, STRLEN len)\*(C'\fR or \f(CWSvPV_nolen(SV*)\fR
227 to fetch the SV's raw internal buffer. This is tricky, though; if your Perl
228 string
229 is \f(CW"\exff\exff"\fR, then depending on the SV's internal encoding you might get
230 back a 2\-byte \fBOR\fR a 4\-byte \f(CW\*(C`char*\*(C'\fR.
231 Moreover, if it's the 4\-byte string, that could come from either Perl
232 \&\f(CW"\exff\exff"\fR stored UTF\-8 encoded, or Perl \f(CW"\exc3\exbf\exc3\exbf"\fR stored
233 as raw octets. To differentiate between these you \fBMUST\fR look up the
234 SV's UTF8 bit (cf. \f(CW\*(C`SvUTF8\*(C'\fR) to know whether the source Perl string
235 is 2 characters (\f(CW\*(C`SvUTF8\*(C'\fR would be on) or 4 characters (\f(CW\*(C`SvUTF8\*(C'\fR would be
236 off).
237 .Sp
238 \&\fBIMPORTANT:\fR Use of \f(CW\*(C`SvPV\*(C'\fR, \f(CW\*(C`SvPV_nolen\*(C'\fR, or
239 similarly-named macros \fIwithout\fR looking up the SV's UTF8 bit is
240 almost certainly a bug if non-ASCII input is allowed.
241 .Sp
242 When the UTF8 bit is on, the same \fBCAVEAT\fR about UTF\-8 validity applies
243 here as for \f(CW\*(C`SvPVutf8\*(C'\fR.
244 .RE
245 .RS 4
246 .Sp
247 (See "How do I pass a Perl string to a C library?" for more details.)
248 .Sp
249 In \f(CW\*(C`SvPVbyte\*(C'\fR, \f(CW\*(C`SvPVutf8\*(C'\fR, and \f(CW\*(C`SvPV\*(C'\fR, the length of the \f(CW\*(C`char*\*(C'\fR returned
250 is placed into the
251 variable \f(CW\*(C`len\*(C'\fR (these are macros, so you do \fInot\fR use \f(CW&len\fR). If you do
252 not care what the length of the data is, use \f(CW\*(C`SvPVbyte_nolen\*(C'\fR,
253 \&\f(CW\*(C`SvPVutf8_nolen\*(C'\fR, or \f(CW\*(C`SvPV_nolen\*(C'\fR instead.
254 The global variable \f(CW\*(C`PL_na\*(C'\fR can also be given to
255 \&\f(CW\*(C`SvPVbyte\*(C'\fR/\f(CW\*(C`SvPVutf8\*(C'\fR/\f(CW\*(C`SvPV\*(C'\fR
256 in this case. But that can be quite inefficient because \f(CW\*(C`PL_na\*(C'\fR must
257 be accessed in thread-local storage in threaded Perl. In any case, remember
258 that Perl allows arbitrary strings of data that may both contain NULs and
259 might not be terminated by a \f(CW\*(C`NUL\*(C'\fR.
260 .Sp
261 Also remember that C doesn't allow you to safely say \f(CW\*(C`foo(SvPVbyte(s, len),
262 len);\*(C'\fR. It might work with your
263 compiler, but it won't work for everyone.
264 Break this sort of statement up into separate assignments:
265 .Sp
266 .Vb 5
267 \& SV *s;
268 \& STRLEN len;
269 \& char *ptr;
270 \& ptr = SvPVbyte(s, len);
271 \& foo(ptr, len);
272 .Ve
273 .RE
274 .PP
275 If you want to know if the scalar value is TRUE, you can use:
276 .PP
277 .Vb 1
278 \& SvTRUE(SV*)
279 .Ve
280 .PP
281 Although Perl will automatically grow strings for you, if you need to force
282 Perl to allocate more memory for your SV, you can use the macro
283 .PP
284 .Vb 1
285 \& SvGROW(SV*, STRLEN newlen)
286 .Ve
287 .PP
288 which will determine if more memory needs to be allocated. If so, it will
289 call the function \f(CW\*(C`sv_grow\*(C'\fR. Note that \f(CW\*(C`SvGROW\*(C'\fR can only increase, not
290 decrease, the allocated memory of an SV and that it does not automatically
291 add space for the trailing \f(CW\*(C`NUL\*(C'\fR byte (perl's own string functions typically do
292 \&\f(CW\*(C`SvGROW(sv, len + 1)\*(C'\fR).
293 .PP
294 If you want to write to an existing SV's buffer and set its value to a
295 string, use \fBSvPVbyte_force()\fR or one of its variants to force the SV to be
296 a PV. This will remove any of various types of non-stringness from
297 the SV while preserving the content of the SV in the PV. This can be
298 used, for example, to append data from an API function to a buffer
299 without extra copying:
300 .PP
301 .Vb 11
302 \& (void)SvPVbyte_force(sv, len);
303 \& s = SvGROW(sv, len + needlen + 1);
304 \& /* something that modifies up to needlen bytes at s+len, but
305 \& modifies newlen bytes
306 \& eg. newlen = read(fd, s + len, needlen);
307 \& ignoring errors for these examples
308 \& */
309 \& s[len + newlen] = \*(Aq\e0\*(Aq;
310 \& SvCUR_set(sv, len + newlen);
311 \& SvUTF8_off(sv);
312 \& SvSETMAGIC(sv);
313 .Ve
314 .PP
315 If you already have the data in memory or if you want to keep your
316 code simple, you can use one of the sv_cat*() variants, such as
317 \&\fBsv_catpvn()\fR. If you want to insert anywhere in the string you can use
318 \&\fBsv_insert()\fR or \fBsv_insert_flags()\fR.
319 .PP
320 If you don't need the existing content of the SV, you can avoid some
321 copying with:
322 .PP
323 .Vb 10
324 \& SvPVCLEAR(sv);
325 \& s = SvGROW(sv, needlen + 1);
326 \& /* something that modifies up to needlen bytes at s, but modifies
327 \& newlen bytes
328 \& eg. newlen = read(fd, s, needlen);
329 \& */
330 \& s[newlen] = \*(Aq\e0\*(Aq;
331 \& SvCUR_set(sv, newlen);
332 \& SvPOK_only(sv); /* also clears SVf_UTF8 */
333 \& SvSETMAGIC(sv);
334 .Ve
335 .PP
336 Again, if you already have the data in memory or want to avoid the
337 complexity of the above, you can use \fBsv_setpvn()\fR.
338 .PP
339 If you have a buffer allocated with \fBNewx()\fR and want to set that as the
340 SV's value, you can use \fBsv_usepvn_flags()\fR. That has some requirements
341 if you want to avoid perl re-allocating the buffer to fit the trailing
342 NUL:
343 .PP
344 .Vb 5
345 \& Newx(buf, somesize+1, char);
346 \& /* ... fill in buf ... */
347 \& buf[somesize] = \*(Aq\e0\*(Aq;
348 \& sv_usepvn_flags(sv, buf, somesize, SV_SMAGIC | SV_HAS_TRAILING_NUL);
349 \& /* buf now belongs to perl, don\*(Aqt release it */
350 .Ve
351 .PP
352 If you have an SV and want to know what kind of data Perl thinks is stored
353 in it, you can use the following macros to check the type of SV you have.
354 .PP
355 .Vb 3
356 \& SvIOK(SV*)
357 \& SvNOK(SV*)
358 \& SvPOK(SV*)
359 .Ve
360 .PP
361 Be aware that retrieving the numeric value of an SV can set IOK or NOK
362 on that SV, even when the SV started as a string. Prior to Perl
363 5.36.0 retrieving the string value of an integer could set POK, but
364 this can no longer occur. From 5.36.0 this can be used to distinguish
365 the original representation of an SV and is intended to make life
366 simpler for serializers:
367 .PP
368 .Vb 10
369 \& /* references handled elsewhere */
370 \& if (SvIsBOOL(sv)) {
371 \& /* originally boolean */
372 \& ...
373 \& }
374 \& else if (SvPOK(sv)) {
375 \& /* originally a string */
376 \& ...
377 \& }
378 \& else if (SvNIOK(sv)) {
379 \& /* originally numeric */
380 \& ...
381 \& }
382 \& else {
383 \& /* something special or undef */
384 \& }
385 .Ve
386 .PP
387 You can get and set the current length of the string stored in an SV with
388 the following macros:
389 .PP
390 .Vb 2
391 \& SvCUR(SV*)
392 \& SvCUR_set(SV*, I32 val)
393 .Ve
394 .PP
395 You can also get a pointer to the end of the string stored in the SV
396 with the macro:
397 .PP
398 .Vb 1
399 \& SvEND(SV*)
400 .Ve
401 .PP
402 But note that these last three macros are valid only if \f(CWSvPOK()\fR is true.
403 .PP
404 If you want to append something to the end of string stored in an \f(CW\*(C`SV*\*(C'\fR,
405 you can use the following functions:
406 .PP
407 .Vb 6
408 \& void sv_catpv(SV*, const char*);
409 \& void sv_catpvn(SV*, const char*, STRLEN);
410 \& void sv_catpvf(SV*, const char*, ...);
411 \& void sv_vcatpvfn(SV*, const char*, STRLEN, va_list *, SV **,
412 \& I32, bool);
413 \& void sv_catsv(SV*, SV*);
414 .Ve
415 .PP
416 The first function calculates the length of the string to be appended by
417 using \f(CW\*(C`strlen\*(C'\fR. In the second, you specify the length of the string
418 yourself. The third function processes its arguments like \f(CW\*(C`sprintf\*(C'\fR and
419 appends the formatted output. The fourth function works like \f(CW\*(C`vsprintf\*(C'\fR.
420 You can specify the address and length of an array of SVs instead of the
421 va_list argument. The fifth function
422 extends the string stored in the first
423 SV with the string stored in the second SV. It also forces the second SV
424 to be interpreted as a string.
425 .PP
426 The \f(CW\*(C`sv_cat*()\*(C'\fR functions are not generic enough to operate on values that
427 have "magic". See "Magic Virtual Tables" later in this document.
428 .PP
429 If you know the name of a scalar variable, you can get a pointer to its SV
430 by using the following:
431 .PP
432 .Vb 1
433 \& SV* get_sv("package::varname", 0);
434 .Ve
435 .PP
436 This returns NULL if the variable does not exist.
437 .PP
438 If you want to know if this variable (or any other SV) is actually \f(CW\*(C`defined\*(C'\fR,
439 you can call:
440 .PP
441 .Vb 1
442 \& SvOK(SV*)
443 .Ve
444 .PP
445 The scalar \f(CW\*(C`undef\*(C'\fR value is stored in an SV instance called \f(CW\*(C`PL_sv_undef\*(C'\fR.
446 .PP
447 Its address can be used whenever an \f(CW\*(C`SV*\*(C'\fR is needed. Make sure that
448 you don't try to compare a random sv with \f(CW&PL_sv_undef\fR. For example
449 when interfacing Perl code, it'll work correctly for:
450 .PP
451 .Vb 1
452 \& foo(undef);
453 .Ve
454 .PP
455 But won't work when called as:
456 .PP
457 .Vb 2
458 \& $x = undef;
459 \& foo($x);
460 .Ve
461 .PP
462 So to repeat always use \fBSvOK()\fR to check whether an sv is defined.
463 .PP
464 Also you have to be careful when using \f(CW&PL_sv_undef\fR as a value in
465 AVs or HVs (see "AVs, HVs and undefined values").
466 .PP
467 There are also the two values \f(CW\*(C`PL_sv_yes\*(C'\fR and \f(CW\*(C`PL_sv_no\*(C'\fR, which contain
468 boolean TRUE and FALSE values, respectively. Like \f(CW\*(C`PL_sv_undef\*(C'\fR, their
469 addresses can be used whenever an \f(CW\*(C`SV*\*(C'\fR is needed.
470 .PP
471 Do not be fooled into thinking that \f(CW\*(C`(SV *) 0\*(C'\fR is the same as \f(CW&PL_sv_undef\fR.
472 Take this code:
473 .PP
474 .Vb 5
475 \& SV* sv = (SV*) 0;
476 \& if (I\-am\-to\-return\-a\-real\-value) {
477 \& sv = sv_2mortal(newSViv(42));
478 \& }
479 \& sv_setsv(ST(0), sv);
480 .Ve
481 .PP
482 This code tries to return a new SV (which contains the value 42) if it should
483 return a real value, or undef otherwise. Instead it has returned a NULL
484 pointer which, somewhere down the line, will cause a segmentation violation,
485 bus error, or just weird results. Change the zero to \f(CW&PL_sv_undef\fR in the
486 first line and all will be well.
487 .PP
488 To free an SV that you've created, call \f(CWSvREFCNT_dec(SV*)\fR. Normally this
489 call is not necessary (see "Reference Counts and Mortality").
490 .SS Offsets
491 .IX Subsection "Offsets"
492 Perl provides the function \f(CW\*(C`sv_chop\*(C'\fR to efficiently remove characters
493 from the beginning of a string; you give it an SV and a pointer to
494 somewhere inside the PV, and it discards everything before the
495 pointer. The efficiency comes by means of a little hack: instead of
496 actually removing the characters, \f(CW\*(C`sv_chop\*(C'\fR sets the flag \f(CW\*(C`OOK\*(C'\fR
497 (offset OK) to signal to other functions that the offset hack is in
498 effect, and it moves the PV pointer (called \f(CW\*(C`SvPVX\*(C'\fR) forward
499 by the number of bytes chopped off, and adjusts \f(CW\*(C`SvCUR\*(C'\fR and \f(CW\*(C`SvLEN\*(C'\fR
500 accordingly. (A portion of the space between the old and new PV
501 pointers is used to store the count of chopped bytes.)
502 .PP
503 Hence, at this point, the start of the buffer that we allocated lives
504 at \f(CW\*(C`SvPVX(sv) \- SvIV(sv)\*(C'\fR in memory and the PV pointer is pointing
505 into the middle of this allocated storage.
506 .PP
507 This is best demonstrated by example. Normally copy-on-write will prevent
508 the substitution from operator from using this hack, but if you can craft a
509 string for which copy-on-write is not possible, you can see it in play. In
510 the current implementation, the final byte of a string buffer is used as a
511 copy-on-write reference count. If the buffer is not big enough, then
512 copy-on-write is skipped. First have a look at an empty string:
513 .PP
514 .Vb 7
515 \& % ./perl \-Ilib \-MDevel::Peek \-le \*(Aq$a=""; $a .= ""; Dump $a\*(Aq
516 \& SV = PV(0x7ffb7c008a70) at 0x7ffb7c030390
517 \& REFCNT = 1
518 \& FLAGS = (POK,pPOK)
519 \& PV = 0x7ffb7bc05b50 ""\e0
520 \& CUR = 0
521 \& LEN = 10
522 .Ve
523 .PP
524 Notice here the LEN is 10. (It may differ on your platform.) Extend the
525 length of the string to one less than 10, and do a substitution:
526 .PP
527 .Vb 9
528 \& % ./perl \-Ilib \-MDevel::Peek \-le \*(Aq$a=""; $a.="123456789"; $a=~s/.//; \e
529 \& Dump($a)\*(Aq
530 \& SV = PV(0x7ffa04008a70) at 0x7ffa04030390
531 \& REFCNT = 1
532 \& FLAGS = (POK,OOK,pPOK)
533 \& OFFSET = 1
534 \& PV = 0x7ffa03c05b61 ( "\e1" . ) "23456789"\e0
535 \& CUR = 8
536 \& LEN = 9
537 .Ve
538 .PP
539 Here the number of bytes chopped off (1) is shown next as the OFFSET. The
540 portion of the string between the "real" and the "fake" beginnings is
541 shown in parentheses, and the values of \f(CW\*(C`SvCUR\*(C'\fR and \f(CW\*(C`SvLEN\*(C'\fR reflect
542 the fake beginning, not the real one. (The first character of the string
543 buffer happens to have changed to "\e1" here, not "1", because the current
544 implementation stores the offset count in the string buffer. This is
545 subject to change.)
546 .PP
547 Something similar to the offset hack is performed on AVs to enable
548 efficient shifting and splicing off the beginning of the array; while
549 \&\f(CW\*(C`AvARRAY\*(C'\fR points to the first element in the array that is visible from
550 Perl, \f(CW\*(C`AvALLOC\*(C'\fR points to the real start of the C array. These are
551 usually the same, but a \f(CW\*(C`shift\*(C'\fR operation can be carried out by
552 increasing \f(CW\*(C`AvARRAY\*(C'\fR by one and decreasing \f(CW\*(C`AvFILL\*(C'\fR and \f(CW\*(C`AvMAX\*(C'\fR.
553 Again, the location of the real start of the C array only comes into
554 play when freeing the array. See \f(CW\*(C`av_shift\*(C'\fR in \fIav.c\fR.
555 .SS "What's Really Stored in an SV?"
556 .IX Subsection "What's Really Stored in an SV?"
557 Recall that the usual method of determining the type of scalar you have is
558 to use \f(CW\*(C`Sv*OK\*(C'\fR macros. Because a scalar can be both a number and a string,
559 usually these macros will always return TRUE and calling the \f(CW\*(C`Sv*V\*(C'\fR
560 macros will do the appropriate conversion of string to integer/double or
561 integer/double to string.
562 .PP
563 If you \fIreally\fR need to know if you have an integer, double, or string
564 pointer in an SV, you can use the following three macros instead:
565 .PP
566 .Vb 3
567 \& SvIOKp(SV*)
568 \& SvNOKp(SV*)
569 \& SvPOKp(SV*)
570 .Ve
571 .PP
572 These will tell you if you truly have an integer, double, or string pointer
573 stored in your SV. The "p" stands for private.
574 .PP
575 There are various ways in which the private and public flags may differ.
576 For example, in perl 5.16 and earlier a tied SV may have a valid
577 underlying value in the IV slot (so SvIOKp is true), but the data
578 should be accessed via the FETCH routine rather than directly,
579 so SvIOK is false. (In perl 5.18 onwards, tied scalars use
580 the flags the same way as untied scalars.) Another is when
581 numeric conversion has occurred and precision has been lost: only the
582 private flag is set on 'lossy' values. So when an NV is converted to an
583 IV with loss, SvIOKp, SvNOKp and SvNOK will be set, while SvIOK wont be.
584 .PP
585 In general, though, it's best to use the \f(CW\*(C`Sv*V\*(C'\fR macros.
586 .SS "Working with AVs"
587 .IX Subsection "Working with AVs"
588 There are two main, longstanding ways to create and load an AV. The first
589 method creates an empty AV:
590 .PP
591 .Vb 1
592 \& AV* newAV();
593 .Ve
594 .PP
595 The second method both creates the AV and initially populates it with SVs:
596 .PP
597 .Vb 1
598 \& AV* av_make(SSize_t num, SV **ptr);
599 .Ve
600 .PP
601 The second argument points to an array containing \f(CW\*(C`num\*(C'\fR \f(CW\*(C`SV*\*(C'\fR's. Once the
602 AV has been created, the SVs can be destroyed, if so desired.
603 .PP
604 Perl v5.36 added two new ways to create an AV and allocate a SV** array
605 without populating it. These are more efficient than a \fBnewAV()\fR followed by an
606 \&\fBav_extend()\fR.
607 .PP
608 .Vb 4
609 \& /* Creates but does not initialize (Zero) the SV** array */
610 \& AV *av = newAV_alloc_x(1);
611 \& /* Creates and does initialize (Zero) the SV** array */
612 \& AV *av = newAV_alloc_xz(1);
613 .Ve
614 .PP
615 The numerical argument refers to the number of array elements to allocate, not
616 an array index, and must be >0. The first form must only ever be used when all
617 elements will be initialized before any read occurs. Reading a non-initialized
618 SV* \- i.e. treating a random memory address as a SV* \- is a serious bug.
619 .PP
620 Once the AV has been created, the following operations are possible on it:
621 .PP
622 .Vb 4
623 \& void av_push(AV*, SV*);
624 \& SV* av_pop(AV*);
625 \& SV* av_shift(AV*);
626 \& void av_unshift(AV*, SSize_t num);
627 .Ve
628 .PP
629 These should be familiar operations, with the exception of \f(CW\*(C`av_unshift\*(C'\fR.
630 This routine adds \f(CW\*(C`num\*(C'\fR elements at the front of the array with the \f(CW\*(C`undef\*(C'\fR
631 value. You must then use \f(CW\*(C`av_store\*(C'\fR (described below) to assign values
632 to these new elements.
633 .PP
634 Here are some other functions:
635 .PP
636 .Vb 4
637 \& Size_t av_count(AV*);
638 \& SSize_t av_top_index(AV*);
639 \& SV** av_fetch(AV*, SSize_t key, I32 lval);
640 \& SV** av_store(AV*, SSize_t key, SV* val);
641 .Ve
642 .PP
643 \&\f(CW\*(C`av_count\*(C'\fR returns the number of elements in the array (including
644 any empty slots (undefined ones) that are intermixed with filled-in ones).
645 The \f(CW\*(C`av_top_index\*(C'\fR function returns the highest index value in an array (just
646 like $#array in Perl). If the array is empty, \-1 is returned. It is
647 always equal to \f(CW\*(C`av_count()\ \-\ 1\*(C'\fR. The
648 \&\f(CW\*(C`av_fetch\*(C'\fR function returns the value at index \f(CW\*(C`key\*(C'\fR, but if \f(CW\*(C`lval\*(C'\fR
649 is non-zero, then \f(CW\*(C`av_fetch\*(C'\fR will store an undef value at that index.
650 The \f(CW\*(C`av_store\*(C'\fR function stores the value \f(CW\*(C`val\*(C'\fR at index \f(CW\*(C`key\*(C'\fR, and does
651 not increment the reference count of \f(CW\*(C`val\*(C'\fR. Thus the caller is responsible
652 for taking care of that, and if \f(CW\*(C`av_store\*(C'\fR returns NULL, the caller will
653 have to decrement the reference count to avoid a memory leak. Note that
654 \&\f(CW\*(C`av_fetch\*(C'\fR and \f(CW\*(C`av_store\*(C'\fR both return \f(CW\*(C`SV**\*(C'\fR's, not \f(CW\*(C`SV*\*(C'\fR's as their
655 return value.
656 .PP
657 A few more:
658 .PP
659 .Vb 3
660 \& void av_clear(AV*);
661 \& void av_undef(AV*);
662 \& void av_extend(AV*, SSize_t key);
663 .Ve
664 .PP
665 The \f(CW\*(C`av_clear\*(C'\fR function deletes all the elements in the AV* array, but
666 does not actually delete the array itself. The \f(CW\*(C`av_undef\*(C'\fR function will
667 delete all the elements in the array plus the array itself. The
668 \&\f(CW\*(C`av_extend\*(C'\fR function extends the array so that it contains at least \f(CW\*(C`key+1\*(C'\fR
669 elements. If \f(CW\*(C`key+1\*(C'\fR is less than the currently allocated length of the array,
670 then nothing is done.
671 .PP
672 If you know the name of an array variable, you can get a pointer to its AV
673 by using the following:
674 .PP
675 .Vb 1
676 \& AV* get_av("package::varname", 0);
677 .Ve
678 .PP
679 This returns NULL if the variable does not exist.
680 .PP
681 See "Understanding the Magic of Tied Hashes and Arrays" for more
682 information on how to use the array access functions on tied arrays.
683 .PP
684 \fIMore efficient working with new or vanilla AVs\fR
685 .IX Subsection "More efficient working with new or vanilla AVs"
686 .PP
687 Perl v5.36 and v5.38 introduced streamlined, inlined versions of some
688 functions:
689 .IP \(bu 4
690 \&\f(CW\*(C`av_store_simple\*(C'\fR
691 .IP \(bu 4
692 \&\f(CW\*(C`av_fetch_simple\*(C'\fR
693 .IP \(bu 4
694 \&\f(CW\*(C`av_push_simple\*(C'\fR
695 .PP
696 These are drop-in replacements, but can only be used on straightforward
697 AVs that meet the following criteria:
698 .IP \(bu 4
699 are not magical
700 .IP \(bu 4
701 are not readonly
702 .IP \(bu 4
703 are "real" (refcounted) AVs
704 .IP \(bu 4
705 have an av_top_index value > \-2
706 .PP
707 AVs created using \f(CWnewAV()\fR, \f(CW\*(C`av_make\*(C'\fR, \f(CW\*(C`newAV_alloc_x\*(C'\fR, and
708 \&\f(CW\*(C`newAV_alloc_xz\*(C'\fR are all compatible at the time of creation. It is
709 only if they are declared readonly or unreal, have magic attached, or
710 are otherwise configured unusually that they will stop being compatible.
711 .PP
712 Note that some interpreter functions may attach magic to an AV as part
713 of normal operations. It is therefore safest, unless you are sure of the
714 lifecycle of an AV, to only use these new functions close to the point
715 of AV creation.
716 .SS "Working with HVs"
717 .IX Subsection "Working with HVs"
718 To create an HV, you use the following routine:
719 .PP
720 .Vb 1
721 \& HV* newHV();
722 .Ve
723 .PP
724 Once the HV has been created, the following operations are possible on it:
725 .PP
726 .Vb 2
727 \& SV** hv_store(HV*, const char* key, U32 klen, SV* val, U32 hash);
728 \& SV** hv_fetch(HV*, const char* key, U32 klen, I32 lval);
729 .Ve
730 .PP
731 The \f(CW\*(C`klen\*(C'\fR parameter is the length of the key being passed in (Note that
732 you cannot pass 0 in as a value of \f(CW\*(C`klen\*(C'\fR to tell Perl to measure the
733 length of the key). The \f(CW\*(C`val\*(C'\fR argument contains the SV pointer to the
734 scalar being stored, and \f(CW\*(C`hash\*(C'\fR is the precomputed hash value (zero if
735 you want \f(CW\*(C`hv_store\*(C'\fR to calculate it for you). The \f(CW\*(C`lval\*(C'\fR parameter
736 indicates whether this fetch is actually a part of a store operation, in
737 which case a new undefined value will be added to the HV with the supplied
738 key and \f(CW\*(C`hv_fetch\*(C'\fR will return as if the value had already existed.
739 .PP
740 Remember that \f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_fetch\*(C'\fR return \f(CW\*(C`SV**\*(C'\fR's and not just
741 \&\f(CW\*(C`SV*\*(C'\fR. To access the scalar value, you must first dereference the return
742 value. However, you should check to make sure that the return value is
743 not NULL before dereferencing it.
744 .PP
745 The first of these two functions checks if a hash table entry exists, and the
746 second deletes it.
747 .PP
748 .Vb 2
749 \& bool hv_exists(HV*, const char* key, U32 klen);
750 \& SV* hv_delete(HV*, const char* key, U32 klen, I32 flags);
751 .Ve
752 .PP
753 If \f(CW\*(C`flags\*(C'\fR does not include the \f(CW\*(C`G_DISCARD\*(C'\fR flag then \f(CW\*(C`hv_delete\*(C'\fR will
754 create and return a mortal copy of the deleted value.
755 .PP
756 And more miscellaneous functions:
757 .PP
758 .Vb 2
759 \& void hv_clear(HV*);
760 \& void hv_undef(HV*);
761 .Ve
762 .PP
763 Like their AV counterparts, \f(CW\*(C`hv_clear\*(C'\fR deletes all the entries in the hash
764 table but does not actually delete the hash table. The \f(CW\*(C`hv_undef\*(C'\fR deletes
765 both the entries and the hash table itself.
766 .PP
767 Perl keeps the actual data in a linked list of structures with a typedef of HE.
768 These contain the actual key and value pointers (plus extra administrative
769 overhead). The key is a string pointer; the value is an \f(CW\*(C`SV*\*(C'\fR. However,
770 once you have an \f(CW\*(C`HE*\*(C'\fR, to get the actual key and value, use the routines
771 specified below.
772 .PP
773 .Vb 10
774 \& I32 hv_iterinit(HV*);
775 \& /* Prepares starting point to traverse hash table */
776 \& HE* hv_iternext(HV*);
777 \& /* Get the next entry, and return a pointer to a
778 \& structure that has both the key and value */
779 \& char* hv_iterkey(HE* entry, I32* retlen);
780 \& /* Get the key from an HE structure and also return
781 \& the length of the key string */
782 \& SV* hv_iterval(HV*, HE* entry);
783 \& /* Return an SV pointer to the value of the HE
784 \& structure */
785 \& SV* hv_iternextsv(HV*, char** key, I32* retlen);
786 \& /* This convenience routine combines hv_iternext,
787 \& hv_iterkey, and hv_iterval. The key and retlen
788 \& arguments are return values for the key and its
789 \& length. The value is returned in the SV* argument */
790 .Ve
791 .PP
792 If you know the name of a hash variable, you can get a pointer to its HV
793 by using the following:
794 .PP
795 .Vb 1
796 \& HV* get_hv("package::varname", 0);
797 .Ve
798 .PP
799 This returns NULL if the variable does not exist.
800 .PP
801 The hash algorithm is defined in the \f(CW\*(C`PERL_HASH\*(C'\fR macro:
802 .PP
803 .Vb 1
804 \& PERL_HASH(hash, key, klen)
805 .Ve
806 .PP
807 The exact implementation of this macro varies by architecture and version
808 of perl, and the return value may change per invocation, so the value
809 is only valid for the duration of a single perl process.
810 .PP
811 See "Understanding the Magic of Tied Hashes and Arrays" for more
812 information on how to use the hash access functions on tied hashes.
813 .SS "Hash API Extensions"
814 .IX Subsection "Hash API Extensions"
815 Beginning with version 5.004, the following functions are also supported:
816 .PP
817 .Vb 2
818 \& HE* hv_fetch_ent (HV* tb, SV* key, I32 lval, U32 hash);
819 \& HE* hv_store_ent (HV* tb, SV* key, SV* val, U32 hash);
820 \&
821 \& bool hv_exists_ent (HV* tb, SV* key, U32 hash);
822 \& SV* hv_delete_ent (HV* tb, SV* key, I32 flags, U32 hash);
823 \&
824 \& SV* hv_iterkeysv (HE* entry);
825 .Ve
826 .PP
827 Note that these functions take \f(CW\*(C`SV*\*(C'\fR keys, which simplifies writing
828 of extension code that deals with hash structures. These functions
829 also allow passing of \f(CW\*(C`SV*\*(C'\fR keys to \f(CW\*(C`tie\*(C'\fR functions without forcing
830 you to stringify the keys (unlike the previous set of functions).
831 .PP
832 They also return and accept whole hash entries (\f(CW\*(C`HE*\*(C'\fR), making their
833 use more efficient (since the hash number for a particular string
834 doesn't have to be recomputed every time). See perlapi for detailed
835 descriptions.
836 .PP
837 The following macros must always be used to access the contents of hash
838 entries. Note that the arguments to these macros must be simple
839 variables, since they may get evaluated more than once. See
840 perlapi for detailed descriptions of these macros.
841 .PP
842 .Vb 6
843 \& HePV(HE* he, STRLEN len)
844 \& HeVAL(HE* he)
845 \& HeHASH(HE* he)
846 \& HeSVKEY(HE* he)
847 \& HeSVKEY_force(HE* he)
848 \& HeSVKEY_set(HE* he, SV* sv)
849 .Ve
850 .PP
851 These two lower level macros are defined, but must only be used when
852 dealing with keys that are not \f(CW\*(C`SV*\*(C'\fRs:
853 .PP
854 .Vb 2
855 \& HeKEY(HE* he)
856 \& HeKLEN(HE* he)
857 .Ve
858 .PP
859 Note that both \f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_store_ent\*(C'\fR do not increment the
860 reference count of the stored \f(CW\*(C`val\*(C'\fR, which is the caller's responsibility.
861 If these functions return a NULL value, the caller will usually have to
862 decrement the reference count of \f(CW\*(C`val\*(C'\fR to avoid a memory leak.
863 .SS "AVs, HVs and undefined values"
864 .IX Subsection "AVs, HVs and undefined values"
865 Sometimes you have to store undefined values in AVs or HVs. Although
866 this may be a rare case, it can be tricky. That's because you're
867 used to using \f(CW&PL_sv_undef\fR if you need an undefined SV.
868 .PP
869 For example, intuition tells you that this XS code:
870 .PP
871 .Vb 2
872 \& AV *av = newAV();
873 \& av_store( av, 0, &PL_sv_undef );
874 .Ve
875 .PP
876 is equivalent to this Perl code:
877 .PP
878 .Vb 2
879 \& my @av;
880 \& $av[0] = undef;
881 .Ve
882 .PP
883 Unfortunately, this isn't true. In perl 5.18 and earlier, AVs use \f(CW&PL_sv_undef\fR as a marker
884 for indicating that an array element has not yet been initialized.
885 Thus, \f(CW\*(C`exists $av[0]\*(C'\fR would be true for the above Perl code, but
886 false for the array generated by the XS code. In perl 5.20, storing
887 &PL_sv_undef will create a read-only element, because the scalar
888 &PL_sv_undef itself is stored, not a copy.
889 .PP
890 Similar problems can occur when storing \f(CW&PL_sv_undef\fR in HVs:
891 .PP
892 .Vb 1
893 \& hv_store( hv, "key", 3, &PL_sv_undef, 0 );
894 .Ve
895 .PP
896 This will indeed make the value \f(CW\*(C`undef\*(C'\fR, but if you try to modify
897 the value of \f(CW\*(C`key\*(C'\fR, you'll get the following error:
898 .PP
899 .Vb 1
900 \& Modification of non\-creatable hash value attempted
901 .Ve
902 .PP
903 In perl 5.8.0, \f(CW&PL_sv_undef\fR was also used to mark placeholders
904 in restricted hashes. This caused such hash entries not to appear
905 when iterating over the hash or when checking for the keys
906 with the \f(CW\*(C`hv_exists\*(C'\fR function.
907 .PP
908 You can run into similar problems when you store \f(CW&PL_sv_yes\fR or
909 \&\f(CW&PL_sv_no\fR into AVs or HVs. Trying to modify such elements
910 will give you the following error:
911 .PP
912 .Vb 1
913 \& Modification of a read\-only value attempted
914 .Ve
915 .PP
916 To make a long story short, you can use the special variables
917 \&\f(CW&PL_sv_undef\fR, \f(CW&PL_sv_yes\fR and \f(CW&PL_sv_no\fR with AVs and
918 HVs, but you have to make sure you know what you're doing.
919 .PP
920 Generally, if you want to store an undefined value in an AV
921 or HV, you should not use \f(CW&PL_sv_undef\fR, but rather create a
922 new undefined value using the \f(CW\*(C`newSV\*(C'\fR function, for example:
923 .PP
924 .Vb 2
925 \& av_store( av, 42, newSV(0) );
926 \& hv_store( hv, "foo", 3, newSV(0), 0 );
927 .Ve
928 .SS References
929 .IX Subsection "References"
930 References are a special type of scalar that point to other data types
931 (including other references).
932 .PP
933 To create a reference, use either of the following functions:
934 .PP
935 .Vb 2
936 \& SV* newRV_inc((SV*) thing);
937 \& SV* newRV_noinc((SV*) thing);
938 .Ve
939 .PP
940 The \f(CW\*(C`thing\*(C'\fR argument can be any of an \f(CW\*(C`SV*\*(C'\fR, \f(CW\*(C`AV*\*(C'\fR, or \f(CW\*(C`HV*\*(C'\fR. The
941 functions are identical except that \f(CW\*(C`newRV_inc\*(C'\fR increments the reference
942 count of the \f(CW\*(C`thing\*(C'\fR, while \f(CW\*(C`newRV_noinc\*(C'\fR does not. For historical
943 reasons, \f(CW\*(C`newRV\*(C'\fR is a synonym for \f(CW\*(C`newRV_inc\*(C'\fR.
944 .PP
945 Once you have a reference, you can use the following macro to dereference
946 the reference:
947 .PP
948 .Vb 1
949 \& SvRV(SV*)
950 .Ve
951 .PP
952 then call the appropriate routines, casting the returned \f(CW\*(C`SV*\*(C'\fR to either an
953 \&\f(CW\*(C`AV*\*(C'\fR or \f(CW\*(C`HV*\*(C'\fR, if required.
954 .PP
955 To determine if an SV is a reference, you can use the following macro:
956 .PP
957 .Vb 1
958 \& SvROK(SV*)
959 .Ve
960 .PP
961 To discover what type of value the reference refers to, use the following
962 macro and then check the return value.
963 .PP
964 .Vb 1
965 \& SvTYPE(SvRV(SV*))
966 .Ve
967 .PP
968 The most useful types that will be returned are:
969 .PP
970 .Vb 4
971 \& SVt_PVAV Array
972 \& SVt_PVHV Hash
973 \& SVt_PVCV Code
974 \& SVt_PVGV Glob (possibly a file handle)
975 .Ve
976 .PP
977 Any numerical value returned which is less than SVt_PVAV will be a scalar
978 of some form.
979 .PP
980 See "svtype" in perlapi for more details.
981 .SS "Blessed References and Class Objects"
982 .IX Subsection "Blessed References and Class Objects"
983 References are also used to support object-oriented programming. In perl's
984 OO lexicon, an object is simply a reference that has been blessed into a
985 package (or class). Once blessed, the programmer may now use the reference
986 to access the various methods in the class.
987 .PP
988 A reference can be blessed into a package with the following function:
989 .PP
990 .Vb 1
991 \& SV* sv_bless(SV* sv, HV* stash);
992 .Ve
993 .PP
994 The \f(CW\*(C`sv\*(C'\fR argument must be a reference value. The \f(CW\*(C`stash\*(C'\fR argument
995 specifies which class the reference will belong to. See
996 "Stashes and Globs" for information on converting class names into stashes.
997 .PP
998 /* Still under construction */
999 .PP
1000 The following function upgrades rv to reference if not already one.
1001 Creates a new SV for rv to point to. If \f(CW\*(C`classname\*(C'\fR is non-null, the SV
1002 is blessed into the specified class. SV is returned.
1003 .PP
1004 .Vb 1
1005 \& SV* newSVrv(SV* rv, const char* classname);
1006 .Ve
1007 .PP
1008 The following three functions copy integer, unsigned integer or double
1009 into an SV whose reference is \f(CW\*(C`rv\*(C'\fR. SV is blessed if \f(CW\*(C`classname\*(C'\fR is
1010 non-null.
1011 .PP
1012 .Vb 3
1013 \& SV* sv_setref_iv(SV* rv, const char* classname, IV iv);
1014 \& SV* sv_setref_uv(SV* rv, const char* classname, UV uv);
1015 \& SV* sv_setref_nv(SV* rv, const char* classname, NV iv);
1016 .Ve
1017 .PP
1018 The following function copies the pointer value (\fIthe address, not the
1019 string!\fR) into an SV whose reference is rv. SV is blessed if \f(CW\*(C`classname\*(C'\fR
1020 is non-null.
1021 .PP
1022 .Vb 1
1023 \& SV* sv_setref_pv(SV* rv, const char* classname, void* pv);
1024 .Ve
1025 .PP
1026 The following function copies a string into an SV whose reference is \f(CW\*(C`rv\*(C'\fR.
1027 Set length to 0 to let Perl calculate the string length. SV is blessed if
1028 \&\f(CW\*(C`classname\*(C'\fR is non-null.
1029 .PP
1030 .Vb 2
1031 \& SV* sv_setref_pvn(SV* rv, const char* classname, char* pv,
1032 \& STRLEN length);
1033 .Ve
1034 .PP
1035 The following function tests whether the SV is blessed into the specified
1036 class. It does not check inheritance relationships.
1037 .PP
1038 .Vb 1
1039 \& int sv_isa(SV* sv, const char* name);
1040 .Ve
1041 .PP
1042 The following function tests whether the SV is a reference to a blessed object.
1043 .PP
1044 .Vb 1
1045 \& int sv_isobject(SV* sv);
1046 .Ve
1047 .PP
1048 The following function tests whether the SV is derived from the specified
1049 class. SV can be either a reference to a blessed object or a string
1050 containing a class name. This is the function implementing the
1051 \&\f(CW\*(C`UNIVERSAL::isa\*(C'\fR functionality.
1052 .PP
1053 .Vb 1
1054 \& bool sv_derived_from(SV* sv, const char* name);
1055 .Ve
1056 .PP
1057 To check if you've got an object derived from a specific class you have
1058 to write:
1059 .PP
1060 .Vb 1
1061 \& if (sv_isobject(sv) && sv_derived_from(sv, class)) { ... }
1062 .Ve
1063 .SS "Creating New Variables"
1064 .IX Subsection "Creating New Variables"
1065 To create a new Perl variable with an undef value which can be accessed from
1066 your Perl script, use the following routines, depending on the variable type.
1067 .PP
1068 .Vb 3
1069 \& SV* get_sv("package::varname", GV_ADD);
1070 \& AV* get_av("package::varname", GV_ADD);
1071 \& HV* get_hv("package::varname", GV_ADD);
1072 .Ve
1073 .PP
1074 Notice the use of GV_ADD as the second parameter. The new variable can now
1075 be set, using the routines appropriate to the data type.
1076 .PP
1077 There are additional macros whose values may be bitwise OR'ed with the
1078 \&\f(CW\*(C`GV_ADD\*(C'\fR argument to enable certain extra features. Those bits are:
1079 .IP GV_ADDMULTI 4
1080 .IX Item "GV_ADDMULTI"
1081 Marks the variable as multiply defined, thus preventing the:
1082 .Sp
1083 .Vb 1
1084 \& Name <varname> used only once: possible typo
1085 .Ve
1086 .Sp
1087 warning.
1088 .IP GV_ADDWARN 4
1089 .IX Item "GV_ADDWARN"
1090 Issues the warning:
1091 .Sp
1092 .Vb 1
1093 \& Had to create <varname> unexpectedly
1094 .Ve
1095 .Sp
1096 if the variable did not exist before the function was called.
1097 .PP
1098 If you do not specify a package name, the variable is created in the current
1099 package.
1100 .SS "Reference Counts and Mortality"
1101 .IX Subsection "Reference Counts and Mortality"
1102 Perl uses a reference count-driven garbage collection mechanism. SVs,
1103 AVs, or HVs (xV for short in the following) start their life with a
1104 reference count of 1. If the reference count of an xV ever drops to 0,
1105 then it will be destroyed and its memory made available for reuse.
1106 At the most basic internal level, reference counts can be manipulated
1107 with the following macros:
1108 .PP
1109 .Vb 3
1110 \& int SvREFCNT(SV* sv);
1111 \& SV* SvREFCNT_inc(SV* sv);
1112 \& void SvREFCNT_dec(SV* sv);
1113 .Ve
1114 .PP
1115 (There are also suffixed versions of the increment and decrement macros,
1116 for situations where the full generality of these basic macros can be
1117 exchanged for some performance.)
1118 .PP
1119 However, the way a programmer should think about references is not so
1120 much in terms of the bare reference count, but in terms of \fIownership\fR
1121 of references. A reference to an xV can be owned by any of a variety
1122 of entities: another xV, the Perl interpreter, an XS data structure,
1123 a piece of running code, or a dynamic scope. An xV generally does not
1124 know what entities own the references to it; it only knows how many
1125 references there are, which is the reference count.
1126 .PP
1127 To correctly maintain reference counts, it is essential to keep track
1128 of what references the XS code is manipulating. The programmer should
1129 always know where a reference has come from and who owns it, and be
1130 aware of any creation or destruction of references, and any transfers
1131 of ownership. Because ownership isn't represented explicitly in the xV
1132 data structures, only the reference count need be actually maintained
1133 by the code, and that means that this understanding of ownership is not
1134 actually evident in the code. For example, transferring ownership of a
1135 reference from one owner to another doesn't change the reference count
1136 at all, so may be achieved with no actual code. (The transferring code
1137 doesn't touch the referenced object, but does need to ensure that the
1138 former owner knows that it no longer owns the reference, and that the
1139 new owner knows that it now does.)
1140 .PP
1141 An xV that is visible at the Perl level should not become unreferenced
1142 and thus be destroyed. Normally, an object will only become unreferenced
1143 when it is no longer visible, often by the same means that makes it
1144 invisible. For example, a Perl reference value (RV) owns a reference to
1145 its referent, so if the RV is overwritten that reference gets destroyed,
1146 and the no-longer-reachable referent may be destroyed as a result.
1147 .PP
1148 Many functions have some kind of reference manipulation as
1149 part of their purpose. Sometimes this is documented in terms
1150 of ownership of references, and sometimes it is (less helpfully)
1151 documented in terms of changes to reference counts. For example, the
1152 \&\fBnewRV_inc()\fR function is documented to create a new RV
1153 (with reference count 1) and increment the reference count of the referent
1154 that was supplied by the caller. This is best understood as creating
1155 a new reference to the referent, which is owned by the created RV,
1156 and returning to the caller ownership of the sole reference to the RV.
1157 The \fBnewRV_noinc()\fR function instead does not
1158 increment the reference count of the referent, but the RV nevertheless
1159 ends up owning a reference to the referent. It is therefore implied
1160 that the caller of \f(CWnewRV_noinc()\fR is relinquishing a reference to the
1161 referent, making this conceptually a more complicated operation even
1162 though it does less to the data structures.
1163 .PP
1164 For example, imagine you want to return a reference from an XSUB
1165 function. Inside the XSUB routine, you create an SV which initially
1166 has just a single reference, owned by the XSUB routine. This reference
1167 needs to be disposed of before the routine is complete, otherwise it
1168 will leak, preventing the SV from ever being destroyed. So to create
1169 an RV referencing the SV, it is most convenient to pass the SV to
1170 \&\f(CWnewRV_noinc()\fR, which consumes that reference. Now the XSUB routine
1171 no longer owns a reference to the SV, but does own a reference to the RV,
1172 which in turn owns a reference to the SV. The ownership of the reference
1173 to the RV is then transferred by the process of returning the RV from
1174 the XSUB.
1175 .PP
1176 There are some convenience functions available that can help with the
1177 destruction of xVs. These functions introduce the concept of "mortality".
1178 Much documentation speaks of an xV itself being mortal, but this is
1179 misleading. It is really \fIa reference to\fR an xV that is mortal, and it
1180 is possible for there to be more than one mortal reference to a single xV.
1181 For a reference to be mortal means that it is owned by the temps stack,
1182 one of perl's many internal stacks, which will destroy that reference
1183 "a short time later". Usually the "short time later" is the end of
1184 the current Perl statement. However, it gets more complicated around
1185 dynamic scopes: there can be multiple sets of mortal references hanging
1186 around at the same time, with different death dates. Internally, the
1187 actual determinant for when mortal xV references are destroyed depends
1188 on two macros, SAVETMPS and FREETMPS. See perlcall and perlxs
1189 and "Temporaries Stack" below for more details on these macros.
1190 .PP
1191 Mortal references are mainly used for xVs that are placed on perl's
1192 main stack. The stack is problematic for reference tracking, because it
1193 contains a lot of xV references, but doesn't own those references: they
1194 are not counted. Currently, there are many bugs resulting from xVs being
1195 destroyed while referenced by the stack, because the stack's uncounted
1196 references aren't enough to keep the xVs alive. So when putting an
1197 (uncounted) reference on the stack, it is vitally important to ensure that
1198 there will be a counted reference to the same xV that will last at least
1199 as long as the uncounted reference. But it's also important that that
1200 counted reference be cleaned up at an appropriate time, and not unduly
1201 prolong the xV's life. For there to be a mortal reference is often the
1202 best way to satisfy this requirement, especially if the xV was created
1203 especially to be put on the stack and would otherwise be unreferenced.
1204 .PP
1205 To create a mortal reference, use the functions:
1206 .PP
1207 .Vb 3
1208 \& SV* sv_newmortal()
1209 \& SV* sv_mortalcopy(SV*)
1210 \& SV* sv_2mortal(SV*)
1211 .Ve
1212 .PP
1213 \&\f(CWsv_newmortal()\fR creates an SV (with the undefined value) whose sole
1214 reference is mortal. \f(CWsv_mortalcopy()\fR creates an xV whose value is a
1215 copy of a supplied xV and whose sole reference is mortal. \f(CWsv_2mortal()\fR
1216 mortalises an existing xV reference: it transfers ownership of a reference
1217 from the caller to the temps stack. Because \f(CW\*(C`sv_newmortal\*(C'\fR gives the new
1218 SV no value, it must normally be given one via \f(CW\*(C`sv_setpv\*(C'\fR, \f(CW\*(C`sv_setiv\*(C'\fR,
1219 etc. :
1220 .PP
1221 .Vb 2
1222 \& SV *tmp = sv_newmortal();
1223 \& sv_setiv(tmp, an_integer);
1224 .Ve
1225 .PP
1226 As that is multiple C statements it is quite common so see this idiom instead:
1227 .PP
1228 .Vb 1
1229 \& SV *tmp = sv_2mortal(newSViv(an_integer));
1230 .Ve
1231 .PP
1232 The mortal routines are not just for SVs; AVs and HVs can be
1233 made mortal by passing their address (type-casted to \f(CW\*(C`SV*\*(C'\fR) to the
1234 \&\f(CW\*(C`sv_2mortal\*(C'\fR or \f(CW\*(C`sv_mortalcopy\*(C'\fR routines.
1235 .SS "Stashes and Globs"
1236 .IX Subsection "Stashes and Globs"
1237 A \fBstash\fR is a hash that contains all variables that are defined
1238 within a package. Each key of the stash is a symbol
1239 name (shared by all the different types of objects that have the same
1240 name), and each value in the hash table is a GV (Glob Value). This GV
1241 in turn contains references to the various objects of that name,
1242 including (but not limited to) the following:
1243 .PP
1244 .Vb 6
1245 \& Scalar Value
1246 \& Array Value
1247 \& Hash Value
1248 \& I/O Handle
1249 \& Format
1250 \& Subroutine
1251 .Ve
1252 .PP
1253 There is a single stash called \f(CW\*(C`PL_defstash\*(C'\fR that holds the items that exist
1254 in the \f(CW\*(C`main\*(C'\fR package. To get at the items in other packages, append the
1255 string "::" to the package name. The items in the \f(CW\*(C`Foo\*(C'\fR package are in
1256 the stash \f(CW\*(C`Foo::\*(C'\fR in PL_defstash. The items in the \f(CW\*(C`Bar::Baz\*(C'\fR package are
1257 in the stash \f(CW\*(C`Baz::\*(C'\fR in \f(CW\*(C`Bar::\*(C'\fR's stash.
1258 .PP
1259 To get the stash pointer for a particular package, use the function:
1260 .PP
1261 .Vb 2
1262 \& HV* gv_stashpv(const char* name, I32 flags)
1263 \& HV* gv_stashsv(SV*, I32 flags)
1264 .Ve
1265 .PP
1266 The first function takes a literal string, the second uses the string stored
1267 in the SV. Remember that a stash is just a hash table, so you get back an
1268 \&\f(CW\*(C`HV*\*(C'\fR. The \f(CW\*(C`flags\*(C'\fR flag will create a new package if it is set to GV_ADD.
1269 .PP
1270 The name that \f(CW\*(C`gv_stash*v\*(C'\fR wants is the name of the package whose symbol table
1271 you want. The default package is called \f(CW\*(C`main\*(C'\fR. If you have multiply nested
1272 packages, pass their names to \f(CW\*(C`gv_stash*v\*(C'\fR, separated by \f(CW\*(C`::\*(C'\fR as in the Perl
1273 language itself.
1274 .PP
1275 Alternately, if you have an SV that is a blessed reference, you can find
1276 out the stash pointer by using:
1277 .PP
1278 .Vb 1
1279 \& HV* SvSTASH(SvRV(SV*));
1280 .Ve
1281 .PP
1282 then use the following to get the package name itself:
1283 .PP
1284 .Vb 1
1285 \& char* HvNAME(HV* stash);
1286 .Ve
1287 .PP
1288 If you need to bless or re-bless an object you can use the following
1289 function:
1290 .PP
1291 .Vb 1
1292 \& SV* sv_bless(SV*, HV* stash)
1293 .Ve
1294 .PP
1295 where the first argument, an \f(CW\*(C`SV*\*(C'\fR, must be a reference, and the second
1296 argument is a stash. The returned \f(CW\*(C`SV*\*(C'\fR can now be used in the same way
1297 as any other SV.
1298 .PP
1299 For more information on references and blessings, consult perlref.
1300 .SS "I/O Handles"
1301 .IX Subsection "I/O Handles"
1302 Like AVs and HVs, IO objects are another type of non-scalar SV which
1303 may contain input and output PerlIO objects or a \f(CW\*(C`DIR *\*(C'\fR
1304 from \fBopendir()\fR.
1305 .PP
1306 You can create a new IO object:
1307 .PP
1308 .Vb 1
1309 \& IO* newIO();
1310 .Ve
1311 .PP
1312 Unlike other SVs, a new IO object is automatically blessed into the
1313 IO::File class.
1314 .PP
1315 The IO object contains an input and output PerlIO handle:
1316 .PP
1317 .Vb 2
1318 \& PerlIO *IoIFP(IO *io);
1319 \& PerlIO *IoOFP(IO *io);
1320 .Ve
1321 .PP
1322 Typically if the IO object has been opened on a file, the input handle
1323 is always present, but the output handle is only present if the file
1324 is open for output. For a file, if both are present they will be the
1325 same PerlIO object.
1326 .PP
1327 Distinct input and output PerlIO objects are created for sockets and
1328 character devices.
1329 .PP
1330 The IO object also contains other data associated with Perl I/O
1331 handles:
1332 .PP
1333 .Vb 12
1334 \& IV IoLINES(io); /* $. */
1335 \& IV IoPAGE(io); /* $% */
1336 \& IV IoPAGE_LEN(io); /* $= */
1337 \& IV IoLINES_LEFT(io); /* $\- */
1338 \& char *IoTOP_NAME(io); /* $^ */
1339 \& GV *IoTOP_GV(io); /* $^ */
1340 \& char *IoFMT_NAME(io); /* $~ */
1341 \& GV *IoFMT_GV(io); /* $~ */
1342 \& char *IoBOTTOM_NAME(io);
1343 \& GV *IoBOTTOM_GV(io);
1344 \& char IoTYPE(io);
1345 \& U8 IoFLAGS(io);
1346 \&
1347 \& =for apidoc_sections $io_scn, $formats_section
1348 \&=for apidoc_section $reports
1349 \&=for apidoc Amh|IV|IoLINES|IO *io
1350 \&=for apidoc Amh|IV|IoPAGE|IO *io
1351 \&=for apidoc Amh|IV|IoPAGE_LEN|IO *io
1352 \&=for apidoc Amh|IV|IoLINES_LEFT|IO *io
1353 \&=for apidoc Amh|char *|IoTOP_NAME|IO *io
1354 \&=for apidoc Amh|GV *|IoTOP_GV|IO *io
1355 \&=for apidoc Amh|char *|IoFMT_NAME|IO *io
1356 \&=for apidoc Amh|GV *|IoFMT_GV|IO *io
1357 \&=for apidoc Amh|char *|IoBOTTOM_NAME|IO *io
1358 \&=for apidoc Amh|GV *|IoBOTTOM_GV|IO *io
1359 \&=for apidoc_section $io
1360 \&=for apidoc Amh|char|IoTYPE|IO *io
1361 \&=for apidoc Amh|U8|IoFLAGS|IO *io
1362 .Ve
1363 .PP
1364 Most of these are involved with formats.
1365 .PP
1366 \&\fBIoFLAGs()\fR may contain a combination of flags, the most interesting of
1367 which are \f(CW\*(C`IOf_FLUSH\*(C'\fR (\f(CW$|\fR) for autoflush and \f(CW\*(C`IOf_UNTAINT\*(C'\fR,
1368 settable with IO::Handle's \fBuntaint()\fR method.
1369 .PP
1370 The IO object may also contains a directory handle:
1371 .PP
1372 .Vb 1
1373 \& DIR *IoDIRP(io);
1374 .Ve
1375 .PP
1376 suitable for use with \fBPerlDir_read()\fR etc.
1377 .PP
1378 All of these accessors macros are lvalues, there are no distinct
1379 \&\f(CW_set()\fR macros to modify the members of the IO object.
1380 .SS "Double-Typed SVs"
1381 .IX Subsection "Double-Typed SVs"
1382 Scalar variables normally contain only one type of value, an integer,
1383 double, pointer, or reference. Perl will automatically convert the
1384 actual scalar data from the stored type into the requested type.
1385 .PP
1386 Some scalar variables contain more than one type of scalar data. For
1387 example, the variable \f(CW$!\fR contains either the numeric value of \f(CW\*(C`errno\*(C'\fR
1388 or its string equivalent from either \f(CW\*(C`strerror\*(C'\fR or \f(CW\*(C`sys_errlist[]\*(C'\fR.
1389 .PP
1390 To force multiple data values into an SV, you must do two things: use the
1391 \&\f(CW\*(C`sv_set*v\*(C'\fR routines to add the additional scalar type, then set a flag
1392 so that Perl will believe it contains more than one type of data. The
1393 four macros to set the flags are:
1394 .PP
1395 .Vb 4
1396 \& SvIOK_on
1397 \& SvNOK_on
1398 \& SvPOK_on
1399 \& SvROK_on
1400 .Ve
1401 .PP
1402 The particular macro you must use depends on which \f(CW\*(C`sv_set*v\*(C'\fR routine
1403 you called first. This is because every \f(CW\*(C`sv_set*v\*(C'\fR routine turns on
1404 only the bit for the particular type of data being set, and turns off
1405 all the rest.
1406 .PP
1407 For example, to create a new Perl variable called "dberror" that contains
1408 both the numeric and descriptive string error values, you could use the
1409 following code:
1410 .PP
1411 .Vb 2
1412 \& extern int dberror;
1413 \& extern char *dberror_list;
1414 \&
1415 \& SV* sv = get_sv("dberror", GV_ADD);
1416 \& sv_setiv(sv, (IV) dberror);
1417 \& sv_setpv(sv, dberror_list[dberror]);
1418 \& SvIOK_on(sv);
1419 .Ve
1420 .PP
1421 If the order of \f(CW\*(C`sv_setiv\*(C'\fR and \f(CW\*(C`sv_setpv\*(C'\fR had been reversed, then the
1422 macro \f(CW\*(C`SvPOK_on\*(C'\fR would need to be called instead of \f(CW\*(C`SvIOK_on\*(C'\fR.
1423 .SS "Read-Only Values"
1424 .IX Subsection "Read-Only Values"
1425 In Perl 5.16 and earlier, copy-on-write (see the next section) shared a
1426 flag bit with read-only scalars. So the only way to test whether
1427 \&\f(CW\*(C`sv_setsv\*(C'\fR, etc., will raise a "Modification of a read-only value" error
1428 in those versions is:
1429 .PP
1430 .Vb 1
1431 \& SvREADONLY(sv) && !SvIsCOW(sv)
1432 .Ve
1433 .PP
1434 Under Perl 5.18 and later, SvREADONLY only applies to read-only variables,
1435 and, under 5.20, copy-on-write scalars can also be read-only, so the above
1436 check is incorrect. You just want:
1437 .PP
1438 .Vb 1
1439 \& SvREADONLY(sv)
1440 .Ve
1441 .PP
1442 If you need to do this check often, define your own macro like this:
1443 .PP
1444 .Vb 5
1445 \& #if PERL_VERSION >= 18
1446 \& # define SvTRULYREADONLY(sv) SvREADONLY(sv)
1447 \& #else
1448 \& # define SvTRULYREADONLY(sv) (SvREADONLY(sv) && !SvIsCOW(sv))
1449 \& #endif
1450 .Ve
1451 .SS "Copy on Write"
1452 .IX Subsection "Copy on Write"
1453 Perl implements a copy-on-write (COW) mechanism for scalars, in which
1454 string copies are not immediately made when requested, but are deferred
1455 until made necessary by one or the other scalar changing. This is mostly
1456 transparent, but one must take care not to modify string buffers that are
1457 shared by multiple SVs.
1458 .PP
1459 You can test whether an SV is using copy-on-write with \f(CWSvIsCOW(sv)\fR.
1460 .PP
1461 You can force an SV to make its own copy of its string buffer by calling \f(CWsv_force_normal(sv)\fR or SvPV_force_nolen(sv).
1462 .PP
1463 If you want to make the SV drop its string buffer, use
1464 \&\f(CW\*(C`sv_force_normal_flags(sv, SV_COW_DROP_PV)\*(C'\fR or simply
1465 \&\f(CW\*(C`sv_setsv(sv, NULL)\*(C'\fR.
1466 .PP
1467 All of these functions will croak on read-only scalars (see the previous
1468 section for more on those).
1469 .PP
1470 To test that your code is behaving correctly and not modifying COW buffers,
1471 on systems that support \fBmmap\fR\|(2) (i.e., Unix) you can configure perl with
1472 \&\f(CW\*(C`\-Accflags=\-DPERL_DEBUG_READONLY_COW\*(C'\fR and it will turn buffer violations
1473 into crashes. You will find it to be marvellously slow, so you may want to
1474 skip perl's own tests.
1475 .SS "Magic Variables"
1476 .IX Subsection "Magic Variables"
1477 [This section still under construction. Ignore everything here. Post no
1478 bills. Everything not permitted is forbidden.]
1479 .PP
1480 Any SV may be magical, that is, it has special features that a normal
1481 SV does not have. These features are stored in the SV structure in a
1482 linked list of \f(CW\*(C`struct magic\*(C'\fR's, typedef'ed to \f(CW\*(C`MAGIC\*(C'\fR.
1483 .PP
1484 .Vb 10
1485 \& struct magic {
1486 \& MAGIC* mg_moremagic;
1487 \& MGVTBL* mg_virtual;
1488 \& U16 mg_private;
1489 \& char mg_type;
1490 \& U8 mg_flags;
1491 \& I32 mg_len;
1492 \& SV* mg_obj;
1493 \& char* mg_ptr;
1494 \& };
1495 .Ve
1496 .PP
1497 Note this is current as of patchlevel 0, and could change at any time.
1498 .SS "Assigning Magic"
1499 .IX Subsection "Assigning Magic"
1500 Perl adds magic to an SV using the sv_magic function:
1501 .PP
1502 .Vb 1
1503 \& void sv_magic(SV* sv, SV* obj, int how, const char* name, I32 namlen);
1504 .Ve
1505 .PP
1506 The \f(CW\*(C`sv\*(C'\fR argument is a pointer to the SV that is to acquire a new magical
1507 feature.
1508 .PP
1509 If \f(CW\*(C`sv\*(C'\fR is not already magical, Perl uses the \f(CW\*(C`SvUPGRADE\*(C'\fR macro to
1510 convert \f(CW\*(C`sv\*(C'\fR to type \f(CW\*(C`SVt_PVMG\*(C'\fR.
1511 Perl then continues by adding new magic
1512 to the beginning of the linked list of magical features. Any prior entry
1513 of the same type of magic is deleted. Note that this can be overridden,
1514 and multiple instances of the same type of magic can be associated with an
1515 SV.
1516 .PP
1517 The \f(CW\*(C`name\*(C'\fR and \f(CW\*(C`namlen\*(C'\fR arguments are used to associate a string with
1518 the magic, typically the name of a variable. \f(CW\*(C`namlen\*(C'\fR is stored in the
1519 \&\f(CW\*(C`mg_len\*(C'\fR field and if \f(CW\*(C`name\*(C'\fR is non-null then either a \f(CW\*(C`savepvn\*(C'\fR copy of
1520 \&\f(CW\*(C`name\*(C'\fR or \f(CW\*(C`name\*(C'\fR itself is stored in the \f(CW\*(C`mg_ptr\*(C'\fR field, depending on
1521 whether \f(CW\*(C`namlen\*(C'\fR is greater than zero or equal to zero respectively. As a
1522 special case, if \f(CW\*(C`(name && namlen == HEf_SVKEY)\*(C'\fR then \f(CW\*(C`name\*(C'\fR is assumed
1523 to contain an \f(CW\*(C`SV*\*(C'\fR and is stored as-is with its REFCNT incremented.
1524 .PP
1525 The sv_magic function uses \f(CW\*(C`how\*(C'\fR to determine which, if any, predefined
1526 "Magic Virtual Table" should be assigned to the \f(CW\*(C`mg_virtual\*(C'\fR field.
1527 See the "Magic Virtual Tables" section below. The \f(CW\*(C`how\*(C'\fR argument is also
1528 stored in the \f(CW\*(C`mg_type\*(C'\fR field. The value of
1529 \&\f(CW\*(C`how\*(C'\fR should be chosen from the set of macros
1530 \&\f(CW\*(C`PERL_MAGIC_foo\*(C'\fR found in \fIperl.h\fR. Note that before
1531 these macros were added, Perl internals used to directly use character
1532 literals, so you may occasionally come across old code or documentation
1533 referring to 'U' magic rather than \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR for example.
1534 .PP
1535 The \f(CW\*(C`obj\*(C'\fR argument is stored in the \f(CW\*(C`mg_obj\*(C'\fR field of the \f(CW\*(C`MAGIC\*(C'\fR
1536 structure. If it is not the same as the \f(CW\*(C`sv\*(C'\fR argument, the reference
1537 count of the \f(CW\*(C`obj\*(C'\fR object is incremented. If it is the same, or if
1538 the \f(CW\*(C`how\*(C'\fR argument is \f(CW\*(C`PERL_MAGIC_arylen\*(C'\fR, \f(CW\*(C`PERL_MAGIC_regdatum\*(C'\fR,
1539 \&\f(CW\*(C`PERL_MAGIC_regdata\*(C'\fR, or if it is a NULL pointer, then \f(CW\*(C`obj\*(C'\fR is merely
1540 stored, without the reference count being incremented.
1541 .PP
1542 See also \f(CW\*(C`sv_magicext\*(C'\fR in perlapi for a more flexible way to add magic
1543 to an SV.
1544 .PP
1545 There is also a function to add magic to an \f(CW\*(C`HV\*(C'\fR:
1546 .PP
1547 .Vb 1
1548 \& void hv_magic(HV *hv, GV *gv, int how);
1549 .Ve
1550 .PP
1551 This simply calls \f(CW\*(C`sv_magic\*(C'\fR and coerces the \f(CW\*(C`gv\*(C'\fR argument into an \f(CW\*(C`SV\*(C'\fR.
1552 .PP
1553 To remove the magic from an SV, call the function sv_unmagic:
1554 .PP
1555 .Vb 1
1556 \& int sv_unmagic(SV *sv, int type);
1557 .Ve
1558 .PP
1559 The \f(CW\*(C`type\*(C'\fR argument should be equal to the \f(CW\*(C`how\*(C'\fR value when the \f(CW\*(C`SV\*(C'\fR
1560 was initially made magical.
1561 .PP
1562 However, note that \f(CW\*(C`sv_unmagic\*(C'\fR removes all magic of a certain \f(CW\*(C`type\*(C'\fR from the
1563 \&\f(CW\*(C`SV\*(C'\fR. If you want to remove only certain
1564 magic of a \f(CW\*(C`type\*(C'\fR based on the magic
1565 virtual table, use \f(CW\*(C`sv_unmagicext\*(C'\fR instead:
1566 .PP
1567 .Vb 1
1568 \& int sv_unmagicext(SV *sv, int type, MGVTBL *vtbl);
1569 .Ve
1570 .SS "Magic Virtual Tables"
1571 .IX Subsection "Magic Virtual Tables"
1572 The \f(CW\*(C`mg_virtual\*(C'\fR field in the \f(CW\*(C`MAGIC\*(C'\fR structure is a pointer to an
1573 \&\f(CW\*(C`MGVTBL\*(C'\fR, which is a structure of function pointers and stands for
1574 "Magic Virtual Table" to handle the various operations that might be
1575 applied to that variable.
1576 .PP
1577 The \f(CW\*(C`MGVTBL\*(C'\fR has five (or sometimes eight) pointers to the following
1578 routine types:
1579 .PP
1580 .Vb 5
1581 \& int (*svt_get) (pTHX_ SV* sv, MAGIC* mg);
1582 \& int (*svt_set) (pTHX_ SV* sv, MAGIC* mg);
1583 \& U32 (*svt_len) (pTHX_ SV* sv, MAGIC* mg);
1584 \& int (*svt_clear)(pTHX_ SV* sv, MAGIC* mg);
1585 \& int (*svt_free) (pTHX_ SV* sv, MAGIC* mg);
1586 \&
1587 \& int (*svt_copy) (pTHX_ SV *sv, MAGIC* mg, SV *nsv,
1588 \& const char *name, I32 namlen);
1589 \& int (*svt_dup) (pTHX_ MAGIC *mg, CLONE_PARAMS *param);
1590 \& int (*svt_local)(pTHX_ SV *nsv, MAGIC *mg);
1591 .Ve
1592 .PP
1593 This MGVTBL structure is set at compile-time in \fIperl.h\fR and there are
1594 currently 32 types. These different structures contain pointers to various
1595 routines that perform additional actions depending on which function is
1596 being called.
1597 .PP
1598 .Vb 8
1599 \& Function pointer Action taken
1600 \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-
1601 \& svt_get Do something before the value of the SV is
1602 \& retrieved.
1603 \& svt_set Do something after the SV is assigned a value.
1604 \& svt_len Report on the SV\*(Aqs length.
1605 \& svt_clear Clear something the SV represents.
1606 \& svt_free Free any extra storage associated with the SV.
1607 \&
1608 \& svt_copy copy tied variable magic to a tied element
1609 \& svt_dup duplicate a magic structure during thread cloning
1610 \& svt_local copy magic to local value during \*(Aqlocal\*(Aq
1611 .Ve
1612 .PP
1613 For instance, the MGVTBL structure called \f(CW\*(C`vtbl_sv\*(C'\fR (which corresponds
1614 to an \f(CW\*(C`mg_type\*(C'\fR of \f(CW\*(C`PERL_MAGIC_sv\*(C'\fR) contains:
1615 .PP
1616 .Vb 1
1617 \& { magic_get, magic_set, magic_len, 0, 0 }
1618 .Ve
1619 .PP
1620 Thus, when an SV is determined to be magical and of type \f(CW\*(C`PERL_MAGIC_sv\*(C'\fR,
1621 if a get operation is being performed, the routine \f(CW\*(C`magic_get\*(C'\fR is
1622 called. All the various routines for the various magical types begin
1623 with \f(CW\*(C`magic_\*(C'\fR. NOTE: the magic routines are not considered part of
1624 the Perl API, and may not be exported by the Perl library.
1625 .PP
1626 The last three slots are a recent addition, and for source code
1627 compatibility they are only checked for if one of the three flags
1628 \&\f(CW\*(C`MGf_COPY\*(C'\fR, \f(CW\*(C`MGf_DUP\*(C'\fR, or \f(CW\*(C`MGf_LOCAL\*(C'\fR is set in mg_flags.
1629 This means that most code can continue declaring
1630 a vtable as a 5\-element value. These three are
1631 currently used exclusively by the threading code, and are highly subject
1632 to change.
1633 .PP
1634 The current kinds of Magic Virtual Tables are:
1635 .PP
1636 .Vb 10
1637 \& mg_type
1638 \& (old\-style char and macro) MGVTBL Type of magic
1639 \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-
1640 \& \e0 PERL_MAGIC_sv vtbl_sv Special scalar variable
1641 \& # PERL_MAGIC_arylen vtbl_arylen Array length ($#ary)
1642 \& % PERL_MAGIC_rhash (none) Extra data for restricted
1643 \& hashes
1644 \& * PERL_MAGIC_debugvar vtbl_debugvar $DB::single, signal, trace
1645 \& vars
1646 \& . PERL_MAGIC_pos vtbl_pos pos() lvalue
1647 \& : PERL_MAGIC_symtab (none) Extra data for symbol
1648 \& tables
1649 \& < PERL_MAGIC_backref vtbl_backref For weak ref data
1650 \& @ PERL_MAGIC_arylen_p (none) To move arylen out of XPVAV
1651 \& B PERL_MAGIC_bm vtbl_regexp Boyer\-Moore
1652 \& (fast string search)
1653 \& c PERL_MAGIC_overload_table vtbl_ovrld Holds overload table
1654 \& (AMT) on stash
1655 \& D PERL_MAGIC_regdata vtbl_regdata Regex match position data
1656 \& (@+ and @\- vars)
1657 \& d PERL_MAGIC_regdatum vtbl_regdatum Regex match position data
1658 \& element
1659 \& E PERL_MAGIC_env vtbl_env %ENV hash
1660 \& e PERL_MAGIC_envelem vtbl_envelem %ENV hash element
1661 \& f PERL_MAGIC_fm vtbl_regexp Formline
1662 \& (\*(Aqcompiled\*(Aq format)
1663 \& g PERL_MAGIC_regex_global vtbl_mglob m//g target
1664 \& H PERL_MAGIC_hints vtbl_hints %^H hash
1665 \& h PERL_MAGIC_hintselem vtbl_hintselem %^H hash element
1666 \& I PERL_MAGIC_isa vtbl_isa @ISA array
1667 \& i PERL_MAGIC_isaelem vtbl_isaelem @ISA array element
1668 \& k PERL_MAGIC_nkeys vtbl_nkeys scalar(keys()) lvalue
1669 \& L PERL_MAGIC_dbfile (none) Debugger %_<filename
1670 \& l PERL_MAGIC_dbline vtbl_dbline Debugger %_<filename
1671 \& element
1672 \& N PERL_MAGIC_shared (none) Shared between threads
1673 \& n PERL_MAGIC_shared_scalar (none) Shared between threads
1674 \& o PERL_MAGIC_collxfrm vtbl_collxfrm Locale transformation
1675 \& P PERL_MAGIC_tied vtbl_pack Tied array or hash
1676 \& p PERL_MAGIC_tiedelem vtbl_packelem Tied array or hash element
1677 \& q PERL_MAGIC_tiedscalar vtbl_packelem Tied scalar or handle
1678 \& r PERL_MAGIC_qr vtbl_regexp Precompiled qr// regex
1679 \& S PERL_MAGIC_sig vtbl_sig %SIG hash
1680 \& s PERL_MAGIC_sigelem vtbl_sigelem %SIG hash element
1681 \& t PERL_MAGIC_taint vtbl_taint Taintedness
1682 \& U PERL_MAGIC_uvar vtbl_uvar Available for use by
1683 \& extensions
1684 \& u PERL_MAGIC_uvar_elem (none) Reserved for use by
1685 \& extensions
1686 \& V PERL_MAGIC_vstring (none) SV was vstring literal
1687 \& v PERL_MAGIC_vec vtbl_vec vec() lvalue
1688 \& w PERL_MAGIC_utf8 vtbl_utf8 Cached UTF\-8 information
1689 \& X PERL_MAGIC_destruct vtbl_destruct destruct callback
1690 \& x PERL_MAGIC_substr vtbl_substr substr() lvalue
1691 \& Y PERL_MAGIC_nonelem vtbl_nonelem Array element that does not
1692 \& exist
1693 \& y PERL_MAGIC_defelem vtbl_defelem Shadow "foreach" iterator
1694 \& variable / smart parameter
1695 \& vivification
1696 \& Z PERL_MAGIC_hook vtbl_hook %{^HOOK} hash
1697 \& z PERL_MAGIC_hookelem vtbl_hookelem %{^HOOK} hash element
1698 \& \e PERL_MAGIC_lvref vtbl_lvref Lvalue reference
1699 \& constructor
1700 \& ] PERL_MAGIC_checkcall vtbl_checkcall Inlining/mutation of call
1701 \& to this CV
1702 \& ^ PERL_MAGIC_extvalue (none) Value magic available for
1703 \& use by extensions
1704 \& ~ PERL_MAGIC_ext (none) Variable magic available
1705 \& for use by extensions
1706 .Ve
1707 .PP
1708 When an uppercase and lowercase letter both exist in the table, then the
1709 uppercase letter is typically used to represent some kind of composite type
1710 (a list or a hash), and the lowercase letter is used to represent an element
1711 of that composite type. Some internals code makes use of this case
1712 relationship. However, 'v' and 'V' (vec and v\-string) are in no way related.
1713 .PP
1714 The \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR, \f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR and \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic types
1715 are defined specifically for use by extensions and will not be used by perl
1716 itself. Extensions can use \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR or \f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR magic to
1717 \&'attach' private information to variables (typically objects). This is
1718 especially useful because there is no way for normal perl code to corrupt this
1719 private information (unlike using extra elements of a hash object).
1720 \&\f(CW\*(C`PERL_MAGIC_extvalue\*(C'\fR is value magic (unlike \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR and
1721 \&\f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR) meaning that on localization the new value will not be
1722 magical.
1723 .PP
1724 Similarly, \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic can be used much like \fBtie()\fR to call a
1725 C function any time a scalar's value is used or changed. The \f(CW\*(C`MAGIC\*(C'\fR's
1726 \&\f(CW\*(C`mg_ptr\*(C'\fR field points to a \f(CW\*(C`ufuncs\*(C'\fR structure:
1727 .PP
1728 .Vb 5
1729 \& struct ufuncs {
1730 \& I32 (*uf_val)(pTHX_ IV, SV*);
1731 \& I32 (*uf_set)(pTHX_ IV, SV*);
1732 \& IV uf_index;
1733 \& };
1734 .Ve
1735 .PP
1736 When the SV is read from or written to, the \f(CW\*(C`uf_val\*(C'\fR or \f(CW\*(C`uf_set\*(C'\fR
1737 function will be called with \f(CW\*(C`uf_index\*(C'\fR as the first arg and a pointer to
1738 the SV as the second. A simple example of how to add \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR
1739 magic is shown below. Note that the ufuncs structure is copied by
1740 sv_magic, so you can safely allocate it on the stack.
1741 .PP
1742 .Vb 10
1743 \& void
1744 \& Umagic(sv)
1745 \& SV *sv;
1746 \& PREINIT:
1747 \& struct ufuncs uf;
1748 \& CODE:
1749 \& uf.uf_val = &my_get_fn;
1750 \& uf.uf_set = &my_set_fn;
1751 \& uf.uf_index = 0;
1752 \& sv_magic(sv, 0, PERL_MAGIC_uvar, (char*)&uf, sizeof(uf));
1753 .Ve
1754 .PP
1755 Attaching \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR to arrays is permissible but has no effect.
1756 .PP
1757 For hashes there is a specialized hook that gives control over hash
1758 keys (but not values). This hook calls \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR 'get' magic
1759 if the "set" function in the \f(CW\*(C`ufuncs\*(C'\fR structure is NULL. The hook
1760 is activated whenever the hash is accessed with a key specified as
1761 an \f(CW\*(C`SV\*(C'\fR through the functions \f(CW\*(C`hv_store_ent\*(C'\fR, \f(CW\*(C`hv_fetch_ent\*(C'\fR,
1762 \&\f(CW\*(C`hv_delete_ent\*(C'\fR, and \f(CW\*(C`hv_exists_ent\*(C'\fR. Accessing the key as a string
1763 through the functions without the \f(CW\*(C`..._ent\*(C'\fR suffix circumvents the
1764 hook. See "GUTS" in Hash::Util::FieldHash for a detailed description.
1765 .PP
1766 Note that because multiple extensions may be using \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR
1767 or \f(CW\*(C`PERL_MAGIC_uvar\*(C'\fR magic, it is important for extensions to take
1768 extra care to avoid conflict. Typically only using the magic on
1769 objects blessed into the same class as the extension is sufficient.
1770 For \f(CW\*(C`PERL_MAGIC_ext\*(C'\fR magic, it is usually a good idea to define an
1771 \&\f(CW\*(C`MGVTBL\*(C'\fR, even if all its fields will be \f(CW0\fR, so that individual
1772 \&\f(CW\*(C`MAGIC\*(C'\fR pointers can be identified as a particular kind of magic
1773 using their magic virtual table. \f(CW\*(C`mg_findext\*(C'\fR provides an easy way
1774 to do that:
1775 .PP
1776 .Vb 1
1777 \& STATIC MGVTBL my_vtbl = { 0, 0, 0, 0, 0, 0, 0, 0 };
1778 \&
1779 \& MAGIC *mg;
1780 \& if ((mg = mg_findext(sv, PERL_MAGIC_ext, &my_vtbl))) {
1781 \& /* this is really ours, not another module\*(Aqs PERL_MAGIC_ext */
1782 \& my_priv_data_t *priv = (my_priv_data_t *)mg\->mg_ptr;
1783 \& ...
1784 \& }
1785 .Ve
1786 .PP
1787 Also note that the \f(CW\*(C`sv_set*()\*(C'\fR and \f(CW\*(C`sv_cat*()\*(C'\fR functions described
1788 earlier do \fBnot\fR invoke 'set' magic on their targets. This must
1789 be done by the user either by calling the \f(CWSvSETMAGIC()\fR macro after
1790 calling these functions, or by using one of the \f(CW\*(C`sv_set*_mg()\*(C'\fR or
1791 \&\f(CW\*(C`sv_cat*_mg()\*(C'\fR functions. Similarly, generic C code must call the
1792 \&\f(CWSvGETMAGIC()\fR macro to invoke any 'get' magic if they use an SV
1793 obtained from external sources in functions that don't handle magic.
1794 See perlapi for a description of these functions.
1795 For example, calls to the \f(CW\*(C`sv_cat*()\*(C'\fR functions typically need to be
1796 followed by \f(CWSvSETMAGIC()\fR, but they don't need a prior \f(CWSvGETMAGIC()\fR
1797 since their implementation handles 'get' magic.
1798 .SS "Finding Magic"
1799 .IX Subsection "Finding Magic"
1800 .Vb 2
1801 \& MAGIC *mg_find(SV *sv, int type); /* Finds the magic pointer of that
1802 \& * type */
1803 .Ve
1804 .PP
1805 This routine returns a pointer to a \f(CW\*(C`MAGIC\*(C'\fR structure stored in the SV.
1806 If the SV does not have that magical
1807 feature, \f(CW\*(C`NULL\*(C'\fR is returned. If the
1808 SV has multiple instances of that magical feature, the first one will be
1809 returned. \f(CW\*(C`mg_findext\*(C'\fR can be used
1810 to find a \f(CW\*(C`MAGIC\*(C'\fR structure of an SV
1811 based on both its magic type and its magic virtual table:
1812 .PP
1813 .Vb 1
1814 \& MAGIC *mg_findext(SV *sv, int type, MGVTBL *vtbl);
1815 .Ve
1816 .PP
1817 Also, if the SV passed to \f(CW\*(C`mg_find\*(C'\fR or \f(CW\*(C`mg_findext\*(C'\fR is not of type
1818 SVt_PVMG, Perl may core dump.
1819 .PP
1820 .Vb 1
1821 \& int mg_copy(SV* sv, SV* nsv, const char* key, STRLEN klen);
1822 .Ve
1823 .PP
1824 This routine checks to see what types of magic \f(CW\*(C`sv\*(C'\fR has. If the mg_type
1825 field is an uppercase letter, then the mg_obj is copied to \f(CW\*(C`nsv\*(C'\fR, but
1826 the mg_type field is changed to be the lowercase letter.
1827 .SS "Understanding the Magic of Tied Hashes and Arrays"
1828 .IX Subsection "Understanding the Magic of Tied Hashes and Arrays"
1829 Tied hashes and arrays are magical beasts of the \f(CW\*(C`PERL_MAGIC_tied\*(C'\fR
1830 magic type.
1831 .PP
1832 WARNING: As of the 5.004 release, proper usage of the array and hash
1833 access functions requires understanding a few caveats. Some
1834 of these caveats are actually considered bugs in the API, to be fixed
1835 in later releases, and are bracketed with [MAYCHANGE] below. If
1836 you find yourself actually applying such information in this section, be
1837 aware that the behavior may change in the future, umm, without warning.
1838 .PP
1839 The perl tie function associates a variable with an object that implements
1840 the various GET, SET, etc methods. To perform the equivalent of the perl
1841 tie function from an XSUB, you must mimic this behaviour. The code below
1842 carries out the necessary steps \-\- firstly it creates a new hash, and then
1843 creates a second hash which it blesses into the class which will implement
1844 the tie methods. Lastly it ties the two hashes together, and returns a
1845 reference to the new tied hash. Note that the code below does NOT call the
1846 TIEHASH method in the MyTie class \-
1847 see "Calling Perl Routines from within C Programs" for details on how
1848 to do this.
1849 .PP
1850 .Vb 10
1851 \& SV*
1852 \& mytie()
1853 \& PREINIT:
1854 \& HV *hash;
1855 \& HV *stash;
1856 \& SV *tie;
1857 \& CODE:
1858 \& hash = newHV();
1859 \& tie = newRV_noinc((SV*)newHV());
1860 \& stash = gv_stashpv("MyTie", GV_ADD);
1861 \& sv_bless(tie, stash);
1862 \& hv_magic(hash, (GV*)tie, PERL_MAGIC_tied);
1863 \& SvREFCNT_dec(tie); /* hv_magic() increases tie ref count */
1864 \& RETVAL = newRV_noinc(hash);
1865 \& OUTPUT:
1866 \& RETVAL
1867 .Ve
1868 .PP
1869 The \f(CW\*(C`av_store\*(C'\fR function, when given a tied array argument, merely
1870 copies the magic of the array onto the value to be "stored", using
1871 \&\f(CW\*(C`mg_copy\*(C'\fR. It may also return NULL, indicating that the value did not
1872 actually need to be stored in the array. [MAYCHANGE] After a call to
1873 \&\f(CW\*(C`av_store\*(C'\fR on a tied array, the caller will usually need to call
1874 \&\f(CWmg_set(val)\fR to actually invoke the perl level "STORE" method on the
1875 TIEARRAY object. If \f(CW\*(C`av_store\*(C'\fR did return NULL, a call to
1876 \&\f(CWSvREFCNT_dec(val)\fR will also be usually necessary to avoid a memory
1877 leak. [/MAYCHANGE]
1878 .PP
1879 The previous paragraph is applicable verbatim to tied hash access using the
1880 \&\f(CW\*(C`hv_store\*(C'\fR and \f(CW\*(C`hv_store_ent\*(C'\fR functions as well.
1881 .PP
1882 \&\f(CW\*(C`av_fetch\*(C'\fR and the corresponding hash functions \f(CW\*(C`hv_fetch\*(C'\fR and
1883 \&\f(CW\*(C`hv_fetch_ent\*(C'\fR actually return an undefined mortal value whose magic
1884 has been initialized using \f(CW\*(C`mg_copy\*(C'\fR. Note the value so returned does not
1885 need to be deallocated, as it is already mortal. [MAYCHANGE] But you will
1886 need to call \f(CWmg_get()\fR on the returned value in order to actually invoke
1887 the perl level "FETCH" method on the underlying TIE object. Similarly,
1888 you may also call \f(CWmg_set()\fR on the return value after possibly assigning
1889 a suitable value to it using \f(CW\*(C`sv_setsv\*(C'\fR, which will invoke the "STORE"
1890 method on the TIE object. [/MAYCHANGE]
1891 .PP
1892 [MAYCHANGE]
1893 In other words, the array or hash fetch/store functions don't really
1894 fetch and store actual values in the case of tied arrays and hashes. They
1895 merely call \f(CW\*(C`mg_copy\*(C'\fR to attach magic to the values that were meant to be
1896 "stored" or "fetched". Later calls to \f(CW\*(C`mg_get\*(C'\fR and \f(CW\*(C`mg_set\*(C'\fR actually
1897 do the job of invoking the TIE methods on the underlying objects. Thus
1898 the magic mechanism currently implements a kind of lazy access to arrays
1899 and hashes.
1900 .PP
1901 Currently (as of perl version 5.004), use of the hash and array access
1902 functions requires the user to be aware of whether they are operating on
1903 "normal" hashes and arrays, or on their tied variants. The API may be
1904 changed to provide more transparent access to both tied and normal data
1905 types in future versions.
1906 [/MAYCHANGE]
1907 .PP
1908 You would do well to understand that the TIEARRAY and TIEHASH interfaces
1909 are mere sugar to invoke some perl method calls while using the uniform hash
1910 and array syntax. The use of this sugar imposes some overhead (typically
1911 about two to four extra opcodes per FETCH/STORE operation, in addition to
1912 the creation of all the mortal variables required to invoke the methods).
1913 This overhead will be comparatively small if the TIE methods are themselves
1914 substantial, but if they are only a few statements long, the overhead
1915 will not be insignificant.
1916 .SS "Localizing changes"
1917 .IX Subsection "Localizing changes"
1918 Perl has a very handy construction
1919 .PP
1920 .Vb 4
1921 \& {
1922 \& local $var = 2;
1923 \& ...
1924 \& }
1925 .Ve
1926 .PP
1927 This construction is \fIapproximately\fR equivalent to
1928 .PP
1929 .Vb 6
1930 \& {
1931 \& my $oldvar = $var;
1932 \& $var = 2;
1933 \& ...
1934 \& $var = $oldvar;
1935 \& }
1936 .Ve
1937 .PP
1938 The biggest difference is that the first construction would
1939 reinstate the initial value of \f(CW$var\fR, irrespective of how control exits
1940 the block: \f(CW\*(C`goto\*(C'\fR, \f(CW\*(C`return\*(C'\fR, \f(CW\*(C`die\*(C'\fR/\f(CW\*(C`eval\*(C'\fR, etc. It is a little bit
1941 more efficient as well.
1942 .PP
1943 There is a way to achieve a similar task from C via Perl API: create a
1944 \&\fIpseudo-block\fR, and arrange for some changes to be automatically
1945 undone at the end of it, either explicit, or via a non-local exit (via
1946 \&\fBdie()\fR). A \fIblock\fR\-like construct is created by a pair of
1947 \&\f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR macros (see "Returning a Scalar" in perlcall).
1948 Such a construct may be created specially for some important localized
1949 task, or an existing one (like boundaries of enclosing Perl
1950 subroutine/block, or an existing pair for freeing TMPs) may be
1951 used. (In the second case the overhead of additional localization must
1952 be almost negligible.) Note that any XSUB is automatically enclosed in
1953 an \f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR pair.
1954 .PP
1955 Inside such a \fIpseudo-block\fR the following service is available:
1956 .ie n .IP """SAVEINT(int i)""" 4
1957 .el .IP "\f(CWSAVEINT(int i)\fR" 4
1958 .IX Item "SAVEINT(int i)"
1959 .PD 0
1960 .ie n .IP """SAVEIV(IV i)""" 4
1961 .el .IP "\f(CWSAVEIV(IV i)\fR" 4
1962 .IX Item "SAVEIV(IV i)"
1963 .ie n .IP """SAVEI32(I32 i)""" 4
1964 .el .IP "\f(CWSAVEI32(I32 i)\fR" 4
1965 .IX Item "SAVEI32(I32 i)"
1966 .ie n .IP """SAVELONG(long i)""" 4
1967 .el .IP "\f(CWSAVELONG(long i)\fR" 4
1968 .IX Item "SAVELONG(long i)"
1969 .ie n .IP """SAVEI8(I8 i)""" 4
1970 .el .IP "\f(CWSAVEI8(I8 i)\fR" 4
1971 .IX Item "SAVEI8(I8 i)"
1972 .ie n .IP """SAVEI16(I16 i)""" 4
1973 .el .IP "\f(CWSAVEI16(I16 i)\fR" 4
1974 .IX Item "SAVEI16(I16 i)"
1975 .ie n .IP """SAVEBOOL(int i)""" 4
1976 .el .IP "\f(CWSAVEBOOL(int i)\fR" 4
1977 .IX Item "SAVEBOOL(int i)"
1978 .ie n .IP """SAVESTRLEN(STRLEN i)""" 4
1979 .el .IP "\f(CWSAVESTRLEN(STRLEN i)\fR" 4
1980 .IX Item "SAVESTRLEN(STRLEN i)"
1981 .PD
1982 These macros arrange things to restore the value of integer variable
1983 \&\f(CW\*(C`i\*(C'\fR at the end of the enclosing \fIpseudo-block\fR.
1984 .ie n .IP SAVESPTR(s) 4
1985 .el .IP \f(CWSAVESPTR(s)\fR 4
1986 .IX Item "SAVESPTR(s)"
1987 .PD 0
1988 .ie n .IP SAVEPPTR(p) 4
1989 .el .IP \f(CWSAVEPPTR(p)\fR 4
1990 .IX Item "SAVEPPTR(p)"
1991 .PD
1992 These macros arrange things to restore the value of pointers \f(CW\*(C`s\*(C'\fR and
1993 \&\f(CW\*(C`p\*(C'\fR. \f(CW\*(C`s\*(C'\fR must be a pointer of a type which survives conversion to
1994 \&\f(CW\*(C`SV*\*(C'\fR and back, \f(CW\*(C`p\*(C'\fR should be able to survive conversion to \f(CW\*(C`char*\*(C'\fR
1995 and back.
1996 .ie n .IP """SAVERCPV(char **ppv)""" 4
1997 .el .IP "\f(CWSAVERCPV(char **ppv)\fR" 4
1998 .IX Item "SAVERCPV(char **ppv)"
1999 This macro arranges to restore the value of a \f(CW\*(C`char *\*(C'\fR variable which
2000 was allocated with a call to \f(CWrcpv_new()\fR to its previous state when
2001 the current pseudo block is completed. The pointer stored in \f(CW*ppv\fR at
2002 the time of the call will be refcount incremented and stored on the save
2003 stack. Later when the current \fIpseudo-block\fR is completed the value
2004 stored in \f(CW*ppv\fR will be refcount decremented, and the previous value
2005 restored from the savestack which will also be refcount decremented.
2006 .Sp
2007 This is the \f(CW\*(C`RCPV\*(C'\fR equivalent of \f(CWSAVEGENERICSV()\fR.
2008 .ie n .IP """SAVEGENERICSV(SV **psv)""" 4
2009 .el .IP "\f(CWSAVEGENERICSV(SV **psv)\fR" 4
2010 .IX Item "SAVEGENERICSV(SV **psv)"
2011 This macro arranges to restore the value of a \f(CW\*(C`SV *\*(C'\fR variable to its
2012 previous state when the current pseudo block is completed. The pointer
2013 stored in \f(CW*psv\fR at the time of the call will be refcount incremented
2014 and stored on the save stack. Later when the current \fIpseudo-block\fR is
2015 completed the value stored in \f(CW*ppv\fR will be refcount decremented, and
2016 the previous value restored from the savestack which will also be refcount
2017 decremented. This the C equivalent of \f(CW\*(C`local $sv\*(C'\fR.
2018 .ie n .IP """SAVEFREESV(SV *sv)""" 4
2019 .el .IP "\f(CWSAVEFREESV(SV *sv)\fR" 4
2020 .IX Item "SAVEFREESV(SV *sv)"
2021 The refcount of \f(CW\*(C`sv\*(C'\fR will be decremented at the end of
2022 \&\fIpseudo-block\fR. This is similar to \f(CW\*(C`sv_2mortal\*(C'\fR in that it is also a
2023 mechanism for doing a delayed \f(CW\*(C`SvREFCNT_dec\*(C'\fR. However, while \f(CW\*(C`sv_2mortal\*(C'\fR
2024 extends the lifetime of \f(CW\*(C`sv\*(C'\fR until the beginning of the next statement,
2025 \&\f(CW\*(C`SAVEFREESV\*(C'\fR extends it until the end of the enclosing scope. These
2026 lifetimes can be wildly different.
2027 .Sp
2028 Also compare \f(CW\*(C`SAVEMORTALIZESV\*(C'\fR.
2029 .ie n .IP """SAVEMORTALIZESV(SV *sv)""" 4
2030 .el .IP "\f(CWSAVEMORTALIZESV(SV *sv)\fR" 4
2031 .IX Item "SAVEMORTALIZESV(SV *sv)"
2032 Just like \f(CW\*(C`SAVEFREESV\*(C'\fR, but mortalizes \f(CW\*(C`sv\*(C'\fR at the end of the current
2033 scope instead of decrementing its reference count. This usually has the
2034 effect of keeping \f(CW\*(C`sv\*(C'\fR alive until the statement that called the currently
2035 live scope has finished executing.
2036 .ie n .IP """SAVEFREEOP(OP *op)""" 4
2037 .el .IP "\f(CWSAVEFREEOP(OP *op)\fR" 4
2038 .IX Item "SAVEFREEOP(OP *op)"
2039 The \f(CW\*(C`OP *\*(C'\fR is \f(CWop_free()\fRed at the end of \fIpseudo-block\fR.
2040 .ie n .IP SAVEFREEPV(p) 4
2041 .el .IP \f(CWSAVEFREEPV(p)\fR 4
2042 .IX Item "SAVEFREEPV(p)"
2043 The chunk of memory which is pointed to by \f(CW\*(C`p\*(C'\fR is \f(CWSafefree()\fRed at the
2044 end of the current \fIpseudo-block\fR.
2045 .ie n .IP """SAVEFREERCPV(char *pv)""" 4
2046 .el .IP "\f(CWSAVEFREERCPV(char *pv)\fR" 4
2047 .IX Item "SAVEFREERCPV(char *pv)"
2048 Ensures that a \f(CW\*(C`char *\*(C'\fR which was created by a call to \f(CWrcpv_new()\fR is
2049 \&\f(CWrcpv_free()\fRed at the end of the current \fIpseudo-block\fR.
2050 .Sp
2051 This is the RCPV equivalent of \f(CWSAVEFREESV()\fR.
2052 .ie n .IP """SAVECLEARSV(SV *sv)""" 4
2053 .el .IP "\f(CWSAVECLEARSV(SV *sv)\fR" 4
2054 .IX Item "SAVECLEARSV(SV *sv)"
2055 Clears a slot in the current scratchpad which corresponds to \f(CW\*(C`sv\*(C'\fR at
2056 the end of \fIpseudo-block\fR.
2057 .ie n .IP """SAVEDELETE(HV *hv, char *key, I32 length)""" 4
2058 .el .IP "\f(CWSAVEDELETE(HV *hv, char *key, I32 length)\fR" 4
2059 .IX Item "SAVEDELETE(HV *hv, char *key, I32 length)"
2060 The key \f(CW\*(C`key\*(C'\fR of \f(CW\*(C`hv\*(C'\fR is deleted at the end of \fIpseudo-block\fR. The
2061 string pointed to by \f(CW\*(C`key\*(C'\fR is \fBSafefree()\fRed. If one has a \fIkey\fR in
2062 short-lived storage, the corresponding string may be reallocated like
2063 this:
2064 .Sp
2065 .Vb 1
2066 \& SAVEDELETE(PL_defstash, savepv(tmpbuf), strlen(tmpbuf));
2067 .Ve
2068 .ie n .IP """SAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)""" 4
2069 .el .IP "\f(CWSAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)\fR" 4
2070 .IX Item "SAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)"
2071 At the end of \fIpseudo-block\fR the function \f(CW\*(C`f\*(C'\fR is called with the
2072 only argument \f(CW\*(C`p\*(C'\fR which may be NULL.
2073 .ie n .IP """SAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)""" 4
2074 .el .IP "\f(CWSAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)\fR" 4
2075 .IX Item "SAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)"
2076 At the end of \fIpseudo-block\fR the function \f(CW\*(C`f\*(C'\fR is called with the
2077 implicit context argument (if any), and \f(CW\*(C`p\*(C'\fR which may be NULL.
2078 .Sp
2079 Note the \fIend of the current pseudo-block\fR may occur much later than
2080 the \fIend of the current statement\fR. You may wish to look at the
2081 \&\f(CWMORTALSVFUNC_X()\fR macro instead.
2082 .ie n .IP """MORTALSVFUNC_X(SVFUNC_t f, SV *sv)""" 4
2083 .el .IP "\f(CWMORTALSVFUNC_X(SVFUNC_t f, SV *sv)\fR" 4
2084 .IX Item "MORTALSVFUNC_X(SVFUNC_t f, SV *sv)"
2085 At the end of \fIthe current statement\fR the function \f(CW\*(C`f\*(C'\fR is called with
2086 the implicit context argument (if any), and \f(CW\*(C`sv\*(C'\fR which may be NULL.
2087 .Sp
2088 Be aware that the parameter argument to the destructor function differs
2089 from the related \f(CWSAVEDESTRUCTOR_X()\fR in that it MUST be either NULL or
2090 an \f(CW\*(C`SV*\*(C'\fR.
2091 .Sp
2092 Note the \fIend of the current statement\fR may occur much before the
2093 the \fIend of the current pseudo-block\fR. You may wish to look at the
2094 \&\f(CWSAVEDESTRUCTOR_X()\fR macro instead.
2095 .ie n .IP """MORTALDESTRUCTOR_SV(SV *coderef, SV *args)""" 4
2096 .el .IP "\f(CWMORTALDESTRUCTOR_SV(SV *coderef, SV *args)\fR" 4
2097 .IX Item "MORTALDESTRUCTOR_SV(SV *coderef, SV *args)"
2098 At the end of \fIthe current statement\fR the Perl function contained in
2099 \&\f(CW\*(C`coderef\*(C'\fR is called with the arguments provided (if any) in \f(CW\*(C`args\*(C'\fR.
2100 See the documentation for \f(CWmortal_destructor_sv()\fR for details on
2101 the \f(CW\*(C`args\*(C'\fR parameter is handled.
2102 .Sp
2103 Note the \fIend of the current statement\fR may occur much before the
2104 the \fIend of the current pseudo-block\fR. If you wish to call a perl
2105 function at the end of the current pseudo block you should use the
2106 \&\f(CWSAVEDESTRUCTOR_X()\fR API instead, which will require you create a
2107 C wrapper to call the Perl function.
2108 .ie n .IP SAVESTACK_POS() 4
2109 .el .IP \f(CWSAVESTACK_POS()\fR 4
2110 .IX Item "SAVESTACK_POS()"
2111 The current offset on the Perl internal stack (cf. \f(CW\*(C`SP\*(C'\fR) is restored
2112 at the end of \fIpseudo-block\fR.
2113 .PP
2114 The following API list contains functions, thus one needs to
2115 provide pointers to the modifiable data explicitly (either C pointers,
2116 or Perlish \f(CW\*(C`GV *\*(C'\fRs). Where the above macros take \f(CW\*(C`int\*(C'\fR, a similar
2117 function takes \f(CW\*(C`int *\*(C'\fR.
2118 .PP
2119 Other macros above have functions implementing them, but its probably
2120 best to just use the macro, and not those or the ones below.
2121 .ie n .IP """SV* save_scalar(GV *gv)""" 4
2122 .el .IP "\f(CWSV* save_scalar(GV *gv)\fR" 4
2123 .IX Item "SV* save_scalar(GV *gv)"
2124 Equivalent to Perl code \f(CW\*(C`local $gv\*(C'\fR.
2125 .ie n .IP """AV* save_ary(GV *gv)""" 4
2126 .el .IP "\f(CWAV* save_ary(GV *gv)\fR" 4
2127 .IX Item "AV* save_ary(GV *gv)"
2128 .PD 0
2129 .ie n .IP """HV* save_hash(GV *gv)""" 4
2130 .el .IP "\f(CWHV* save_hash(GV *gv)\fR" 4
2131 .IX Item "HV* save_hash(GV *gv)"
2132 .PD
2133 Similar to \f(CW\*(C`save_scalar\*(C'\fR, but localize \f(CW@gv\fR and \f(CW%gv\fR.
2134 .ie n .IP """void save_item(SV *item)""" 4
2135 .el .IP "\f(CWvoid save_item(SV *item)\fR" 4
2136 .IX Item "void save_item(SV *item)"
2137 Duplicates the current value of \f(CW\*(C`SV\*(C'\fR. On the exit from the current
2138 \&\f(CW\*(C`ENTER\*(C'\fR/\f(CW\*(C`LEAVE\*(C'\fR \fIpseudo-block\fR the value of \f(CW\*(C`SV\*(C'\fR will be restored
2139 using the stored value. It doesn't handle magic. Use \f(CW\*(C`save_scalar\*(C'\fR if
2140 magic is affected.
2141 .ie n .IP """SV* save_svref(SV **sptr)""" 4
2142 .el .IP "\f(CWSV* save_svref(SV **sptr)\fR" 4
2143 .IX Item "SV* save_svref(SV **sptr)"
2144 Similar to \f(CW\*(C`save_scalar\*(C'\fR, but will reinstate an \f(CW\*(C`SV *\*(C'\fR.
2145 .ie n .IP """void save_aptr(AV **aptr)""" 4
2146 .el .IP "\f(CWvoid save_aptr(AV **aptr)\fR" 4
2147 .IX Item "void save_aptr(AV **aptr)"
2148 .PD 0
2149 .ie n .IP """void save_hptr(HV **hptr)""" 4
2150 .el .IP "\f(CWvoid save_hptr(HV **hptr)\fR" 4
2151 .IX Item "void save_hptr(HV **hptr)"
2152 .PD
2153 Similar to \f(CW\*(C`save_svref\*(C'\fR, but localize \f(CW\*(C`AV *\*(C'\fR and \f(CW\*(C`HV *\*(C'\fR.
2154 .PP
2155 The \f(CW\*(C`Alias\*(C'\fR module implements localization of the basic types within the
2156 \&\fIcaller's scope\fR. People who are interested in how to localize things in
2157 the containing scope should take a look there too.
2158 .SH Subroutines
2159 .IX Header "Subroutines"
2160 .SS "XSUBs and the Argument Stack"
2161 .IX Subsection "XSUBs and the Argument Stack"
2162 The XSUB mechanism is a simple way for Perl programs to access C subroutines.
2163 An XSUB routine will have a stack that contains the arguments from the Perl
2164 program, and a way to map from the Perl data structures to a C equivalent.
2165 .PP
2166 The stack arguments are accessible through the \f(CWST(n)\fR macro, which returns
2167 the \f(CW\*(C`n\*(C'\fR'th stack argument. Argument 0 is the first argument passed in the
2168 Perl subroutine call. These arguments are \f(CW\*(C`SV*\*(C'\fR, and can be used anywhere
2169 an \f(CW\*(C`SV*\*(C'\fR is used.
2170 .PP
2171 Most of the time, output from the C routine can be handled through use of
2172 the RETVAL and OUTPUT directives. However, there are some cases where the
2173 argument stack is not already long enough to handle all the return values.
2174 An example is the POSIX \fBtzname()\fR call, which takes no arguments, but returns
2175 two, the local time zone's standard and summer time abbreviations.
2176 .PP
2177 To handle this situation, the PPCODE directive is used and the stack is
2178 extended using the macro:
2179 .PP
2180 .Vb 1
2181 \& EXTEND(SP, num);
2182 .Ve
2183 .PP
2184 where \f(CW\*(C`SP\*(C'\fR is the macro that represents the local copy of the stack pointer,
2185 and \f(CW\*(C`num\*(C'\fR is the number of elements the stack should be extended by.
2186 .PP
2187 Now that there is room on the stack, values can be pushed on it using \f(CW\*(C`PUSHs\*(C'\fR
2188 macro. The pushed values will often need to be "mortal" (See
2189 "Reference Counts and Mortality"):
2190 .PP
2191 .Vb 7
2192 \& PUSHs(sv_2mortal(newSViv(an_integer)))
2193 \& PUSHs(sv_2mortal(newSVuv(an_unsigned_integer)))
2194 \& PUSHs(sv_2mortal(newSVnv(a_double)))
2195 \& PUSHs(sv_2mortal(newSVpv("Some String",0)))
2196 \& /* Although the last example is better written as the more
2197 \& * efficient: */
2198 \& PUSHs(newSVpvs_flags("Some String", SVs_TEMP))
2199 .Ve
2200 .PP
2201 And now the Perl program calling \f(CW\*(C`tzname\*(C'\fR, the two values will be assigned
2202 as in:
2203 .PP
2204 .Vb 1
2205 \& ($standard_abbrev, $summer_abbrev) = POSIX::tzname;
2206 .Ve
2207 .PP
2208 An alternate (and possibly simpler) method to pushing values on the stack is
2209 to use the macro:
2210 .PP
2211 .Vb 1
2212 \& XPUSHs(SV*)
2213 .Ve
2214 .PP
2215 This macro automatically adjusts the stack for you, if needed. Thus, you
2216 do not need to call \f(CW\*(C`EXTEND\*(C'\fR to extend the stack.
2217 .PP
2218 Despite their suggestions in earlier versions of this document the macros
2219 \&\f(CW\*(C`(X)PUSH[iunp]\*(C'\fR are \fInot\fR suited to XSUBs which return multiple results.
2220 For that, either stick to the \f(CW\*(C`(X)PUSHs\*(C'\fR macros shown above, or use the new
2221 \&\f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros instead; see "Putting a C value on Perl stack".
2222 .PP
2223 For more information, consult perlxs and perlxstut.
2224 .SS "Autoloading with XSUBs"
2225 .IX Subsection "Autoloading with XSUBs"
2226 If an AUTOLOAD routine is an XSUB, as with Perl subroutines, Perl puts the
2227 fully-qualified name of the autoloaded subroutine in the \f(CW$AUTOLOAD\fR variable
2228 of the XSUB's package.
2229 .PP
2230 But it also puts the same information in certain fields of the XSUB itself:
2231 .PP
2232 .Vb 4
2233 \& HV *stash = CvSTASH(cv);
2234 \& const char *subname = SvPVX(cv);
2235 \& STRLEN name_length = SvCUR(cv); /* in bytes */
2236 \& U32 is_utf8 = SvUTF8(cv);
2237 .Ve
2238 .PP
2239 \&\f(CWSvPVX(cv)\fR contains just the sub name itself, not including the package.
2240 For an AUTOLOAD routine in UNIVERSAL or one of its superclasses,
2241 \&\f(CWCvSTASH(cv)\fR returns NULL during a method call on a nonexistent package.
2242 .PP
2243 \&\fBNote\fR: Setting \f(CW$AUTOLOAD\fR stopped working in 5.6.1, which did not support
2244 XS AUTOLOAD subs at all. Perl 5.8.0 introduced the use of fields in the
2245 XSUB itself. Perl 5.16.0 restored the setting of \f(CW$AUTOLOAD\fR. If you need
2246 to support 5.8\-5.14, use the XSUB's fields.
2247 .SS "Calling Perl Routines from within C Programs"
2248 .IX Subsection "Calling Perl Routines from within C Programs"
2249 There are four routines that can be used to call a Perl subroutine from
2250 within a C program. These four are:
2251 .PP
2252 .Vb 4
2253 \& I32 call_sv(SV*, I32);
2254 \& I32 call_pv(const char*, I32);
2255 \& I32 call_method(const char*, I32);
2256 \& I32 call_argv(const char*, I32, char**);
2257 .Ve
2258 .PP
2259 The routine most often used is \f(CW\*(C`call_sv\*(C'\fR. The \f(CW\*(C`SV*\*(C'\fR argument
2260 contains either the name of the Perl subroutine to be called, or a
2261 reference to the subroutine. The second argument consists of flags
2262 that control the context in which the subroutine is called, whether
2263 or not the subroutine is being passed arguments, how errors should be
2264 trapped, and how to treat return values.
2265 .PP
2266 All four routines return the number of arguments that the subroutine returned
2267 on the Perl stack.
2268 .PP
2269 These routines used to be called \f(CW\*(C`perl_call_sv\*(C'\fR, etc., before Perl v5.6.0,
2270 but those names are now deprecated; macros of the same name are provided for
2271 compatibility.
2272 .PP
2273 When using any of these routines (except \f(CW\*(C`call_argv\*(C'\fR), the programmer
2274 must manipulate the Perl stack. These include the following macros and
2275 functions:
2276 .PP
2277 .Vb 11
2278 \& dSP
2279 \& SP
2280 \& PUSHMARK()
2281 \& PUTBACK
2282 \& SPAGAIN
2283 \& ENTER
2284 \& SAVETMPS
2285 \& FREETMPS
2286 \& LEAVE
2287 \& XPUSH*()
2288 \& POP*()
2289 .Ve
2290 .PP
2291 For a detailed description of calling conventions from C to Perl,
2292 consult perlcall.
2293 .SS "Putting a C value on Perl stack"
2294 .IX Subsection "Putting a C value on Perl stack"
2295 A lot of opcodes (this is an elementary operation in the internal perl
2296 stack machine) put an SV* on the stack. However, as an optimization
2297 the corresponding SV is (usually) not recreated each time. The opcodes
2298 reuse specially assigned SVs (\fItarget\fRs) which are (as a corollary)
2299 not constantly freed/created.
2300 .PP
2301 Each of the targets is created only once (but see
2302 "Scratchpads and recursion" below), and when an opcode needs to put
2303 an integer, a double, or a string on the stack, it just sets the
2304 corresponding parts of its \fItarget\fR and puts the \fItarget\fR on stack.
2305 .PP
2306 The macro to put this target on stack is \f(CW\*(C`PUSHTARG\*(C'\fR, and it is
2307 directly used in some opcodes, as well as indirectly in zillions of
2308 others, which use it via \f(CW\*(C`(X)PUSH[iunp]\*(C'\fR.
2309 .PP
2310 Because the target is reused, you must be careful when pushing multiple
2311 values on the stack. The following code will not do what you think:
2312 .PP
2313 .Vb 2
2314 \& XPUSHi(10);
2315 \& XPUSHi(20);
2316 .Ve
2317 .PP
2318 This translates as "set \f(CW\*(C`TARG\*(C'\fR to 10, push a pointer to \f(CW\*(C`TARG\*(C'\fR onto
2319 the stack; set \f(CW\*(C`TARG\*(C'\fR to 20, push a pointer to \f(CW\*(C`TARG\*(C'\fR onto the stack".
2320 At the end of the operation, the stack does not contain the values 10
2321 and 20, but actually contains two pointers to \f(CW\*(C`TARG\*(C'\fR, which we have set
2322 to 20.
2323 .PP
2324 If you need to push multiple different values then you should either use
2325 the \f(CW\*(C`(X)PUSHs\*(C'\fR macros, or else use the new \f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros,
2326 none of which make use of \f(CW\*(C`TARG\*(C'\fR. The \f(CW\*(C`(X)PUSHs\*(C'\fR macros simply push an
2327 SV* on the stack, which, as noted under "XSUBs and the Argument Stack",
2328 will often need to be "mortal". The new \f(CW\*(C`m(X)PUSH[iunp]\*(C'\fR macros make
2329 this a little easier to achieve by creating a new mortal for you (via
2330 \&\f(CW\*(C`(X)PUSHmortal\*(C'\fR), pushing that onto the stack (extending it if necessary
2331 in the case of the \f(CW\*(C`mXPUSH[iunp]\*(C'\fR macros), and then setting its value.
2332 Thus, instead of writing this to "fix" the example above:
2333 .PP
2334 .Vb 2
2335 \& XPUSHs(sv_2mortal(newSViv(10)))
2336 \& XPUSHs(sv_2mortal(newSViv(20)))
2337 .Ve
2338 .PP
2339 you can simply write:
2340 .PP
2341 .Vb 2
2342 \& mXPUSHi(10)
2343 \& mXPUSHi(20)
2344 .Ve
2345 .PP
2346 On a related note, if you do use \f(CW\*(C`(X)PUSH[iunp]\*(C'\fR, then you're going to
2347 need a \f(CW\*(C`dTARG\*(C'\fR in your variable declarations so that the \f(CW\*(C`*PUSH*\*(C'\fR
2348 macros can make use of the local variable \f(CW\*(C`TARG\*(C'\fR. See also
2349 \&\f(CW\*(C`dTARGET\*(C'\fR and \f(CW\*(C`dXSTARG\*(C'\fR.
2350 .SS Scratchpads
2351 .IX Subsection "Scratchpads"
2352 The question remains on when the SVs which are \fItarget\fRs for opcodes
2353 are created. The answer is that they are created when the current
2354 unit\-\-a subroutine or a file (for opcodes for statements outside of
2355 subroutines)\-\-is compiled. During this time a special anonymous Perl
2356 array is created, which is called a scratchpad for the current unit.
2357 .PP
2358 A scratchpad keeps SVs which are lexicals for the current unit and are
2359 targets for opcodes. A previous version of this document
2360 stated that one can deduce that an SV lives on a scratchpad
2361 by looking on its flags: lexicals have \f(CW\*(C`SVs_PADMY\*(C'\fR set, and
2362 \&\fItarget\fRs have \f(CW\*(C`SVs_PADTMP\*(C'\fR set. But this has never been fully true.
2363 \&\f(CW\*(C`SVs_PADMY\*(C'\fR could be set on a variable that no longer resides in any pad.
2364 While \fItarget\fRs do have \f(CW\*(C`SVs_PADTMP\*(C'\fR set, it can also be set on variables
2365 that have never resided in a pad, but nonetheless act like \fItarget\fRs. As
2366 of perl 5.21.5, the \f(CW\*(C`SVs_PADMY\*(C'\fR flag is no longer used and is defined as
2367 0. \f(CWSvPADMY()\fR now returns true for anything without \f(CW\*(C`SVs_PADTMP\*(C'\fR.
2368 .PP
2369 The correspondence between OPs and \fItarget\fRs is not 1\-to\-1. Different
2370 OPs in the compile tree of the unit can use the same target, if this
2371 would not conflict with the expected life of the temporary.
2372 .SS "Scratchpads and recursion"
2373 .IX Subsection "Scratchpads and recursion"
2374 In fact it is not 100% true that a compiled unit contains a pointer to
2375 the scratchpad AV. In fact it contains a pointer to an AV of
2376 (initially) one element, and this element is the scratchpad AV. Why do
2377 we need an extra level of indirection?
2378 .PP
2379 The answer is \fBrecursion\fR, and maybe \fBthreads\fR. Both
2380 these can create several execution pointers going into the same
2381 subroutine. For the subroutine-child not write over the temporaries
2382 for the subroutine-parent (lifespan of which covers the call to the
2383 child), the parent and the child should have different
2384 scratchpads. (\fIAnd\fR the lexicals should be separate anyway!)
2385 .PP
2386 So each subroutine is born with an array of scratchpads (of length 1).
2387 On each entry to the subroutine it is checked that the current
2388 depth of the recursion is not more than the length of this array, and
2389 if it is, new scratchpad is created and pushed into the array.
2390 .PP
2391 The \fItarget\fRs on this scratchpad are \f(CW\*(C`undef\*(C'\fRs, but they are already
2392 marked with correct flags.
2393 .SH "Memory Allocation"
2394 .IX Header "Memory Allocation"
2395 .SS Allocation
2396 .IX Subsection "Allocation"
2397 All memory meant to be used with the Perl API functions should be manipulated
2398 using the macros described in this section. The macros provide the necessary
2399 transparency between differences in the actual malloc implementation that is
2400 used within perl.
2401 .PP
2402 The following three macros are used to initially allocate memory :
2403 .PP
2404 .Vb 3
2405 \& Newx(pointer, number, type);
2406 \& Newxc(pointer, number, type, cast);
2407 \& Newxz(pointer, number, type);
2408 .Ve
2409 .PP
2410 The first argument \f(CW\*(C`pointer\*(C'\fR should be the name of a variable that will
2411 point to the newly allocated memory.
2412 .PP
2413 The second and third arguments \f(CW\*(C`number\*(C'\fR and \f(CW\*(C`type\*(C'\fR specify how many of
2414 the specified type of data structure should be allocated. The argument
2415 \&\f(CW\*(C`type\*(C'\fR is passed to \f(CW\*(C`sizeof\*(C'\fR. The final argument to \f(CW\*(C`Newxc\*(C'\fR, \f(CW\*(C`cast\*(C'\fR,
2416 should be used if the \f(CW\*(C`pointer\*(C'\fR argument is different from the \f(CW\*(C`type\*(C'\fR
2417 argument.
2418 .PP
2419 Unlike the \f(CW\*(C`Newx\*(C'\fR and \f(CW\*(C`Newxc\*(C'\fR macros, the \f(CW\*(C`Newxz\*(C'\fR macro calls \f(CW\*(C`memzero\*(C'\fR
2420 to zero out all the newly allocated memory.
2421 .SS Reallocation
2422 .IX Subsection "Reallocation"
2423 .Vb 3
2424 \& Renew(pointer, number, type);
2425 \& Renewc(pointer, number, type, cast);
2426 \& Safefree(pointer)
2427 .Ve
2428 .PP
2429 These three macros are used to change a memory buffer size or to free a
2430 piece of memory no longer needed. The arguments to \f(CW\*(C`Renew\*(C'\fR and \f(CW\*(C`Renewc\*(C'\fR
2431 match those of \f(CW\*(C`New\*(C'\fR and \f(CW\*(C`Newc\*(C'\fR with the exception of not needing the
2432 "magic cookie" argument.
2433 .SS Moving
2434 .IX Subsection "Moving"
2435 .Vb 3
2436 \& Move(source, dest, number, type);
2437 \& Copy(source, dest, number, type);
2438 \& Zero(dest, number, type);
2439 .Ve
2440 .PP
2441 These three macros are used to move, copy, or zero out previously allocated
2442 memory. The \f(CW\*(C`source\*(C'\fR and \f(CW\*(C`dest\*(C'\fR arguments point to the source and
2443 destination starting points. Perl will move, copy, or zero out \f(CW\*(C`number\*(C'\fR
2444 instances of the size of the \f(CW\*(C`type\*(C'\fR data structure (using the \f(CW\*(C`sizeof\*(C'\fR
2445 function).
2446 .SH PerlIO
2447 .IX Header "PerlIO"
2448 The most recent development releases of Perl have been experimenting with
2449 removing Perl's dependency on the "normal" standard I/O suite and allowing
2450 other stdio implementations to be used. This involves creating a new
2451 abstraction layer that then calls whichever implementation of stdio Perl
2452 was compiled with. All XSUBs should now use the functions in the PerlIO
2453 abstraction layer and not make any assumptions about what kind of stdio
2454 is being used.
2455 .PP
2456 For a complete description of the PerlIO abstraction, consult perlapio.
2457 .SH "Compiled code"
2458 .IX Header "Compiled code"
2459 .SS "Code tree"
2460 .IX Subsection "Code tree"
2461 Here we describe the internal form your code is converted to by
2462 Perl. Start with a simple example:
2463 .PP
2464 .Vb 1
2465 \& $a = $b + $c;
2466 .Ve
2467 .PP
2468 This is converted to a tree similar to this one:
2469 .PP
2470 .Vb 5
2471 \& assign\-to
2472 \& / \e
2473 \& + $a
2474 \& / \e
2475 \& $b $c
2476 .Ve
2477 .PP
2478 (but slightly more complicated). This tree reflects the way Perl
2479 parsed your code, but has nothing to do with the execution order.
2480 There is an additional "thread" going through the nodes of the tree
2481 which shows the order of execution of the nodes. In our simplified
2482 example above it looks like:
2483 .PP
2484 .Vb 1
2485 \& $b \-\-\-> $c \-\-\-> + \-\-\-> $a \-\-\-> assign\-to
2486 .Ve
2487 .PP
2488 But with the actual compile tree for \f(CW\*(C`$a = $b + $c\*(C'\fR it is different:
2489 some nodes \fIoptimized away\fR. As a corollary, though the actual tree
2490 contains more nodes than our simplified example, the execution order
2491 is the same as in our example.
2492 .SS "Examining the tree"
2493 .IX Subsection "Examining the tree"
2494 If you have your perl compiled for debugging (usually done with
2495 \&\f(CW\*(C`\-DDEBUGGING\*(C'\fR on the \f(CW\*(C`Configure\*(C'\fR command line), you may examine the
2496 compiled tree by specifying \f(CW\*(C`\-Dx\*(C'\fR on the Perl command line. The
2497 output takes several lines per node, and for \f(CW\*(C`$b+$c\*(C'\fR it looks like
2498 this:
2499 .PP
2500 .Vb 10
2501 \& 5 TYPE = add ===> 6
2502 \& TARG = 1
2503 \& FLAGS = (SCALAR,KIDS)
2504 \& {
2505 \& TYPE = null ===> (4)
2506 \& (was rv2sv)
2507 \& FLAGS = (SCALAR,KIDS)
2508 \& {
2509 \& 3 TYPE = gvsv ===> 4
2510 \& FLAGS = (SCALAR)
2511 \& GV = main::b
2512 \& }
2513 \& }
2514 \& {
2515 \& TYPE = null ===> (5)
2516 \& (was rv2sv)
2517 \& FLAGS = (SCALAR,KIDS)
2518 \& {
2519 \& 4 TYPE = gvsv ===> 5
2520 \& FLAGS = (SCALAR)
2521 \& GV = main::c
2522 \& }
2523 \& }
2524 .Ve
2525 .PP
2526 This tree has 5 nodes (one per \f(CW\*(C`TYPE\*(C'\fR specifier), only 3 of them are
2527 not optimized away (one per number in the left column). The immediate
2528 children of the given node correspond to \f(CW\*(C`{}\*(C'\fR pairs on the same level
2529 of indentation, thus this listing corresponds to the tree:
2530 .PP
2531 .Vb 5
2532 \& add
2533 \& / \e
2534 \& null null
2535 \& | |
2536 \& gvsv gvsv
2537 .Ve
2538 .PP
2539 The execution order is indicated by \f(CW\*(C`===>\*(C'\fR marks, thus it is \f(CW\*(C`3
2540 4 5 6\*(C'\fR (node \f(CW6\fR is not included into above listing), i.e.,
2541 \&\f(CW\*(C`gvsv gvsv add whatever\*(C'\fR.
2542 .PP
2543 Each of these nodes represents an op, a fundamental operation inside the
2544 Perl core. The code which implements each operation can be found in the
2545 \&\fIpp*.c\fR files; the function which implements the op with type \f(CW\*(C`gvsv\*(C'\fR
2546 is \f(CW\*(C`pp_gvsv\*(C'\fR, and so on. As the tree above shows, different ops have
2547 different numbers of children: \f(CW\*(C`add\*(C'\fR is a binary operator, as one would
2548 expect, and so has two children. To accommodate the various different
2549 numbers of children, there are various types of op data structure, and
2550 they link together in different ways.
2551 .PP
2552 The simplest type of op structure is \f(CW\*(C`OP\*(C'\fR: this has no children. Unary
2553 operators, \f(CW\*(C`UNOP\*(C'\fRs, have one child, and this is pointed to by the
2554 \&\f(CW\*(C`op_first\*(C'\fR field. Binary operators (\f(CW\*(C`BINOP\*(C'\fRs) have not only an
2555 \&\f(CW\*(C`op_first\*(C'\fR field but also an \f(CW\*(C`op_last\*(C'\fR field. The most complex type of
2556 op is a \f(CW\*(C`LISTOP\*(C'\fR, which has any number of children. In this case, the
2557 first child is pointed to by \f(CW\*(C`op_first\*(C'\fR and the last child by
2558 \&\f(CW\*(C`op_last\*(C'\fR. The children in between can be found by iteratively
2559 following the \f(CW\*(C`OpSIBLING\*(C'\fR pointer from the first child to the last (but
2560 see below).
2561 .PP
2562 There are also some other op types: a \f(CW\*(C`PMOP\*(C'\fR holds a regular expression,
2563 and has no children, and a \f(CW\*(C`LOOP\*(C'\fR may or may not have children. If the
2564 \&\f(CW\*(C`op_children\*(C'\fR field is non-zero, it behaves like a \f(CW\*(C`LISTOP\*(C'\fR. To
2565 complicate matters, if a \f(CW\*(C`UNOP\*(C'\fR is actually a \f(CW\*(C`null\*(C'\fR op after
2566 optimization (see "Compile pass 2: context propagation") it will still
2567 have children in accordance with its former type.
2568 .PP
2569 Finally, there is a \f(CW\*(C`LOGOP\*(C'\fR, or logic op. Like a \f(CW\*(C`LISTOP\*(C'\fR, this has one
2570 or more children, but it doesn't have an \f(CW\*(C`op_last\*(C'\fR field: so you have to
2571 follow \f(CW\*(C`op_first\*(C'\fR and then the \f(CW\*(C`OpSIBLING\*(C'\fR chain itself to find the
2572 last child. Instead it has an \f(CW\*(C`op_other\*(C'\fR field, which is comparable to
2573 the \f(CW\*(C`op_next\*(C'\fR field described below, and represents an alternate
2574 execution path. Operators like \f(CW\*(C`and\*(C'\fR, \f(CW\*(C`or\*(C'\fR and \f(CW\*(C`?\*(C'\fR are \f(CW\*(C`LOGOP\*(C'\fRs. Note
2575 that in general, \f(CW\*(C`op_other\*(C'\fR may not point to any of the direct children
2576 of the \f(CW\*(C`LOGOP\*(C'\fR.
2577 .PP
2578 Starting in version 5.21.2, perls built with the experimental
2579 define \f(CW\*(C`\-DPERL_OP_PARENT\*(C'\fR add an extra boolean flag for each op,
2580 \&\f(CW\*(C`op_moresib\*(C'\fR. When not set, this indicates that this is the last op in an
2581 \&\f(CW\*(C`OpSIBLING\*(C'\fR chain. This frees up the \f(CW\*(C`op_sibling\*(C'\fR field on the last
2582 sibling to point back to the parent op. Under this build, that field is
2583 also renamed \f(CW\*(C`op_sibparent\*(C'\fR to reflect its joint role. The macro
2584 \&\f(CWOpSIBLING(o)\fR wraps this special behaviour, and always returns NULL on
2585 the last sibling. With this build the \f(CWop_parent(o)\fR function can be
2586 used to find the parent of any op. Thus for forward compatibility, you
2587 should always use the \f(CWOpSIBLING(o)\fR macro rather than accessing
2588 \&\f(CW\*(C`op_sibling\*(C'\fR directly.
2589 .PP
2590 Another way to examine the tree is to use a compiler back-end module, such
2591 as B::Concise.
2592 .SS "Compile pass 1: check routines"
2593 .IX Subsection "Compile pass 1: check routines"
2594 The tree is created by the compiler while \fIyacc\fR code feeds it
2595 the constructions it recognizes. Since \fIyacc\fR works bottom-up, so does
2596 the first pass of perl compilation.
2597 .PP
2598 What makes this pass interesting for perl developers is that some
2599 optimization may be performed on this pass. This is optimization by
2600 so-called "check routines". The correspondence between node names
2601 and corresponding check routines is described in \fIopcode.pl\fR (do not
2602 forget to run \f(CW\*(C`make regen_headers\*(C'\fR if you modify this file).
2603 .PP
2604 A check routine is called when the node is fully constructed except
2605 for the execution-order thread. Since at this time there are no
2606 back-links to the currently constructed node, one can do most any
2607 operation to the top-level node, including freeing it and/or creating
2608 new nodes above/below it.
2609 .PP
2610 The check routine returns the node which should be inserted into the
2611 tree (if the top-level node was not modified, check routine returns
2612 its argument).
2613 .PP
2614 By convention, check routines have names \f(CW\*(C`ck_*\*(C'\fR. They are usually
2615 called from \f(CW\*(C`new*OP\*(C'\fR subroutines (or \f(CW\*(C`convert\*(C'\fR) (which in turn are
2616 called from \fIperly.y\fR).
2617 .SS "Compile pass 1a: constant folding"
2618 .IX Subsection "Compile pass 1a: constant folding"
2619 Immediately after the check routine is called the returned node is
2620 checked for being compile-time executable. If it is (the value is
2621 judged to be constant) it is immediately executed, and a \fIconstant\fR
2622 node with the "return value" of the corresponding subtree is
2623 substituted instead. The subtree is deleted.
2624 .PP
2625 If constant folding was not performed, the execution-order thread is
2626 created.
2627 .SS "Compile pass 2: context propagation"
2628 .IX Subsection "Compile pass 2: context propagation"
2629 When a context for a part of compile tree is known, it is propagated
2630 down through the tree. At this time the context can have 5 values
2631 (instead of 2 for runtime context): void, boolean, scalar, list, and
2632 lvalue. In contrast with the pass 1 this pass is processed from top
2633 to bottom: a node's context determines the context for its children.
2634 .PP
2635 Additional context-dependent optimizations are performed at this time.
2636 Since at this moment the compile tree contains back-references (via
2637 "thread" pointers), nodes cannot be \fBfree()\fRd now. To allow
2638 optimized-away nodes at this stage, such nodes are \fBnull()\fRified instead
2639 of \fBfree()\fRing (i.e. their type is changed to OP_NULL).
2640 .SS "Compile pass 3: peephole optimization"
2641 .IX Subsection "Compile pass 3: peephole optimization"
2642 After the compile tree for a subroutine (or for an \f(CW\*(C`eval\*(C'\fR or a file)
2643 is created, an additional pass over the code is performed. This pass
2644 is neither top-down or bottom-up, but in the execution order (with
2645 additional complications for conditionals). Optimizations performed
2646 at this stage are subject to the same restrictions as in the pass 2.
2647 .PP
2648 Peephole optimizations are done by calling the function pointed to
2649 by the global variable \f(CW\*(C`PL_peepp\*(C'\fR. By default, \f(CW\*(C`PL_peepp\*(C'\fR just
2650 calls the function pointed to by the global variable \f(CW\*(C`PL_rpeepp\*(C'\fR.
2651 By default, that performs some basic op fixups and optimisations along
2652 the execution-order op chain, and recursively calls \f(CW\*(C`PL_rpeepp\*(C'\fR for
2653 each side chain of ops (resulting from conditionals). Extensions may
2654 provide additional optimisations or fixups, hooking into either the
2655 per-subroutine or recursive stage, like this:
2656 .PP
2657 .Vb 10
2658 \& static peep_t prev_peepp;
2659 \& static void my_peep(pTHX_ OP *o)
2660 \& {
2661 \& /* custom per\-subroutine optimisation goes here */
2662 \& prev_peepp(aTHX_ o);
2663 \& /* custom per\-subroutine optimisation may also go here */
2664 \& }
2665 \& BOOT:
2666 \& prev_peepp = PL_peepp;
2667 \& PL_peepp = my_peep;
2668 \&
2669 \& static peep_t prev_rpeepp;
2670 \& static void my_rpeep(pTHX_ OP *first)
2671 \& {
2672 \& OP *o = first, *t = first;
2673 \& for(; o = o\->op_next, t = t\->op_next) {
2674 \& /* custom per\-op optimisation goes here */
2675 \& o = o\->op_next;
2676 \& if (!o || o == t) break;
2677 \& /* custom per\-op optimisation goes AND here */
2678 \& }
2679 \& prev_rpeepp(aTHX_ orig_o);
2680 \& }
2681 \& BOOT:
2682 \& prev_rpeepp = PL_rpeepp;
2683 \& PL_rpeepp = my_rpeep;
2684 .Ve
2685 .SS "Pluggable runops"
2686 .IX Subsection "Pluggable runops"
2687 The compile tree is executed in a runops function. There are two runops
2688 functions, in \fIrun.c\fR and in \fIdump.c\fR. \f(CW\*(C`Perl_runops_debug\*(C'\fR is used
2689 with DEBUGGING and \f(CW\*(C`Perl_runops_standard\*(C'\fR is used otherwise. For fine
2690 control over the execution of the compile tree it is possible to provide
2691 your own runops function.
2692 .PP
2693 It's probably best to copy one of the existing runops functions and
2694 change it to suit your needs. Then, in the BOOT section of your XS
2695 file, add the line:
2696 .PP
2697 .Vb 1
2698 \& PL_runops = my_runops;
2699 .Ve
2700 .PP
2701 This function should be as efficient as possible to keep your programs
2702 running as fast as possible.
2703 .SS "Compile-time scope hooks"
2704 .IX Subsection "Compile-time scope hooks"
2705 As of perl 5.14 it is possible to hook into the compile-time lexical
2706 scope mechanism using \f(CW\*(C`Perl_blockhook_register\*(C'\fR. This is used like
2707 this:
2708 .PP
2709 .Vb 2
2710 \& STATIC void my_start_hook(pTHX_ int full);
2711 \& STATIC BHK my_hooks;
2712 \&
2713 \& BOOT:
2714 \& BhkENTRY_set(&my_hooks, bhk_start, my_start_hook);
2715 \& Perl_blockhook_register(aTHX_ &my_hooks);
2716 .Ve
2717 .PP
2718 This will arrange to have \f(CW\*(C`my_start_hook\*(C'\fR called at the start of
2719 compiling every lexical scope. The available hooks are:
2720 .ie n .IP """void bhk_start(pTHX_ int full)""" 4
2721 .el .IP "\f(CWvoid bhk_start(pTHX_ int full)\fR" 4
2722 .IX Item "void bhk_start(pTHX_ int full)"
2723 This is called just after starting a new lexical scope. Note that Perl
2724 code like
2725 .Sp
2726 .Vb 1
2727 \& if ($x) { ... }
2728 .Ve
2729 .Sp
2730 creates two scopes: the first starts at the \f(CW\*(C`(\*(C'\fR and has \f(CW\*(C`full == 1\*(C'\fR,
2731 the second starts at the \f(CW\*(C`{\*(C'\fR and has \f(CW\*(C`full == 0\*(C'\fR. Both end at the
2732 \&\f(CW\*(C`}\*(C'\fR, so calls to \f(CW\*(C`start\*(C'\fR and \f(CW\*(C`pre\*(C'\fR/\f(CW\*(C`post_end\*(C'\fR will match. Anything
2733 pushed onto the save stack by this hook will be popped just before the
2734 scope ends (between the \f(CW\*(C`pre_\*(C'\fR and \f(CW\*(C`post_end\*(C'\fR hooks, in fact).
2735 .ie n .IP """void bhk_pre_end(pTHX_ OP **o)""" 4
2736 .el .IP "\f(CWvoid bhk_pre_end(pTHX_ OP **o)\fR" 4
2737 .IX Item "void bhk_pre_end(pTHX_ OP **o)"
2738 This is called at the end of a lexical scope, just before unwinding the
2739 stack. \fIo\fR is the root of the optree representing the scope; it is a
2740 double pointer so you can replace the OP if you need to.
2741 .ie n .IP """void bhk_post_end(pTHX_ OP **o)""" 4
2742 .el .IP "\f(CWvoid bhk_post_end(pTHX_ OP **o)\fR" 4
2743 .IX Item "void bhk_post_end(pTHX_ OP **o)"
2744 This is called at the end of a lexical scope, just after unwinding the
2745 stack. \fIo\fR is as above. Note that it is possible for calls to \f(CW\*(C`pre_\*(C'\fR
2746 and \f(CW\*(C`post_end\*(C'\fR to nest, if there is something on the save stack that
2747 calls string eval.
2748 .ie n .IP """void bhk_eval(pTHX_ OP *const o)""" 4
2749 .el .IP "\f(CWvoid bhk_eval(pTHX_ OP *const o)\fR" 4
2750 .IX Item "void bhk_eval(pTHX_ OP *const o)"
2751 This is called just before starting to compile an \f(CW\*(C`eval STRING\*(C'\fR, \f(CW\*(C`do
2752 FILE\*(C'\fR, \f(CW\*(C`require\*(C'\fR or \f(CW\*(C`use\*(C'\fR, after the eval has been set up. \fIo\fR is the
2753 OP that requested the eval, and will normally be an \f(CW\*(C`OP_ENTEREVAL\*(C'\fR,
2754 \&\f(CW\*(C`OP_DOFILE\*(C'\fR or \f(CW\*(C`OP_REQUIRE\*(C'\fR.
2755 .PP
2756 Once you have your hook functions, you need a \f(CW\*(C`BHK\*(C'\fR structure to put
2757 them in. It's best to allocate it statically, since there is no way to
2758 free it once it's registered. The function pointers should be inserted
2759 into this structure using the \f(CW\*(C`BhkENTRY_set\*(C'\fR macro, which will also set
2760 flags indicating which entries are valid. If you do need to allocate
2761 your \f(CW\*(C`BHK\*(C'\fR dynamically for some reason, be sure to zero it before you
2762 start.
2763 .PP
2764 Once registered, there is no mechanism to switch these hooks off, so if
2765 that is necessary you will need to do this yourself. An entry in \f(CW\*(C`%^H\*(C'\fR
2766 is probably the best way, so the effect is lexically scoped; however it
2767 is also possible to use the \f(CW\*(C`BhkDISABLE\*(C'\fR and \f(CW\*(C`BhkENABLE\*(C'\fR macros to
2768 temporarily switch entries on and off. You should also be aware that
2769 generally speaking at least one scope will have opened before your
2770 extension is loaded, so you will see some \f(CW\*(C`pre\*(C'\fR/\f(CW\*(C`post_end\*(C'\fR pairs that
2771 didn't have a matching \f(CW\*(C`start\*(C'\fR.
2772 .ie n .SH "Examining internal data structures with the ""dump"" functions"
2773 .el .SH "Examining internal data structures with the \f(CWdump\fP functions"
2774 .IX Header "Examining internal data structures with the dump functions"
2775 To aid debugging, the source file \fIdump.c\fR contains a number of
2776 functions which produce formatted output of internal data structures.
2777 .PP
2778 The most commonly used of these functions is \f(CW\*(C`Perl_sv_dump\*(C'\fR; it's used
2779 for dumping SVs, AVs, HVs, and CVs. The \f(CW\*(C`Devel::Peek\*(C'\fR module calls
2780 \&\f(CW\*(C`sv_dump\*(C'\fR to produce debugging output from Perl-space, so users of that
2781 module should already be familiar with its format.
2782 .PP
2783 \&\f(CW\*(C`Perl_op_dump\*(C'\fR can be used to dump an \f(CW\*(C`OP\*(C'\fR structure or any of its
2784 derivatives, and produces output similar to \f(CW\*(C`perl \-Dx\*(C'\fR; in fact,
2785 \&\f(CW\*(C`Perl_dump_eval\*(C'\fR will dump the main root of the code being evaluated,
2786 exactly like \f(CW\*(C`\-Dx\*(C'\fR.
2787 .PP
2788 Other useful functions are \f(CW\*(C`Perl_dump_sub\*(C'\fR, which turns a \f(CW\*(C`GV\*(C'\fR into an
2789 op tree, \f(CW\*(C`Perl_dump_packsubs\*(C'\fR which calls \f(CW\*(C`Perl_dump_sub\*(C'\fR on all the
2790 subroutines in a package like so: (Thankfully, these are all xsubs, so
2791 there is no op tree)
2792 .PP
2793 .Vb 1
2794 \& (gdb) print Perl_dump_packsubs(PL_defstash)
2795 \&
2796 \& SUB attributes::bootstrap = (xsub 0x811fedc 0)
2797 \&
2798 \& SUB UNIVERSAL::can = (xsub 0x811f50c 0)
2799 \&
2800 \& SUB UNIVERSAL::isa = (xsub 0x811f304 0)
2801 \&
2802 \& SUB UNIVERSAL::VERSION = (xsub 0x811f7ac 0)
2803 \&
2804 \& SUB DynaLoader::boot_DynaLoader = (xsub 0x805b188 0)
2805 .Ve
2806 .PP
2807 and \f(CW\*(C`Perl_dump_all\*(C'\fR, which dumps all the subroutines in the stash and
2808 the op tree of the main root.
2809 .SH "How multiple interpreters and concurrency are supported"
2810 .IX Header "How multiple interpreters and concurrency are supported"
2811 .SS "Background and MULTIPLICITY"
2812 .IX Subsection "Background and MULTIPLICITY"
2813 The Perl interpreter can be regarded as a closed box: it has an API
2814 for feeding it code or otherwise making it do things, but it also has
2815 functions for its own use. This smells a lot like an object, and
2816 there is a way for you to build Perl so that you can have multiple
2817 interpreters, with one interpreter represented either as a C structure,
2818 or inside a thread-specific structure. These structures contain all
2819 the context, the state of that interpreter.
2820 .PP
2821 The macro that controls the major Perl build flavor is MULTIPLICITY. The
2822 MULTIPLICITY build has a C structure that packages all the interpreter
2823 state, which is being passed to various perl functions as a "hidden"
2824 first argument. MULTIPLICITY makes multi-threaded perls possible (with the
2825 ithreads threading model, related to the macro USE_ITHREADS.)
2826 .PP
2827 PERL_IMPLICIT_CONTEXT is a legacy synonym for MULTIPLICITY.
2828 .PP
2829 To see whether you have non-const data you can use a BSD (or GNU)
2830 compatible \f(CW\*(C`nm\*(C'\fR:
2831 .PP
2832 .Vb 1
2833 \& nm libperl.a | grep \-v \*(Aq [TURtr] \*(Aq
2834 .Ve
2835 .PP
2836 If this displays any \f(CW\*(C`D\*(C'\fR or \f(CW\*(C`d\*(C'\fR symbols (or possibly \f(CW\*(C`C\*(C'\fR or \f(CW\*(C`c\*(C'\fR),
2837 you have non-const data. The symbols the \f(CW\*(C`grep\*(C'\fR removed are as follows:
2838 \&\f(CW\*(C`Tt\*(C'\fR are \fItext\fR, or code, the \f(CW\*(C`Rr\*(C'\fR are \fIread-only\fR (const) data,
2839 and the \f(CW\*(C`U\*(C'\fR is <undefined>, external symbols referred to.
2840 .PP
2841 The test \fIt/porting/libperl.t\fR does this kind of symbol sanity
2842 checking on \f(CW\*(C`libperl.a\*(C'\fR.
2843 .PP
2844 All this obviously requires a way for the Perl internal functions to be
2845 either subroutines taking some kind of structure as the first
2846 argument, or subroutines taking nothing as the first argument. To
2847 enable these two very different ways of building the interpreter,
2848 the Perl source (as it does in so many other situations) makes heavy
2849 use of macros and subroutine naming conventions.
2850 .PP
2851 First problem: deciding which functions will be public API functions and
2852 which will be private. All functions whose names begin \f(CW\*(C`S_\*(C'\fR are private
2853 (think "S" for "secret" or "static"). All other functions begin with
2854 "Perl_", but just because a function begins with "Perl_" does not mean it is
2855 part of the API. (See "Internal
2856 Functions".) The easiest way to be \fBsure\fR a
2857 function is part of the API is to find its entry in perlapi.
2858 If it exists in perlapi, it's part of the API. If it doesn't, and you
2859 think it should be (i.e., you need it for your extension), submit an issue at
2860 <https://github.com/Perl/perl5/issues> explaining why you think it should be.
2861 .PP
2862 Second problem: there must be a syntax so that the same subroutine
2863 declarations and calls can pass a structure as their first argument,
2864 or pass nothing. To solve this, the subroutines are named and
2865 declared in a particular way. Here's a typical start of a static
2866 function used within the Perl guts:
2867 .PP
2868 .Vb 2
2869 \& STATIC void
2870 \& S_incline(pTHX_ char *s)
2871 .Ve
2872 .PP
2873 STATIC becomes "static" in C, and may be #define'd to nothing in some
2874 configurations in the future.
2875 .PP
2876 A public function (i.e. part of the internal API, but not necessarily
2877 sanctioned for use in extensions) begins like this:
2878 .PP
2879 .Vb 2
2880 \& void
2881 \& Perl_sv_setiv(pTHX_ SV* dsv, IV num)
2882 .Ve
2883 .PP
2884 \&\f(CW\*(C`pTHX_\*(C'\fR is one of a number of macros (in \fIperl.h\fR) that hide the
2885 details of the interpreter's context. THX stands for "thread", "this",
2886 or "thingy", as the case may be. (And no, George Lucas is not involved. :\-)
2887 The first character could be 'p' for a \fBp\fRrototype, 'a' for \fBa\fRrgument,
2888 or 'd' for \fBd\fReclaration, so we have \f(CW\*(C`pTHX\*(C'\fR, \f(CW\*(C`aTHX\*(C'\fR and \f(CW\*(C`dTHX\*(C'\fR, and
2889 their variants.
2890 .PP
2891 When Perl is built without options that set MULTIPLICITY, there is no
2892 first argument containing the interpreter's context. The trailing underscore
2893 in the pTHX_ macro indicates that the macro expansion needs a comma
2894 after the context argument because other arguments follow it. If
2895 MULTIPLICITY is not defined, pTHX_ will be ignored, and the
2896 subroutine is not prototyped to take the extra argument. The form of the
2897 macro without the trailing underscore is used when there are no additional
2898 explicit arguments.
2899 .PP
2900 When a core function calls another, it must pass the context. This
2901 is normally hidden via macros. Consider \f(CW\*(C`sv_setiv\*(C'\fR. It expands into
2902 something like this:
2903 .PP
2904 .Vb 6
2905 \& #ifdef MULTIPLICITY
2906 \& #define sv_setiv(a,b) Perl_sv_setiv(aTHX_ a, b)
2907 \& /* can\*(Aqt do this for vararg functions, see below */
2908 \& #else
2909 \& #define sv_setiv Perl_sv_setiv
2910 \& #endif
2911 .Ve
2912 .PP
2913 This works well, and means that XS authors can gleefully write:
2914 .PP
2915 .Vb 1
2916 \& sv_setiv(foo, bar);
2917 .Ve
2918 .PP
2919 and still have it work under all the modes Perl could have been
2920 compiled with.
2921 .PP
2922 This doesn't work so cleanly for varargs functions, though, as macros
2923 imply that the number of arguments is known in advance. Instead we
2924 either need to spell them out fully, passing \f(CW\*(C`aTHX_\*(C'\fR as the first
2925 argument (the Perl core tends to do this with functions like
2926 Perl_warner), or use a context-free version.
2927 .PP
2928 The context-free version of Perl_warner is called
2929 Perl_warner_nocontext, and does not take the extra argument. Instead
2930 it does \f(CW\*(C`dTHX;\*(C'\fR to get the context from thread-local storage. We
2931 \&\f(CW\*(C`#define warner Perl_warner_nocontext\*(C'\fR so that extensions get source
2932 compatibility at the expense of performance. (Passing an arg is
2933 cheaper than grabbing it from thread-local storage.)
2934 .PP
2935 You can ignore [pad]THXx when browsing the Perl headers/sources.
2936 Those are strictly for use within the core. Extensions and embedders
2937 need only be aware of [pad]THX.
2938 .SS "So what happened to dTHR?"
2939 .IX Subsection "So what happened to dTHR?"
2940 \&\f(CW\*(C`dTHR\*(C'\fR was introduced in perl 5.005 to support the older thread model.
2941 The older thread model now uses the \f(CW\*(C`THX\*(C'\fR mechanism to pass context
2942 pointers around, so \f(CW\*(C`dTHR\*(C'\fR is not useful any more. Perl 5.6.0 and
2943 later still have it for backward source compatibility, but it is defined
2944 to be a no-op.
2945 .SS "How do I use all this in extensions?"
2946 .IX Subsection "How do I use all this in extensions?"
2947 See also "Dealing with embedded perls and threads" in perlclib.
2948 .PP
2949 When Perl is built with MULTIPLICITY, extensions that call
2950 any functions in the Perl API will need to pass the initial context
2951 argument somehow. The kicker is that you will need to write it in
2952 such a way that the extension still compiles when Perl hasn't been
2953 built with MULTIPLICITY enabled.
2954 .PP
2955 There are three ways to do this. First, the easy but inefficient way,
2956 which is also the default, in order to maintain source compatibility
2957 with extensions: whenever \fIXSUB.h\fR is #included, it redefines the aTHX
2958 and aTHX_ macros to call a function that will return the context.
2959 Thus, something like:
2960 .PP
2961 .Vb 1
2962 \& sv_setiv(sv, num);
2963 .Ve
2964 .PP
2965 in your extension will translate to this when MULTIPLICITY is
2966 in effect:
2967 .PP
2968 .Vb 1
2969 \& Perl_sv_setiv(Perl_get_context(), sv, num);
2970 .Ve
2971 .PP
2972 or to this otherwise:
2973 .PP
2974 .Vb 1
2975 \& Perl_sv_setiv(sv, num);
2976 .Ve
2977 .PP
2978 You don't have to do anything new in your extension to get this; since
2979 the Perl library provides \fBPerl_get_context()\fR, it will all just
2980 work.
2981 .PP
2982 The second, more efficient way is to use the following template for
2983 your Foo.xs:
2984 .PP
2985 .Vb 4
2986 \& #define PERL_NO_GET_CONTEXT /* we want efficiency */
2987 \& #include "EXTERN.h"
2988 \& #include "perl.h"
2989 \& #include "XSUB.h"
2990 \&
2991 \& STATIC void my_private_function(int arg1, int arg2);
2992 \&
2993 \& STATIC void
2994 \& my_private_function(int arg1, int arg2)
2995 \& {
2996 \& dTHX; /* fetch context */
2997 \& ... call many Perl API functions ...
2998 \& }
2999 \&
3000 \& [... etc ...]
3001 \&
3002 \& MODULE = Foo PACKAGE = Foo
3003 \&
3004 \& /* typical XSUB */
3005 \&
3006 \& void
3007 \& my_xsub(arg)
3008 \& int arg
3009 \& CODE:
3010 \& my_private_function(arg, 10);
3011 .Ve
3012 .PP
3013 Note that the only two changes from the normal way of writing an
3014 extension is the addition of a \f(CW\*(C`#define PERL_NO_GET_CONTEXT\*(C'\fR before
3015 including the Perl headers, followed by a \f(CW\*(C`dTHX;\*(C'\fR declaration at
3016 the start of every function that will call the Perl API. (You'll
3017 know which functions need this, because the C compiler will complain
3018 that there's an undeclared identifier in those functions.) No changes
3019 are needed for the XSUBs themselves, because the \fBXS()\fR macro is
3020 correctly defined to pass in the implicit context if needed.
3021 .PP
3022 The third, even more efficient way is to ape how it is done within
3023 the Perl guts:
3024 .PP
3025 .Vb 4
3026 \& #define PERL_NO_GET_CONTEXT /* we want efficiency */
3027 \& #include "EXTERN.h"
3028 \& #include "perl.h"
3029 \& #include "XSUB.h"
3030 \&
3031 \& /* pTHX_ only needed for functions that call Perl API */
3032 \& STATIC void my_private_function(pTHX_ int arg1, int arg2);
3033 \&
3034 \& STATIC void
3035 \& my_private_function(pTHX_ int arg1, int arg2)
3036 \& {
3037 \& /* dTHX; not needed here, because THX is an argument */
3038 \& ... call Perl API functions ...
3039 \& }
3040 \&
3041 \& [... etc ...]
3042 \&
3043 \& MODULE = Foo PACKAGE = Foo
3044 \&
3045 \& /* typical XSUB */
3046 \&
3047 \& void
3048 \& my_xsub(arg)
3049 \& int arg
3050 \& CODE:
3051 \& my_private_function(aTHX_ arg, 10);
3052 .Ve
3053 .PP
3054 This implementation never has to fetch the context using a function
3055 call, since it is always passed as an extra argument. Depending on
3056 your needs for simplicity or efficiency, you may mix the previous
3057 two approaches freely.
3058 .PP
3059 Never add a comma after \f(CW\*(C`pTHX\*(C'\fR yourself\-\-always use the form of the
3060 macro with the underscore for functions that take explicit arguments,
3061 or the form without the argument for functions with no explicit arguments.
3062 .SS "Should I do anything special if I call perl from multiple threads?"
3063 .IX Subsection "Should I do anything special if I call perl from multiple threads?"
3064 If you create interpreters in one thread and then proceed to call them in
3065 another, you need to make sure perl's own Thread Local Storage (TLS) slot is
3066 initialized correctly in each of those threads.
3067 .PP
3068 The \f(CW\*(C`perl_alloc\*(C'\fR and \f(CW\*(C`perl_clone\*(C'\fR API functions will automatically set
3069 the TLS slot to the interpreter they created, so that there is no need to do
3070 anything special if the interpreter is always accessed in the same thread that
3071 created it, and that thread did not create or call any other interpreters
3072 afterwards. If that is not the case, you have to set the TLS slot of the
3073 thread before calling any functions in the Perl API on that particular
3074 interpreter. This is done by calling the \f(CW\*(C`PERL_SET_CONTEXT\*(C'\fR macro in that
3075 thread as the first thing you do:
3076 .PP
3077 .Vb 2
3078 \& /* do this before doing anything else with some_perl */
3079 \& PERL_SET_CONTEXT(some_perl);
3080 \&
3081 \& ... other Perl API calls on some_perl go here ...
3082 .Ve
3083 .PP
3084 (You can always get the current context via \f(CW\*(C`PERL_GET_CONTEXT\*(C'\fR.)
3085 .SS "Future Plans and PERL_IMPLICIT_SYS"
3086 .IX Subsection "Future Plans and PERL_IMPLICIT_SYS"
3087 Just as MULTIPLICITY provides a way to bundle up everything
3088 that the interpreter knows about itself and pass it around, so too are
3089 there plans to allow the interpreter to bundle up everything it knows
3090 about the environment it's running on. This is enabled with the
3091 PERL_IMPLICIT_SYS macro. Currently it only works with USE_ITHREADS on
3092 Windows.
3093 .PP
3094 This allows the ability to provide an extra pointer (called the "host"
3095 environment) for all the system calls. This makes it possible for
3096 all the system stuff to maintain their own state, broken down into
3097 seven C structures. These are thin wrappers around the usual system
3098 calls (see \fIwin32/perllib.c\fR) for the default perl executable, but for a
3099 more ambitious host (like the one that would do \fBfork()\fR emulation) all
3100 the extra work needed to pretend that different interpreters are
3101 actually different "processes", would be done here.
3102 .PP
3103 The Perl engine/interpreter and the host are orthogonal entities.
3104 There could be one or more interpreters in a process, and one or
3105 more "hosts", with free association between them.
3106 .SH "Internal Functions"
3107 .IX Header "Internal Functions"
3108 All of Perl's internal functions which will be exposed to the outside
3109 world are prefixed by \f(CW\*(C`Perl_\*(C'\fR so that they will not conflict with XS
3110 functions or functions used in a program in which Perl is embedded.
3111 Similarly, all global variables begin with \f(CW\*(C`PL_\*(C'\fR. (By convention,
3112 static functions start with \f(CW\*(C`S_\*(C'\fR.)
3113 .PP
3114 Inside the Perl core (\f(CW\*(C`PERL_CORE\*(C'\fR defined), you can get at the functions
3115 either with or without the \f(CW\*(C`Perl_\*(C'\fR prefix, thanks to a bunch of defines
3116 that live in \fIembed.h\fR. Note that extension code should \fInot\fR set
3117 \&\f(CW\*(C`PERL_CORE\*(C'\fR; this exposes the full perl internals, and is likely to cause
3118 breakage of the XS in each new perl release.
3119 .PP
3120 The file \fIembed.h\fR is generated automatically from
3121 \&\fIembed.pl\fR and \fIembed.fnc\fR. \fIembed.pl\fR also creates the prototyping
3122 header files for the internal functions, generates the documentation
3123 and a lot of other bits and pieces. It's important that when you add
3124 a new function to the core or change an existing one, you change the
3125 data in the table in \fIembed.fnc\fR as well. Here's a sample entry from
3126 that table:
3127 .PP
3128 .Vb 1
3129 \& Apd |SV** |av_fetch |AV* ar|I32 key|I32 lval
3130 .Ve
3131 .PP
3132 The first column is a set of flags, the second column the return type,
3133 the third column the name. Columns after that are the arguments.
3134 The flags are documented at the top of \fIembed.fnc\fR.
3135 .PP
3136 If you edit \fIembed.pl\fR or \fIembed.fnc\fR, you will need to run
3137 \&\f(CW\*(C`make regen_headers\*(C'\fR to force a rebuild of \fIembed.h\fR and other
3138 auto-generated files.
3139 .SS "Formatted Printing of IVs, UVs, and NVs"
3140 .IX Subsection "Formatted Printing of IVs, UVs, and NVs"
3141 If you are printing IVs, UVs, or NVS instead of the \fBstdio\fR\|(3) style
3142 formatting codes like \f(CW%d\fR, \f(CW%ld\fR, \f(CW%f\fR, you should use the
3143 following macros for portability
3144 .PP
3145 .Vb 7
3146 \& IVdf IV in decimal
3147 \& UVuf UV in decimal
3148 \& UVof UV in octal
3149 \& UVxf UV in hexadecimal
3150 \& NVef NV %e\-like
3151 \& NVff NV %f\-like
3152 \& NVgf NV %g\-like
3153 .Ve
3154 .PP
3155 These will take care of 64\-bit integers and long doubles.
3156 For example:
3157 .PP
3158 .Vb 1
3159 \& printf("IV is %" IVdf "\en", iv);
3160 .Ve
3161 .PP
3162 The \f(CW\*(C`IVdf\*(C'\fR will expand to whatever is the correct format for the IVs.
3163 Note that the spaces are required around the format in case the code is
3164 compiled with C++, to maintain compliance with its standard.
3165 .PP
3166 Note that there are different "long doubles": Perl will use
3167 whatever the compiler has.
3168 .PP
3169 If you are printing addresses of pointers, use \f(CW%p\fR or UVxf combined
3170 with \fBPTR2UV()\fR.
3171 .SS "Formatted Printing of SVs"
3172 .IX Subsection "Formatted Printing of SVs"
3173 The contents of SVs may be printed using the \f(CW\*(C`SVf\*(C'\fR format, like so:
3174 .PP
3175 .Vb 1
3176 \& Perl_croak(aTHX_ "This croaked because: %" SVf "\en", SVfARG(err_msg))
3177 .Ve
3178 .PP
3179 where \f(CW\*(C`err_msg\*(C'\fR is an SV.
3180 .PP
3181 Not all scalar types are printable. Simple values certainly are: one of
3182 IV, UV, NV, or PV. Also, if the SV is a reference to some value,
3183 either it will be dereferenced and the value printed, or information
3184 about the type of that value and its address are displayed. The results
3185 of printing any other type of SV are undefined and likely to lead to an
3186 interpreter crash. NVs are printed using a \f(CW%g\fR\-ish format.
3187 .PP
3188 Note that the spaces are required around the \f(CW\*(C`SVf\*(C'\fR in case the code is
3189 compiled with C++, to maintain compliance with its standard.
3190 .PP
3191 Note that any filehandle being printed to under UTF\-8 must be expecting
3192 UTF\-8 in order to get good results and avoid Wide-character warnings.
3193 One way to do this for typical filehandles is to invoke perl with the
3194 \&\f(CW\*(C`\-C\*(C'\fR parameter. (See "\-C [number/list]" in perlrun.
3195 .PP
3196 You can use this to concatenate two scalars:
3197 .PP
3198 .Vb 4
3199 \& SV *var1 = get_sv("var1", GV_ADD);
3200 \& SV *var2 = get_sv("var2", GV_ADD);
3201 \& SV *var3 = newSVpvf("var1=%" SVf " and var2=%" SVf,
3202 \& SVfARG(var1), SVfARG(var2));
3203 .Ve
3204 .PP
3205 \&\f(CW\*(C`SVf_QUOTEDPREFIX\*(C'\fR is similar to \f(CW\*(C`SVf\*(C'\fR except that it restricts the
3206 number of the characters printed, showing at most the first
3207 \&\f(CW\*(C`PERL_QUOTEDPREFIX_LEN\*(C'\fR characters of the argument, and rendering it with
3208 double quotes and with the contents escaped using double quoted string
3209 escaping rules. If the string is longer than this then ellipses "..."
3210 will be appended after the trailing quote. This is intended for error
3211 messages where the string is assumed to be a class name.
3212 .PP
3213 \&\f(CW\*(C`HvNAMEf\*(C'\fR and \f(CW\*(C`HvNAMEf_QUOTEDPREFIX\*(C'\fR are similar to \f(CW\*(C`SVf\*(C'\fR except they
3214 extract the string, length and utf8 flags from the argument using the
3215 \&\f(CWHvNAME()\fR, \f(CWHvNAMELEN()\fR, \f(CWHvNAMEUTF8()\fR macros. This is intended
3216 for stringifying a class name directly from an stash HV.
3217 .SS "Formatted Printing of Strings"
3218 .IX Subsection "Formatted Printing of Strings"
3219 If you just want the bytes printed in a 7bit NUL-terminated string, you can
3220 just use \f(CW%s\fR (assuming they are all really only 7bit). But if there is a
3221 possibility the value will be encoded as UTF\-8 or contains bytes above
3222 \&\f(CW0x7F\fR (and therefore 8bit), you should instead use the \f(CW\*(C`UTF8f\*(C'\fR format.
3223 And as its parameter, use the \f(CWUTF8fARG()\fR macro:
3224 .PP
3225 .Vb 1
3226 \& chr * msg;
3227 \&
3228 \& /* U+2018: \exE2\ex80\ex98 LEFT SINGLE QUOTATION MARK
3229 \& U+2019: \exE2\ex80\ex99 RIGHT SINGLE QUOTATION MARK */
3230 \& if (can_utf8)
3231 \& msg = "\exE2\ex80\ex98Uses fancy quotes\exE2\ex80\ex99";
3232 \& else
3233 \& msg = "\*(AqUses simple quotes\*(Aq";
3234 \&
3235 \& Perl_croak(aTHX_ "The message is: %" UTF8f "\en",
3236 \& UTF8fARG(can_utf8, strlen(msg), msg));
3237 .Ve
3238 .PP
3239 The first parameter to \f(CW\*(C`UTF8fARG\*(C'\fR is a boolean: 1 if the string is in
3240 UTF\-8; 0 if string is in native byte encoding (Latin1).
3241 The second parameter is the number of bytes in the string to print.
3242 And the third and final parameter is a pointer to the first byte in the
3243 string.
3244 .PP
3245 Note that any filehandle being printed to under UTF\-8 must be expecting
3246 UTF\-8 in order to get good results and avoid Wide-character warnings.
3247 One way to do this for typical filehandles is to invoke perl with the
3248 \&\f(CW\*(C`\-C\*(C'\fR parameter. (See "\-C [number/list]" in perlrun.
3249 .ie n .SS "Formatted Printing of ""Size_t"" and ""SSize_t"""
3250 .el .SS "Formatted Printing of \f(CWSize_t\fP and \f(CWSSize_t\fP"
3251 .IX Subsection "Formatted Printing of Size_t and SSize_t"
3252 The most general way to do this is to cast them to a UV or IV, and
3253 print as in the
3254 previous section.
3255 .PP
3256 But if you're using \f(CWPerlIO_printf()\fR, it's less typing and visual
3257 clutter to use the \f(CW%z\fR length modifier (for \fIsiZe\fR):
3258 .PP
3259 .Vb 1
3260 \& PerlIO_printf("STRLEN is %zu\en", len);
3261 .Ve
3262 .PP
3263 This modifier is not portable, so its use should be restricted to
3264 \&\f(CWPerlIO_printf()\fR.
3265 .ie n .SS "Formatted Printing of ""Ptrdiff_t"", ""intmax_t"", ""short"" and other special sizes"
3266 .el .SS "Formatted Printing of \f(CWPtrdiff_t\fP, \f(CWintmax_t\fP, \f(CWshort\fP and other special sizes"
3267 .IX Subsection "Formatted Printing of Ptrdiff_t, intmax_t, short and other special sizes"
3268 There are modifiers for these special situations if you are using
3269 \&\f(CWPerlIO_printf()\fR. See "size" in perlfunc.
3270 .SS "Pointer-To-Integer and Integer-To-Pointer"
3271 .IX Subsection "Pointer-To-Integer and Integer-To-Pointer"
3272 Because pointer size does not necessarily equal integer size,
3273 use the follow macros to do it right.
3274 .PP
3275 .Vb 4
3276 \& PTR2UV(pointer)
3277 \& PTR2IV(pointer)
3278 \& PTR2NV(pointer)
3279 \& INT2PTR(pointertotype, integer)
3280 .Ve
3281 .PP
3282 For example:
3283 .PP
3284 .Vb 2
3285 \& IV iv = ...;
3286 \& SV *sv = INT2PTR(SV*, iv);
3287 .Ve
3288 .PP
3289 and
3290 .PP
3291 .Vb 2
3292 \& AV *av = ...;
3293 \& UV uv = PTR2UV(av);
3294 .Ve
3295 .PP
3296 There are also
3297 .PP
3298 .Vb 2
3299 \& PTR2nat(pointer) /* pointer to integer of PTRSIZE */
3300 \& PTR2ul(pointer) /* pointer to unsigned long */
3301 .Ve
3302 .PP
3303 And \f(CW\*(C`PTRV\*(C'\fR which gives the native type for an integer the same size as
3304 pointers, such as \f(CW\*(C`unsigned\*(C'\fR or \f(CW\*(C`unsigned long\*(C'\fR.
3305 .SS "Exception Handling"
3306 .IX Subsection "Exception Handling"
3307 There are a couple of macros to do very basic exception handling in XS
3308 modules. You have to define \f(CW\*(C`NO_XSLOCKS\*(C'\fR before including \fIXSUB.h\fR to
3309 be able to use these macros:
3310 .PP
3311 .Vb 2
3312 \& #define NO_XSLOCKS
3313 \& #include "XSUB.h"
3314 .Ve
3315 .PP
3316 You can use these macros if you call code that may croak, but you need
3317 to do some cleanup before giving control back to Perl. For example:
3318 .PP
3319 .Vb 1
3320 \& dXCPT; /* set up necessary variables */
3321 \&
3322 \& XCPT_TRY_START {
3323 \& code_that_may_croak();
3324 \& } XCPT_TRY_END
3325 \&
3326 \& XCPT_CATCH
3327 \& {
3328 \& /* do cleanup here */
3329 \& XCPT_RETHROW;
3330 \& }
3331 .Ve
3332 .PP
3333 Note that you always have to rethrow an exception that has been
3334 caught. Using these macros, it is not possible to just catch the
3335 exception and ignore it. If you have to ignore the exception, you
3336 have to use the \f(CW\*(C`call_*\*(C'\fR function.
3337 .PP
3338 The advantage of using the above macros is that you don't have
3339 to setup an extra function for \f(CW\*(C`call_*\*(C'\fR, and that using these
3340 macros is faster than using \f(CW\*(C`call_*\*(C'\fR.
3341 .SS "Source Documentation"
3342 .IX Subsection "Source Documentation"
3343 There's an effort going on to document the internal functions and
3344 automatically produce reference manuals from them \-\- perlapi is one
3345 such manual which details all the functions which are available to XS
3346 writers. perlintern is the autogenerated manual for the functions
3347 which are not part of the API and are supposedly for internal use only.
3348 .PP
3349 Source documentation is created by putting POD comments into the C
3350 source, like this:
3351 .PP
3352 .Vb 2
3353 \& /*
3354 \& =for apidoc sv_setiv
3355 \&
3356 \& Copies an integer into the given SV. Does not handle \*(Aqset\*(Aq magic. See
3357 \& L<perlapi/sv_setiv_mg>.
3358 \&
3359 \& =cut
3360 \& */
3361 .Ve
3362 .PP
3363 Please try and supply some documentation if you add functions to the
3364 Perl core.
3365 .SS "Backwards compatibility"
3366 .IX Subsection "Backwards compatibility"
3367 The Perl API changes over time. New functions are
3368 added or the interfaces of existing functions are
3369 changed. The \f(CW\*(C`Devel::PPPort\*(C'\fR module tries to
3370 provide compatibility code for some of these changes, so XS writers don't
3371 have to code it themselves when supporting multiple versions of Perl.
3372 .PP
3373 \&\f(CW\*(C`Devel::PPPort\*(C'\fR generates a C header file \fIppport.h\fR that can also
3374 be run as a Perl script. To generate \fIppport.h\fR, run:
3375 .PP
3376 .Vb 1
3377 \& perl \-MDevel::PPPort \-eDevel::PPPort::WriteFile
3378 .Ve
3379 .PP
3380 Besides checking existing XS code, the script can also be used to retrieve
3381 compatibility information for various API calls using the \f(CW\*(C`\-\-api\-info\*(C'\fR
3382 command line switch. For example:
3383 .PP
3384 .Vb 1
3385 \& % perl ppport.h \-\-api\-info=sv_magicext
3386 .Ve
3387 .PP
3388 For details, see \f(CW\*(C`perldoc\ ppport.h\*(C'\fR.
3389 .SH "Unicode Support"
3390 .IX Header "Unicode Support"
3391 Perl 5.6.0 introduced Unicode support. It's important for porters and XS
3392 writers to understand this support and make sure that the code they
3393 write does not corrupt Unicode data.
3394 .SS "What \fBis\fP Unicode, anyway?"
3395 .IX Subsection "What is Unicode, anyway?"
3396 In the olden, less enlightened times, we all used to use ASCII. Most of
3397 us did, anyway. The big problem with ASCII is that it's American. Well,
3398 no, that's not actually the problem; the problem is that it's not
3399 particularly useful for people who don't use the Roman alphabet. What
3400 used to happen was that particular languages would stick their own
3401 alphabet in the upper range of the sequence, between 128 and 255. Of
3402 course, we then ended up with plenty of variants that weren't quite
3403 ASCII, and the whole point of it being a standard was lost.
3404 .PP
3405 Worse still, if you've got a language like Chinese or
3406 Japanese that has hundreds or thousands of characters, then you really
3407 can't fit them into a mere 256, so they had to forget about ASCII
3408 altogether, and build their own systems using pairs of numbers to refer
3409 to one character.
3410 .PP
3411 To fix this, some people formed Unicode, Inc. and
3412 produced a new character set containing all the characters you can
3413 possibly think of and more. There are several ways of representing these
3414 characters, and the one Perl uses is called UTF\-8. UTF\-8 uses
3415 a variable number of bytes to represent a character. You can learn more
3416 about Unicode and Perl's Unicode model in perlunicode.
3417 .PP
3418 (On EBCDIC platforms, Perl uses instead UTF-EBCDIC, which is a form of
3419 UTF\-8 adapted for EBCDIC platforms. Below, we just talk about UTF\-8.
3420 UTF-EBCDIC is like UTF\-8, but the details are different. The macros
3421 hide the differences from you, just remember that the particular numbers
3422 and bit patterns presented below will differ in UTF-EBCDIC.)
3423 .SS "How can I recognise a UTF\-8 string?"
3424 .IX Subsection "How can I recognise a UTF-8 string?"
3425 You can't. This is because UTF\-8 data is stored in bytes just like
3426 non\-UTF\-8 data. The Unicode character 200, (\f(CW0xC8\fR for you hex types)
3427 capital E with a grave accent, is represented by the two bytes
3428 \&\f(CW\*(C`v196.172\*(C'\fR. Unfortunately, the non-Unicode string \f(CW\*(C`chr(196).chr(172)\*(C'\fR
3429 has that byte sequence as well. So you can't tell just by looking \-\- this
3430 is what makes Unicode input an interesting problem.
3431 .PP
3432 In general, you either have to know what you're dealing with, or you
3433 have to guess. The API function \f(CW\*(C`is_utf8_string\*(C'\fR can help; it'll tell
3434 you if a string contains only valid UTF\-8 characters, and the chances
3435 of a non\-UTF\-8 string looking like valid UTF\-8 become very small very
3436 quickly with increasing string length. On a character-by-character
3437 basis, \f(CW\*(C`isUTF8_CHAR\*(C'\fR
3438 will tell you whether the current character in a string is valid UTF\-8.
3439 .SS "How does UTF\-8 represent Unicode characters?"
3440 .IX Subsection "How does UTF-8 represent Unicode characters?"
3441 As mentioned above, UTF\-8 uses a variable number of bytes to store a
3442 character. Characters with values 0...127 are stored in one
3443 byte, just like good ol' ASCII. Character 128 is stored as
3444 \&\f(CW\*(C`v194.128\*(C'\fR; this continues up to character 191, which is
3445 \&\f(CW\*(C`v194.191\*(C'\fR. Now we've run out of bits (191 is binary
3446 \&\f(CW10111111\fR) so we move on; character 192 is \f(CW\*(C`v195.128\*(C'\fR. And
3447 so it goes on, moving to three bytes at character 2048.
3448 "Unicode Encodings" in perlunicode has pictures of how this works.
3449 .PP
3450 Assuming you know you're dealing with a UTF\-8 string, you can find out
3451 how long the first character in it is with the \f(CW\*(C`UTF8SKIP\*(C'\fR macro:
3452 .PP
3453 .Vb 2
3454 \& char *utf = "\e305\e233\e340\e240\e201";
3455 \& I32 len;
3456 \&
3457 \& len = UTF8SKIP(utf); /* len is 2 here */
3458 \& utf += len;
3459 \& len = UTF8SKIP(utf); /* len is 3 here */
3460 .Ve
3461 .PP
3462 Another way to skip over characters in a UTF\-8 string is to use
3463 \&\f(CW\*(C`utf8_hop\*(C'\fR, which takes a string and a number of characters to skip
3464 over. You're on your own about bounds checking, though, so don't use it
3465 lightly.
3466 .PP
3467 All bytes in a multi-byte UTF\-8 character will have the high bit set,
3468 so you can test if you need to do something special with this
3469 character like this (the \f(CWUTF8_IS_INVARIANT()\fR is a macro that tests
3470 whether the byte is encoded as a single byte even in UTF\-8):
3471 .PP
3472 .Vb 7
3473 \& U8 *utf; /* Initialize this to point to the beginning of the
3474 \& sequence to convert */
3475 \& U8 *utf_end; /* Initialize this to 1 beyond the end of the sequence
3476 \& pointed to by \*(Aqutf\*(Aq */
3477 \& UV uv; /* Returned code point; note: a UV, not a U8, not a
3478 \& char */
3479 \& STRLEN len; /* Returned length of character in bytes */
3480 \&
3481 \& if (!UTF8_IS_INVARIANT(*utf))
3482 \& /* Must treat this as UTF\-8 */
3483 \& uv = utf8_to_uvchr_buf(utf, utf_end, &len);
3484 \& else
3485 \& /* OK to treat this character as a byte */
3486 \& uv = *utf;
3487 .Ve
3488 .PP
3489 You can also see in that example that we use \f(CW\*(C`utf8_to_uvchr_buf\*(C'\fR to get the
3490 value of the character; the inverse function \f(CW\*(C`uvchr_to_utf8\*(C'\fR is available
3491 for putting a UV into UTF\-8:
3492 .PP
3493 .Vb 6
3494 \& if (!UVCHR_IS_INVARIANT(uv))
3495 \& /* Must treat this as UTF8 */
3496 \& utf8 = uvchr_to_utf8(utf8, uv);
3497 \& else
3498 \& /* OK to treat this character as a byte */
3499 \& *utf8++ = uv;
3500 .Ve
3501 .PP
3502 You \fBmust\fR convert characters to UVs using the above functions if
3503 you're ever in a situation where you have to match UTF\-8 and non\-UTF\-8
3504 characters. You may not skip over UTF\-8 characters in this case. If you
3505 do this, you'll lose the ability to match hi-bit non\-UTF\-8 characters;
3506 for instance, if your UTF\-8 string contains \f(CW\*(C`v196.172\*(C'\fR, and you skip
3507 that character, you can never match a \f(CWchr(200)\fR in a non\-UTF\-8 string.
3508 So don't do that!
3509 .PP
3510 (Note that we don't have to test for invariant characters in the
3511 examples above. The functions work on any well-formed UTF\-8 input.
3512 It's just that its faster to avoid the function overhead when it's not
3513 needed.)
3514 .SS "How does Perl store UTF\-8 strings?"
3515 .IX Subsection "How does Perl store UTF-8 strings?"
3516 Currently, Perl deals with UTF\-8 strings and non\-UTF\-8 strings
3517 slightly differently. A flag in the SV, \f(CW\*(C`SVf_UTF8\*(C'\fR, indicates that the
3518 string is internally encoded as UTF\-8. Without it, the byte value is the
3519 codepoint number and vice versa. This flag is only meaningful if the SV
3520 is \f(CW\*(C`SvPOK\*(C'\fR or immediately after stringification via \f(CW\*(C`SvPV\*(C'\fR or a
3521 similar macro. You can check and manipulate this flag with the
3522 following macros:
3523 .PP
3524 .Vb 3
3525 \& SvUTF8(sv)
3526 \& SvUTF8_on(sv)
3527 \& SvUTF8_off(sv)
3528 .Ve
3529 .PP
3530 This flag has an important effect on Perl's treatment of the string: if
3531 UTF\-8 data is not properly distinguished, regular expressions,
3532 \&\f(CW\*(C`length\*(C'\fR, \f(CW\*(C`substr\*(C'\fR and other string handling operations will have
3533 undesirable (wrong) results.
3534 .PP
3535 The problem comes when you have, for instance, a string that isn't
3536 flagged as UTF\-8, and contains a byte sequence that could be UTF\-8 \-\-
3537 especially when combining non\-UTF\-8 and UTF\-8 strings.
3538 .PP
3539 Never forget that the \f(CW\*(C`SVf_UTF8\*(C'\fR flag is separate from the PV value; you
3540 need to be sure you don't accidentally knock it off while you're
3541 manipulating SVs. More specifically, you cannot expect to do this:
3542 .PP
3543 .Vb 4
3544 \& SV *sv;
3545 \& SV *nsv;
3546 \& STRLEN len;
3547 \& char *p;
3548 \&
3549 \& p = SvPV(sv, len);
3550 \& frobnicate(p);
3551 \& nsv = newSVpvn(p, len);
3552 .Ve
3553 .PP
3554 The \f(CW\*(C`char*\*(C'\fR string does not tell you the whole story, and you can't
3555 copy or reconstruct an SV just by copying the string value. Check if the
3556 old SV has the UTF8 flag set (\fIafter\fR the \f(CW\*(C`SvPV\*(C'\fR call), and act
3557 accordingly:
3558 .PP
3559 .Vb 6
3560 \& p = SvPV(sv, len);
3561 \& is_utf8 = SvUTF8(sv);
3562 \& frobnicate(p, is_utf8);
3563 \& nsv = newSVpvn(p, len);
3564 \& if (is_utf8)
3565 \& SvUTF8_on(nsv);
3566 .Ve
3567 .PP
3568 In the above, your \f(CW\*(C`frobnicate\*(C'\fR function has been changed to be made
3569 aware of whether or not it's dealing with UTF\-8 data, so that it can
3570 handle the string appropriately.
3571 .PP
3572 Since just passing an SV to an XS function and copying the data of
3573 the SV is not enough to copy the UTF8 flags, even less right is just
3574 passing a \f(CW\*(C`char\ *\*(C'\fR to an XS function.
3575 .PP
3576 For full generality, use the \f(CW\*(C`DO_UTF8\*(C'\fR macro to see if the
3577 string in an SV is to be \fItreated\fR as UTF\-8. This takes into account
3578 if the call to the XS function is being made from within the scope of
3579 \&\f(CW\*(C`use\ bytes\*(C'\fR. If so, the underlying bytes that comprise the
3580 UTF\-8 string are to be exposed, rather than the character they
3581 represent. But this pragma should only really be used for debugging and
3582 perhaps low-level testing at the byte level. Hence most XS code need
3583 not concern itself with this, but various areas of the perl core do need
3584 to support it.
3585 .PP
3586 And this isn't the whole story. Starting in Perl v5.12, strings that
3587 aren't encoded in UTF\-8 may also be treated as Unicode under various
3588 conditions (see "ASCII Rules versus Unicode Rules" in perlunicode).
3589 This is only really a problem for characters whose ordinals are between
3590 128 and 255, and their behavior varies under ASCII versus Unicode rules
3591 in ways that your code cares about (see "The "Unicode Bug"" in perlunicode).
3592 There is no published API for dealing with this, as it is subject to
3593 change, but you can look at the code for \f(CW\*(C`pp_lc\*(C'\fR in \fIpp.c\fR for an
3594 example as to how it's currently done.
3595 .SS "How do I pass a Perl string to a C library?"
3596 .IX Subsection "How do I pass a Perl string to a C library?"
3597 A Perl string, conceptually, is an opaque sequence of code points.
3598 Many C libraries expect their inputs to be "classical" C strings, which are
3599 arrays of octets 1\-255, terminated with a NUL byte. Your job when writing
3600 an interface between Perl and a C library is to define the mapping between
3601 Perl and that library.
3602 .PP
3603 Generally speaking, \f(CW\*(C`SvPVbyte\*(C'\fR and related macros suit this task well.
3604 These assume that your Perl string is a "byte string", i.e., is either
3605 raw, undecoded input into Perl or is pre-encoded to, e.g., UTF\-8.
3606 .PP
3607 Alternatively, if your C library expects UTF\-8 text, you can use
3608 \&\f(CW\*(C`SvPVutf8\*(C'\fR and related macros. This has the same effect as encoding
3609 to UTF\-8 then calling the corresponding \f(CW\*(C`SvPVbyte\*(C'\fR\-related macro.
3610 .PP
3611 Some C libraries may expect other encodings (e.g., UTF\-16LE). To give
3612 Perl strings to such libraries
3613 you must either do that encoding in Perl then use \f(CW\*(C`SvPVbyte\*(C'\fR, or
3614 use an intermediary C library to convert from however Perl stores the
3615 string to the desired encoding.
3616 .PP
3617 Take care also that NULs in your Perl string don't confuse the C
3618 library. If possible, give the string's length to the C library; if that's
3619 not possible, consider rejecting strings that contain NUL bytes.
3620 .PP
3621 \fIWhat about \fR\f(CI\*(C`SvPV\*(C'\fR\fI, \fR\f(CI\*(C`SvPV_nolen\*(C'\fR\fI, etc.?\fR
3622 .IX Subsection "What about SvPV, SvPV_nolen, etc.?"
3623 .PP
3624 Consider a 3\-character Perl string \f(CW\*(C`$foo = "\ex64\ex78\ex8c"\*(C'\fR.
3625 Perl can store these 3 characters either of two ways:
3626 .IP \(bu 4
3627 bytes: 0x64 0x78 0x8c
3628 .IP \(bu 4
3629 UTF\-8: 0x64 0x78 0xc2 0x8c
3630 .PP
3631 Now let's say you convert \f(CW$foo\fR to a C string thus:
3632 .PP
3633 .Vb 2
3634 \& STRLEN strlen;
3635 \& char *str = SvPV(foo_sv, strlen);
3636 .Ve
3637 .PP
3638 At this point \f(CW\*(C`str\*(C'\fR could point to a 3\-byte C string or a 4\-byte one.
3639 .PP
3640 Generally speaking, we want \f(CW\*(C`str\*(C'\fR to be the same regardless of how
3641 Perl stores \f(CW$foo\fR, so the ambiguity here is undesirable. \f(CW\*(C`SvPVbyte\*(C'\fR
3642 and \f(CW\*(C`SvPVutf8\*(C'\fR solve that by giving predictable output: use
3643 \&\f(CW\*(C`SvPVbyte\*(C'\fR if your C library expects byte strings, or \f(CW\*(C`SvPVutf8\*(C'\fR
3644 if it expects UTF\-8.
3645 .PP
3646 If your C library happens to support both encodings, then \f(CW\*(C`SvPV\*(C'\fR\-\-always
3647 in tandem with lookups to \f(CW\*(C`SvUTF8\*(C'\fR!\-\-may be safe and (slightly) more
3648 efficient.
3649 .PP
3650 \&\fBTESTING\fR \fBTIP:\fR Use utf8's \f(CW\*(C`upgrade\*(C'\fR and \f(CW\*(C`downgrade\*(C'\fR functions
3651 in your tests to ensure consistent handling regardless of Perl's
3652 internal encoding.
3653 .SS "How do I convert a string to UTF\-8?"
3654 .IX Subsection "How do I convert a string to UTF-8?"
3655 If you're mixing UTF\-8 and non\-UTF\-8 strings, it is necessary to upgrade
3656 the non\-UTF\-8 strings to UTF\-8. If you've got an SV, the easiest way to do
3657 this is:
3658 .PP
3659 .Vb 1
3660 \& sv_utf8_upgrade(sv);
3661 .Ve
3662 .PP
3663 However, you must not do this, for example:
3664 .PP
3665 .Vb 2
3666 \& if (!SvUTF8(left))
3667 \& sv_utf8_upgrade(left);
3668 .Ve
3669 .PP
3670 If you do this in a binary operator, you will actually change one of the
3671 strings that came into the operator, and, while it shouldn't be noticeable
3672 by the end user, it can cause problems in deficient code.
3673 .PP
3674 Instead, \f(CW\*(C`bytes_to_utf8\*(C'\fR will give you a UTF\-8\-encoded \fBcopy\fR of its
3675 string argument. This is useful for having the data available for
3676 comparisons and so on, without harming the original SV. There's also
3677 \&\f(CW\*(C`utf8_to_bytes\*(C'\fR to go the other way, but naturally, this will fail if
3678 the string contains any characters above 255 that can't be represented
3679 in a single byte.
3680 .SS "How do I compare strings?"
3681 .IX Subsection "How do I compare strings?"
3682 "sv_cmp" in perlapi and "sv_cmp_flags" in perlapi do a lexigraphic
3683 comparison of two SV's, and handle UTF\-8ness properly. Note, however,
3684 that Unicode specifies a much fancier mechanism for collation, available
3685 via the Unicode::Collate module.
3686 .PP
3687 To just compare two strings for equality/non\-equality, you can just use
3688 \&\f(CWmemEQ()\fR and \f(CWmemNE()\fR as usual,
3689 except the strings must be both UTF\-8 or not UTF\-8 encoded.
3690 .PP
3691 To compare two strings case-insensitively, use
3692 \&\f(CWfoldEQ_utf8()\fR (the strings don't have to have
3693 the same UTF\-8ness).
3694 .SS "Is there anything else I need to know?"
3695 .IX Subsection "Is there anything else I need to know?"
3696 Not really. Just remember these things:
3697 .IP \(bu 3
3698 There's no way to tell if a \f(CW\*(C`char\ *\*(C'\fR or \f(CW\*(C`U8\ *\*(C'\fR string is UTF\-8
3699 or not. But you can tell if an SV is to be treated as UTF\-8 by calling
3700 \&\f(CW\*(C`DO_UTF8\*(C'\fR on it, after stringifying it with \f(CW\*(C`SvPV\*(C'\fR or a similar
3701 macro. And, you can tell if SV is actually UTF\-8 (even if it is not to
3702 be treated as such) by looking at its \f(CW\*(C`SvUTF8\*(C'\fR flag (again after
3703 stringifying it). Don't forget to set the flag if something should be
3704 UTF\-8.
3705 Treat the flag as part of the PV, even though it's not \-\- if you pass on
3706 the PV to somewhere, pass on the flag too.
3707 .IP \(bu 3
3708 If a string is UTF\-8, \fBalways\fR use \f(CW\*(C`utf8_to_uvchr_buf\*(C'\fR to get at the value,
3709 unless \f(CWUTF8_IS_INVARIANT(*s)\fR in which case you can use \f(CW*s\fR.
3710 .IP \(bu 3
3711 When writing a character UV to a UTF\-8 string, \fBalways\fR use
3712 \&\f(CW\*(C`uvchr_to_utf8\*(C'\fR, unless \f(CW\*(C`UVCHR_IS_INVARIANT(uv))\*(C'\fR in which case
3713 you can use \f(CW\*(C`*s = uv\*(C'\fR.
3714 .IP \(bu 3
3715 Mixing UTF\-8 and non\-UTF\-8 strings is
3716 tricky. Use \f(CW\*(C`bytes_to_utf8\*(C'\fR to get
3717 a new string which is UTF\-8 encoded, and then combine them.
3718 .SH "Custom Operators"
3719 .IX Header "Custom Operators"
3720 Custom operator support is an experimental feature that allows you to
3721 define your own ops. This is primarily to allow the building of
3722 interpreters for other languages in the Perl core, but it also allows
3723 optimizations through the creation of "macro-ops" (ops which perform the
3724 functions of multiple ops which are usually executed together, such as
3725 \&\f(CW\*(C`gvsv, gvsv, add\*(C'\fR.)
3726 .PP
3727 This feature is implemented as a new op type, \f(CW\*(C`OP_CUSTOM\*(C'\fR. The Perl
3728 core does not "know" anything special about this op type, and so it will
3729 not be involved in any optimizations. This also means that you can
3730 define your custom ops to be any op structure \-\- unary, binary, list and
3731 so on \-\- you like.
3732 .PP
3733 It's important to know what custom operators won't do for you. They
3734 won't let you add new syntax to Perl, directly. They won't even let you
3735 add new keywords, directly. In fact, they won't change the way Perl
3736 compiles a program at all. You have to do those changes yourself, after
3737 Perl has compiled the program. You do this either by manipulating the op
3738 tree using a \f(CW\*(C`CHECK\*(C'\fR block and the \f(CW\*(C`B::Generate\*(C'\fR module, or by adding
3739 a custom peephole optimizer with the \f(CW\*(C`optimize\*(C'\fR module.
3740 .PP
3741 When you do this, you replace ordinary Perl ops with custom ops by
3742 creating ops with the type \f(CW\*(C`OP_CUSTOM\*(C'\fR and the \f(CW\*(C`op_ppaddr\*(C'\fR of your own
3743 PP function. This should be defined in XS code, and should look like
3744 the PP ops in \f(CW\*(C`pp_*.c\*(C'\fR. You are responsible for ensuring that your op
3745 takes the appropriate number of values from the stack, and you are
3746 responsible for adding stack marks if necessary.
3747 .PP
3748 You should also "register" your op with the Perl interpreter so that it
3749 can produce sensible error and warning messages. Since it is possible to
3750 have multiple custom ops within the one "logical" op type \f(CW\*(C`OP_CUSTOM\*(C'\fR,
3751 Perl uses the value of \f(CW\*(C`o\->op_ppaddr\*(C'\fR to determine which custom op
3752 it is dealing with. You should create an \f(CW\*(C`XOP\*(C'\fR structure for each
3753 ppaddr you use, set the properties of the custom op with
3754 \&\f(CW\*(C`XopENTRY_set\*(C'\fR, and register the structure against the ppaddr using
3755 \&\f(CW\*(C`Perl_custom_op_register\*(C'\fR. A trivial example might look like:
3756 .PP
3757 .Vb 2
3758 \& static XOP my_xop;
3759 \& static OP *my_pp(pTHX);
3760 \&
3761 \& BOOT:
3762 \& XopENTRY_set(&my_xop, xop_name, "myxop");
3763 \& XopENTRY_set(&my_xop, xop_desc, "Useless custom op");
3764 \& Perl_custom_op_register(aTHX_ my_pp, &my_xop);
3765 .Ve
3766 .PP
3767 The available fields in the structure are:
3768 .IP xop_name 4
3769 .IX Item "xop_name"
3770 A short name for your op. This will be included in some error messages,
3771 and will also be returned as \f(CW\*(C`$op\->name\*(C'\fR by the B module, so
3772 it will appear in the output of module like B::Concise.
3773 .IP xop_desc 4
3774 .IX Item "xop_desc"
3775 A short description of the function of the op.
3776 .IP xop_class 4
3777 .IX Item "xop_class"
3778 Which of the various \f(CW*OP\fR structures this op uses. This should be one of
3779 the \f(CW\*(C`OA_*\*(C'\fR constants from \fIop.h\fR, namely
3780 .RS 4
3781 .IP OA_BASEOP 4
3782 .IX Item "OA_BASEOP"
3783 .PD 0
3784 .IP OA_UNOP 4
3785 .IX Item "OA_UNOP"
3786 .IP OA_BINOP 4
3787 .IX Item "OA_BINOP"
3788 .IP OA_LOGOP 4
3789 .IX Item "OA_LOGOP"
3790 .IP OA_LISTOP 4
3791 .IX Item "OA_LISTOP"
3792 .IP OA_PMOP 4
3793 .IX Item "OA_PMOP"
3794 .IP OA_SVOP 4
3795 .IX Item "OA_SVOP"
3796 .IP OA_PADOP 4
3797 .IX Item "OA_PADOP"
3798 .IP OA_PVOP_OR_SVOP 4
3799 .IX Item "OA_PVOP_OR_SVOP"
3800 .PD
3801 This should be interpreted as '\f(CW\*(C`PVOP\*(C'\fR' only. The \f(CW\*(C`_OR_SVOP\*(C'\fR is because
3802 the only core \f(CW\*(C`PVOP\*(C'\fR, \f(CW\*(C`OP_TRANS\*(C'\fR, can sometimes be a \f(CW\*(C`SVOP\*(C'\fR instead.
3803 .IP OA_LOOP 4
3804 .IX Item "OA_LOOP"
3805 .PD 0
3806 .IP OA_COP 4
3807 .IX Item "OA_COP"
3808 .RE
3809 .RS 4
3810 .PD
3811 .Sp
3812 The other \f(CW\*(C`OA_*\*(C'\fR constants should not be used.
3813 .RE
3814 .IP xop_peep 4
3815 .IX Item "xop_peep"
3816 This member is of type \f(CW\*(C`Perl_cpeep_t\*(C'\fR, which expands to \f(CW\*(C`void
3817 (*Perl_cpeep_t)(aTHX_ OP *o, OP *oldop)\*(C'\fR. If it is set, this function
3818 will be called from \f(CW\*(C`Perl_rpeep\*(C'\fR when ops of this type are encountered
3819 by the peephole optimizer. \fIo\fR is the OP that needs optimizing;
3820 \&\fIoldop\fR is the previous OP optimized, whose \f(CW\*(C`op_next\*(C'\fR points to \fIo\fR.
3821 .PP
3822 \&\f(CW\*(C`B::Generate\*(C'\fR directly supports the creation of custom ops by name.
3823 .SH Stacks
3824 .IX Header "Stacks"
3825 Descriptions above occasionally refer to "the stack", but there are in fact
3826 many stack-like data structures within the perl interpreter. When otherwise
3827 unqualified, "the stack" usually refers to the value stack.
3828 .PP
3829 The various stacks have different purposes, and operate in slightly different
3830 ways. Their differences are noted below.
3831 .SS "Value Stack"
3832 .IX Subsection "Value Stack"
3833 This stack stores the values that regular perl code is operating on, usually
3834 intermediate values of expressions within a statement. The stack itself is
3835 formed of an array of SV pointers.
3836 .PP
3837 The base of this stack is pointed to by the interpreter variable
3838 \&\f(CW\*(C`PL_stack_base\*(C'\fR, of type \f(CW\*(C`SV **\*(C'\fR.
3839 .PP
3840 The head of the stack is \f(CW\*(C`PL_stack_sp\*(C'\fR, and points to the most
3841 recently-pushed item.
3842 .PP
3843 Items are pushed to the stack by using the \f(CWPUSHs()\fR macro or its variants
3844 described above; \f(CWXPUSHs()\fR, \f(CWmPUSHs()\fR, \f(CWmXPUSHs()\fR and the typed
3845 versions. Note carefully that the non\-\f(CW\*(C`X\*(C'\fR versions of these macros do not
3846 check the size of the stack and assume it to be big enough. These must be
3847 paired with a suitable check of the stack's size, such as the \f(CW\*(C`EXTEND\*(C'\fR macro
3848 to ensure it is large enough. For example
3849 .PP
3850 .Vb 5
3851 \& EXTEND(SP, 4);
3852 \& mPUSHi(10);
3853 \& mPUSHi(20);
3854 \& mPUSHi(30);
3855 \& mPUSHi(40);
3856 .Ve
3857 .PP
3858 This is slightly more performant than making four separate checks in four
3859 separate \f(CWmXPUSHi()\fR calls.
3860 .PP
3861 As a further performance optimisation, the various \f(CW\*(C`PUSH\*(C'\fR macros all operate
3862 using a local variable \f(CW\*(C`SP\*(C'\fR, rather than the interpreter-global variable
3863 \&\f(CW\*(C`PL_stack_sp\*(C'\fR. This variable is declared by the \f(CW\*(C`dSP\*(C'\fR macro \- though it is
3864 normally implied by XSUBs and similar so it is rare you have to consider it
3865 directly. Once declared, the \f(CW\*(C`PUSH\*(C'\fR macros will operate only on this local
3866 variable, so before invoking any other perl core functions you must use the
3867 \&\f(CW\*(C`PUTBACK\*(C'\fR macro to return the value from the local \f(CW\*(C`SP\*(C'\fR variable back to
3868 the interpreter variable. Similarly, after calling a perl core function which
3869 may have had reason to move the stack or push/pop values to it, you must use
3870 the \f(CW\*(C`SPAGAIN\*(C'\fR macro which refreshes the local \f(CW\*(C`SP\*(C'\fR value back from the
3871 interpreter one.
3872 .PP
3873 Items are popped from the stack by using the \f(CW\*(C`POPs\*(C'\fR macro or its typed
3874 versions, There is also a macro \f(CW\*(C`TOPs\*(C'\fR that inspects the topmost item without
3875 removing it.
3876 .PP
3877 Note specifically that SV pointers on the value stack do not contribute to the
3878 overall reference count of the xVs being referred to. If newly-created xVs are
3879 being pushed to the stack you must arrange for them to be destroyed at a
3880 suitable time; usually by using one of the \f(CW\*(C`mPUSH*\*(C'\fR macros or \f(CWsv_2mortal()\fR
3881 to mortalise the xV.
3882 .SS "Mark Stack"
3883 .IX Subsection "Mark Stack"
3884 The value stack stores individual perl scalar values as temporaries between
3885 expressions. Some perl expressions operate on entire lists; for that purpose
3886 we need to know where on the stack each list begins. This is the purpose of the
3887 mark stack.
3888 .PP
3889 The mark stack stores integers as I32 values, which are the height of the
3890 value stack at the time before the list began; thus the mark itself actually
3891 points to the value stack entry one before the list. The list itself starts at
3892 \&\f(CW\*(C`mark + 1\*(C'\fR.
3893 .PP
3894 The base of this stack is pointed to by the interpreter variable
3895 \&\f(CW\*(C`PL_markstack\*(C'\fR, of type \f(CW\*(C`I32 *\*(C'\fR.
3896 .PP
3897 The head of the stack is \f(CW\*(C`PL_markstack_ptr\*(C'\fR, and points to the most
3898 recently-pushed item.
3899 .PP
3900 Items are pushed to the stack by using the \f(CWPUSHMARK()\fR macro. Even though
3901 the stack itself stores (value) stack indices as integers, the \f(CW\*(C`PUSHMARK\*(C'\fR
3902 macro should be given a stack pointer directly; it will calculate the index
3903 offset by comparing to the \f(CW\*(C`PL_stack_sp\*(C'\fR variable. Thus almost always the
3904 code to perform this is
3905 .PP
3906 .Vb 1
3907 \& PUSHMARK(SP);
3908 .Ve
3909 .PP
3910 Items are popped from the stack by the \f(CW\*(C`POPMARK\*(C'\fR macro. There is also a macro
3911 \&\f(CW\*(C`TOPMARK\*(C'\fR that inspects the topmost item without removing it. These macros
3912 return I32 index values directly. There is also the \f(CW\*(C`dMARK\*(C'\fR macro which
3913 declares a new SV double-pointer variable, called \f(CW\*(C`mark\*(C'\fR, which points at the
3914 marked stack slot; this is the usual macro that C code will use when operating
3915 on lists given on the stack.
3916 .PP
3917 As noted above, the \f(CW\*(C`mark\*(C'\fR variable itself will point at the most recently
3918 pushed value on the value stack before the list begins, and so the list itself
3919 starts at \f(CW\*(C`mark + 1\*(C'\fR. The values of the list may be iterated by code such as
3920 .PP
3921 .Vb 4
3922 \& for(SV **svp = mark + 1; svp <= PL_stack_sp; svp++) {
3923 \& SV *item = *svp;
3924 \& ...
3925 \& }
3926 .Ve
3927 .PP
3928 Note specifically in the case that the list is already empty, \f(CW\*(C`mark\*(C'\fR will
3929 equal \f(CW\*(C`PL_stack_sp\*(C'\fR.
3930 .PP
3931 Because the \f(CW\*(C`mark\*(C'\fR variable is converted to a pointer on the value stack,
3932 extra care must be taken if \f(CW\*(C`EXTEND\*(C'\fR or any of the \f(CW\*(C`XPUSH\*(C'\fR macros are
3933 invoked within the function, because the stack may need to be moved to
3934 extend it and so the existing pointer will now be invalid. If this may be a
3935 problem, a possible solution is to track the mark offset as an integer and
3936 track the mark itself later on after the stack had been moved.
3937 .PP
3938 .Vb 1
3939 \& I32 markoff = POPMARK;
3940 \&
3941 \& ...
3942 \&
3943 \& SP **mark = PL_stack_base + markoff;
3944 .Ve
3945 .SS "Temporaries Stack"
3946 .IX Subsection "Temporaries Stack"
3947 As noted above, xV references on the main value stack do not contribute to the
3948 reference count of an xV, and so another mechanism is used to track when
3949 temporary values which live on the stack must be released. This is the job of
3950 the temporaries stack.
3951 .PP
3952 The temporaries stack stores pointers to xVs whose reference counts will be
3953 decremented soon.
3954 .PP
3955 The base of this stack is pointed to by the interpreter variable
3956 \&\f(CW\*(C`PL_tmps_stack\*(C'\fR, of type \f(CW\*(C`SV **\*(C'\fR.
3957 .PP
3958 The head of the stack is indexed by \f(CW\*(C`PL_tmps_ix\*(C'\fR, an integer which stores the
3959 index in the array of the most recently-pushed item.
3960 .PP
3961 There is no public API to directly push items to the temporaries stack. Instead,
3962 the API function \f(CWsv_2mortal()\fR is used to mortalize an xV, adding its
3963 address to the temporaries stack.
3964 .PP
3965 Likewise, there is no public API to read values from the temporaries stack.
3966 Instead, the macros \f(CW\*(C`SAVETMPS\*(C'\fR and \f(CW\*(C`FREETMPS\*(C'\fR are used. The \f(CW\*(C`SAVETMPS\*(C'\fR
3967 macro establishes the base levels of the temporaries stack, by capturing the
3968 current value of \f(CW\*(C`PL_tmps_ix\*(C'\fR into \f(CW\*(C`PL_tmps_floor\*(C'\fR and saving the previous
3969 value to the save stack. Thereafter, whenever \f(CW\*(C`FREETMPS\*(C'\fR is invoked all of
3970 the temporaries that have been pushed since that level are reclaimed.
3971 .PP
3972 While it is common to see these two macros in pairs within an \f(CW\*(C`ENTER\*(C'\fR/
3973 \&\f(CW\*(C`LEAVE\*(C'\fR pair, it is not necessary to match them. It is permitted to invoke
3974 \&\f(CW\*(C`FREETMPS\*(C'\fR multiple times since the most recent \f(CW\*(C`SAVETMPS\*(C'\fR; for example in a
3975 loop iterating over elements of a list. While you can invoke \f(CW\*(C`SAVETMPS\*(C'\fR
3976 multiple times within a scope pair, it is unlikely to be useful. Subsequent
3977 invocations will move the temporaries floor further up, thus effectively
3978 trapping the existing temporaries to only be released at the end of the scope.
3979 .SS "Save Stack"
3980 .IX Subsection "Save Stack"
3981 The save stack is used by perl to implement the \f(CW\*(C`local\*(C'\fR keyword and other
3982 similar behaviours; any cleanup operations that need to be performed when
3983 leaving the current scope. Items pushed to this stack generally capture the
3984 current value of some internal variable or state, which will be restored when
3985 the scope is unwound due to leaving, \f(CW\*(C`return\*(C'\fR, \f(CW\*(C`die\*(C'\fR, \f(CW\*(C`goto\*(C'\fR or other
3986 reasons.
3987 .PP
3988 Whereas other perl internal stacks store individual items all of the same type
3989 (usually SV pointers or integers), the items pushed to the save stack are
3990 formed of many different types, having multiple fields to them. For example,
3991 the \f(CW\*(C`SAVEt_INT\*(C'\fR type needs to store both the address of the \f(CW\*(C`int\*(C'\fR variable
3992 to restore, and the value to restore it to. This information could have been
3993 stored using fields of a \f(CW\*(C`struct\*(C'\fR, but would have to be large enough to store
3994 three pointers in the largest case, which would waste a lot of space in most
3995 of the smaller cases.
3996 .PP
3997 Instead, the stack stores information in a variable-length encoding of \f(CW\*(C`ANY\*(C'\fR
3998 structures. The final value pushed is stored in the \f(CW\*(C`UV\*(C'\fR field which encodes
3999 the kind of item held by the preceding items; the count and types of which
4000 will depend on what kind of item is being stored. The kind field is pushed
4001 last because that will be the first field to be popped when unwinding items
4002 from the stack.
4003 .PP
4004 The base of this stack is pointed to by the interpreter variable
4005 \&\f(CW\*(C`PL_savestack\*(C'\fR, of type \f(CW\*(C`ANY *\*(C'\fR.
4006 .PP
4007 The head of the stack is indexed by \f(CW\*(C`PL_savestack_ix\*(C'\fR, an integer which
4008 stores the index in the array at which the next item should be pushed. (Note
4009 that this is different to most other stacks, which reference the most
4010 recently-pushed item).
4011 .PP
4012 Items are pushed to the save stack by using the various \f(CW\*(C`SAVE...()\*(C'\fR macros.
4013 Many of these macros take a variable and store both its address and current
4014 value on the save stack, ensuring that value gets restored on scope exit.
4015 .PP
4016 .Vb 5
4017 \& SAVEI8(i8)
4018 \& SAVEI16(i16)
4019 \& SAVEI32(i32)
4020 \& SAVEINT(i)
4021 \& ...
4022 .Ve
4023 .PP
4024 There are also a variety of other special-purpose macros which save particular
4025 types or values of interest. \f(CW\*(C`SAVETMPS\*(C'\fR has already been mentioned above.
4026 Others include \f(CW\*(C`SAVEFREEPV\*(C'\fR which arranges for a PV (i.e. a string buffer) to
4027 be freed, or \f(CW\*(C`SAVEDESTRUCTOR\*(C'\fR which arranges for a given function pointer to
4028 be invoked on scope exit. A full list of such macros can be found in
4029 \&\fIscope.h\fR.
4030 .PP
4031 There is no public API for popping individual values or items from the save
4032 stack. Instead, via the scope stack, the \f(CW\*(C`ENTER\*(C'\fR and \f(CW\*(C`LEAVE\*(C'\fR pair form a way
4033 to start and stop nested scopes. Leaving a nested scope via \f(CW\*(C`LEAVE\*(C'\fR will
4034 restore all of the saved values that had been pushed since the most recent
4035 \&\f(CW\*(C`ENTER\*(C'\fR.
4036 .SS "Scope Stack"
4037 .IX Subsection "Scope Stack"
4038 As with the mark stack to the value stack, the scope stack forms a pair with
4039 the save stack. The scope stack stores the height of the save stack at which
4040 nested scopes begin, and allows the save stack to be unwound back to that
4041 point when the scope is left.
4042 .PP
4043 When perl is built with debugging enabled, there is a second part to this
4044 stack storing human-readable string names describing the type of stack
4045 context. Each push operation saves the name as well as the height of the save
4046 stack, and each pop operation checks the topmost name with what is expected,
4047 causing an assertion failure if the name does not match.
4048 .PP
4049 The base of this stack is pointed to by the interpreter variable
4050 \&\f(CW\*(C`PL_scopestack\*(C'\fR, of type \f(CW\*(C`I32 *\*(C'\fR. If enabled, the scope stack names are
4051 stored in a separate array pointed to by \f(CW\*(C`PL_scopestack_name\*(C'\fR, of type
4052 \&\f(CW\*(C`const char **\*(C'\fR.
4053 .PP
4054 The head of the stack is indexed by \f(CW\*(C`PL_scopestack_ix\*(C'\fR, an integer which
4055 stores the index of the array or arrays at which the next item should be
4056 pushed. (Note that this is different to most other stacks, which reference the
4057 most recently-pushed item).
4058 .PP
4059 Values are pushed to the scope stack using the \f(CW\*(C`ENTER\*(C'\fR macro, which begins a
4060 new nested scope. Any items pushed to the save stack are then restored at the
4061 next nested invocation of the \f(CW\*(C`LEAVE\*(C'\fR macro.
4062 .SH "Dynamic Scope and the Context Stack"
4063 .IX Header "Dynamic Scope and the Context Stack"
4064 \&\fBNote:\fR this section describes a non-public internal API that is subject
4065 to change without notice.
4066 .SS "Introduction to the context stack"
4067 .IX Subsection "Introduction to the context stack"
4068 In Perl, dynamic scoping refers to the runtime nesting of things like
4069 subroutine calls, evals etc, as well as the entering and exiting of block
4070 scopes. For example, the restoring of a \f(CW\*(C`local\*(C'\fRised variable is
4071 determined by the dynamic scope.
4072 .PP
4073 Perl tracks the dynamic scope by a data structure called the context
4074 stack, which is an array of \f(CW\*(C`PERL_CONTEXT\*(C'\fR structures, and which is
4075 itself a big union for all the types of context. Whenever a new scope is
4076 entered (such as a block, a \f(CW\*(C`for\*(C'\fR loop, or a subroutine call), a new
4077 context entry is pushed onto the stack. Similarly when leaving a block or
4078 returning from a subroutine call etc. a context is popped. Since the
4079 context stack represents the current dynamic scope, it can be searched.
4080 For example, \f(CW\*(C`next LABEL\*(C'\fR searches back through the stack looking for a
4081 loop context that matches the label; \f(CW\*(C`return\*(C'\fR pops contexts until it
4082 finds a sub or eval context or similar; \f(CW\*(C`caller\*(C'\fR examines sub contexts on
4083 the stack.
4084 .PP
4085 Each context entry is labelled with a context type, \f(CW\*(C`cx_type\*(C'\fR. Typical
4086 context types are \f(CW\*(C`CXt_SUB\*(C'\fR, \f(CW\*(C`CXt_EVAL\*(C'\fR etc., as well as \f(CW\*(C`CXt_BLOCK\*(C'\fR
4087 and \f(CW\*(C`CXt_NULL\*(C'\fR which represent a basic scope (as pushed by \f(CW\*(C`pp_enter\*(C'\fR)
4088 and a sort block. The type determines which part of the context union are
4089 valid.
4090 .PP
4091 The main division in the context struct is between a substitution scope
4092 (\f(CW\*(C`CXt_SUBST\*(C'\fR) and block scopes, which are everything else. The former is
4093 just used while executing \f(CW\*(C`s///e\*(C'\fR, and won't be discussed further
4094 here.
4095 .PP
4096 All the block scope types share a common base, which corresponds to
4097 \&\f(CW\*(C`CXt_BLOCK\*(C'\fR. This stores the old values of various scope-related
4098 variables like \f(CW\*(C`PL_curpm\*(C'\fR, as well as information about the current
4099 scope, such as \f(CW\*(C`gimme\*(C'\fR. On scope exit, the old variables are restored.
4100 .PP
4101 Particular block scope types store extra per-type information. For
4102 example, \f(CW\*(C`CXt_SUB\*(C'\fR stores the currently executing CV, while the various
4103 for loop types might hold the original loop variable SV. On scope exit,
4104 the per-type data is processed; for example the CV has its reference count
4105 decremented, and the original loop variable is restored.
4106 .PP
4107 The macro \f(CW\*(C`cxstack\*(C'\fR returns the base of the current context stack, while
4108 \&\f(CW\*(C`cxstack_ix\*(C'\fR is the index of the current frame within that stack.
4109 .PP
4110 In fact, the context stack is actually part of a stack-of-stacks system;
4111 whenever something unusual is done such as calling a \f(CW\*(C`DESTROY\*(C'\fR or tie
4112 handler, a new stack is pushed, then popped at the end.
4113 .PP
4114 Note that the API described here changed considerably in perl 5.24; prior
4115 to that, big macros like \f(CW\*(C`PUSHBLOCK\*(C'\fR and \f(CW\*(C`POPSUB\*(C'\fR were used; in 5.24
4116 they were replaced by the inline static functions described below. In
4117 addition, the ordering and detail of how these macros/function work
4118 changed in many ways, often subtly. In particular they didn't handle
4119 saving the savestack and temps stack positions, and required additional
4120 \&\f(CW\*(C`ENTER\*(C'\fR, \f(CW\*(C`SAVETMPS\*(C'\fR and \f(CW\*(C`LEAVE\*(C'\fR compared to the new functions. The
4121 old-style macros will not be described further.
4122 .SS "Pushing contexts"
4123 .IX Subsection "Pushing contexts"
4124 For pushing a new context, the two basic functions are
4125 \&\f(CW\*(C`cx = cx_pushblock()\*(C'\fR, which pushes a new basic context block and returns
4126 its address, and a family of similar functions with names like
4127 \&\f(CWcx_pushsub(cx)\fR which populate the additional type-dependent fields in
4128 the \f(CW\*(C`cx\*(C'\fR struct. Note that \f(CW\*(C`CXt_NULL\*(C'\fR and \f(CW\*(C`CXt_BLOCK\*(C'\fR don't have their
4129 own push functions, as they don't store any data beyond that pushed by
4130 \&\f(CW\*(C`cx_pushblock\*(C'\fR.
4131 .PP
4132 The fields of the context struct and the arguments to the \f(CW\*(C`cx_*\*(C'\fR
4133 functions are subject to change between perl releases, representing
4134 whatever is convenient or efficient for that release.
4135 .PP
4136 A typical context stack pushing can be found in \f(CW\*(C`pp_entersub\*(C'\fR; the
4137 following shows a simplified and stripped-down example of a non-XS call,
4138 along with comments showing roughly what each function does.
4139 .PP
4140 .Vb 6
4141 \& dMARK;
4142 \& U8 gimme = GIMME_V;
4143 \& bool hasargs = cBOOL(PL_op\->op_flags & OPf_STACKED);
4144 \& OP *retop = PL_op\->op_next;
4145 \& I32 old_ss_ix = PL_savestack_ix;
4146 \& CV *cv = ....;
4147 \&
4148 \& /* ... make mortal copies of stack args which are PADTMPs here ... */
4149 \&
4150 \& /* ... do any additional savestack pushes here ... */
4151 \&
4152 \& /* Now push a new context entry of type \*(AqCXt_SUB\*(Aq; initially just
4153 \& * doing the actions common to all block types: */
4154 \&
4155 \& cx = cx_pushblock(CXt_SUB, gimme, MARK, old_ss_ix);
4156 \&
4157 \& /* this does (approximately):
4158 \& CXINC; /* cxstack_ix++ (grow if necessary) */
4159 \& cx = CX_CUR(); /* and get the address of new frame */
4160 \& cx\->cx_type = CXt_SUB;
4161 \& cx\->blk_gimme = gimme;
4162 \& cx\->blk_oldsp = MARK \- PL_stack_base;
4163 \& cx\->blk_oldsaveix = old_ss_ix;
4164 \& cx\->blk_oldcop = PL_curcop;
4165 \& cx\->blk_oldmarksp = PL_markstack_ptr \- PL_markstack;
4166 \& cx\->blk_oldscopesp = PL_scopestack_ix;
4167 \& cx\->blk_oldpm = PL_curpm;
4168 \& cx\->blk_old_tmpsfloor = PL_tmps_floor;
4169 \&
4170 \& PL_tmps_floor = PL_tmps_ix;
4171 \& */
4172 \&
4173 \&
4174 \& /* then update the new context frame with subroutine\-specific info,
4175 \& * such as the CV about to be executed: */
4176 \&
4177 \& cx_pushsub(cx, cv, retop, hasargs);
4178 \&
4179 \& /* this does (approximately):
4180 \& cx\->blk_sub.cv = cv;
4181 \& cx\->blk_sub.olddepth = CvDEPTH(cv);
4182 \& cx\->blk_sub.prevcomppad = PL_comppad;
4183 \& cx\->cx_type |= (hasargs) ? CXp_HASARGS : 0;
4184 \& cx\->blk_sub.retop = retop;
4185 \& SvREFCNT_inc_simple_void_NN(cv);
4186 \& */
4187 .Ve
4188 .PP
4189 Note that \f(CWcx_pushblock()\fR sets two new floors: for the args stack (to
4190 \&\f(CW\*(C`MARK\*(C'\fR) and the temps stack (to \f(CW\*(C`PL_tmps_ix\*(C'\fR). While executing at this
4191 scope level, every \f(CW\*(C`nextstate\*(C'\fR (amongst others) will reset the args and
4192 tmps stack levels to these floors. Note that since \f(CW\*(C`cx_pushblock\*(C'\fR uses
4193 the current value of \f(CW\*(C`PL_tmps_ix\*(C'\fR rather than it being passed as an arg,
4194 this dictates at what point \f(CW\*(C`cx_pushblock\*(C'\fR should be called. In
4195 particular, any new mortals which should be freed only on scope exit
4196 (rather than at the next \f(CW\*(C`nextstate\*(C'\fR) should be created first.
4197 .PP
4198 Most callers of \f(CW\*(C`cx_pushblock\*(C'\fR simply set the new args stack floor to the
4199 top of the previous stack frame, but for \f(CW\*(C`CXt_LOOP_LIST\*(C'\fR it stores the
4200 items being iterated over on the stack, and so sets \f(CW\*(C`blk_oldsp\*(C'\fR to the
4201 top of these items instead. Note that, contrary to its name, \f(CW\*(C`blk_oldsp\*(C'\fR
4202 doesn't always represent the value to restore \f(CW\*(C`PL_stack_sp\*(C'\fR to on scope
4203 exit.
4204 .PP
4205 Note the early capture of \f(CW\*(C`PL_savestack_ix\*(C'\fR to \f(CW\*(C`old_ss_ix\*(C'\fR, which is
4206 later passed as an arg to \f(CW\*(C`cx_pushblock\*(C'\fR. In the case of \f(CW\*(C`pp_entersub\*(C'\fR,
4207 this is because, although most values needing saving are stored in fields
4208 of the context struct, an extra value needs saving only when the debugger
4209 is running, and it doesn't make sense to bloat the struct for this rare
4210 case. So instead it is saved on the savestack. Since this value gets
4211 calculated and saved before the context is pushed, it is necessary to pass
4212 the old value of \f(CW\*(C`PL_savestack_ix\*(C'\fR to \f(CW\*(C`cx_pushblock\*(C'\fR, to ensure that the
4213 saved value gets freed during scope exit. For most users of
4214 \&\f(CW\*(C`cx_pushblock\*(C'\fR, where nothing needs pushing on the save stack,
4215 \&\f(CW\*(C`PL_savestack_ix\*(C'\fR is just passed directly as an arg to \f(CW\*(C`cx_pushblock\*(C'\fR.
4216 .PP
4217 Note that where possible, values should be saved in the context struct
4218 rather than on the save stack; it's much faster that way.
4219 .PP
4220 Normally \f(CW\*(C`cx_pushblock\*(C'\fR should be immediately followed by the appropriate
4221 \&\f(CW\*(C`cx_pushfoo\*(C'\fR, with nothing between them; this is because if code
4222 in-between could die (e.g. a warning upgraded to fatal), then the context
4223 stack unwinding code in \f(CW\*(C`dounwind\*(C'\fR would see (in the example above) a
4224 \&\f(CW\*(C`CXt_SUB\*(C'\fR context frame, but without all the subroutine-specific fields
4225 set, and crashes would soon ensue.
4226 .PP
4227 Where the two must be separate, initially set the type to \f(CW\*(C`CXt_NULL\*(C'\fR or
4228 \&\f(CW\*(C`CXt_BLOCK\*(C'\fR, and later change it to \f(CW\*(C`CXt_foo\*(C'\fR when doing the
4229 \&\f(CW\*(C`cx_pushfoo\*(C'\fR. This is exactly what \f(CW\*(C`pp_enteriter\*(C'\fR does, once it's
4230 determined which type of loop it's pushing.
4231 .SS "Popping contexts"
4232 .IX Subsection "Popping contexts"
4233 Contexts are popped using \f(CWcx_popsub()\fR etc. and \f(CWcx_popblock()\fR. Note
4234 however, that unlike \f(CW\*(C`cx_pushblock\*(C'\fR, neither of these functions actually
4235 decrement the current context stack index; this is done separately using
4236 \&\f(CWCX_POP()\fR.
4237 .PP
4238 There are two main ways that contexts are popped. During normal execution
4239 as scopes are exited, functions like \f(CW\*(C`pp_leave\*(C'\fR, \f(CW\*(C`pp_leaveloop\*(C'\fR and
4240 \&\f(CW\*(C`pp_leavesub\*(C'\fR process and pop just one context using \f(CW\*(C`cx_popfoo\*(C'\fR and
4241 \&\f(CW\*(C`cx_popblock\*(C'\fR. On the other hand, things like \f(CW\*(C`pp_return\*(C'\fR and \f(CW\*(C`next\*(C'\fR
4242 may have to pop back several scopes until a sub or loop context is found,
4243 and exceptions (such as \f(CW\*(C`die\*(C'\fR) need to pop back contexts until an eval
4244 context is found. Both of these are accomplished by \f(CWdounwind()\fR, which
4245 is capable of processing and popping all contexts above the target one.
4246 .PP
4247 Here is a typical example of context popping, as found in \f(CW\*(C`pp_leavesub\*(C'\fR
4248 (simplified slightly):
4249 .PP
4250 .Vb 4
4251 \& U8 gimme;
4252 \& PERL_CONTEXT *cx;
4253 \& SV **oldsp;
4254 \& OP *retop;
4255 \&
4256 \& cx = CX_CUR();
4257 \&
4258 \& gimme = cx\->blk_gimme;
4259 \& oldsp = PL_stack_base + cx\->blk_oldsp; /* last arg of previous frame */
4260 \&
4261 \& if (gimme == G_VOID)
4262 \& PL_stack_sp = oldsp;
4263 \& else
4264 \& leave_adjust_stacks(oldsp, oldsp, gimme, 0);
4265 \&
4266 \& CX_LEAVE_SCOPE(cx);
4267 \& cx_popsub(cx);
4268 \& cx_popblock(cx);
4269 \& retop = cx\->blk_sub.retop;
4270 \& CX_POP(cx);
4271 \&
4272 \& return retop;
4273 .Ve
4274 .PP
4275 The steps above are in a very specific order, designed to be the reverse
4276 order of when the context was pushed. The first thing to do is to copy
4277 and/or protect any return arguments and free any temps in the current
4278 scope. Scope exits like an rvalue sub normally return a mortal copy of
4279 their return args (as opposed to lvalue subs). It is important to make
4280 this copy before the save stack is popped or variables are restored, or
4281 bad things like the following can happen:
4282 .PP
4283 .Vb 2
4284 \& sub f { my $x =...; $x } # $x freed before we get to copy it
4285 \& sub f { /(...)/; $1 } # PL_curpm restored before $1 copied
4286 .Ve
4287 .PP
4288 Although we wish to free any temps at the same time, we have to be careful
4289 not to free any temps which are keeping return args alive; nor to free the
4290 temps we have just created while mortal copying return args. Fortunately,
4291 \&\f(CWleave_adjust_stacks()\fR is capable of making mortal copies of return args,
4292 shifting args down the stack, and only processing those entries on the
4293 temps stack that are safe to do so.
4294 .PP
4295 In void context no args are returned, so it's more efficient to skip
4296 calling \f(CWleave_adjust_stacks()\fR. Also in void context, a \f(CW\*(C`nextstate\*(C'\fR op
4297 is likely to be imminently called which will do a \f(CW\*(C`FREETMPS\*(C'\fR, so there's
4298 no need to do that either.
4299 .PP
4300 The next step is to pop savestack entries: \f(CWCX_LEAVE_SCOPE(cx)\fR is just
4301 defined as \f(CWLEAVE_SCOPE(cx\->blk_oldsaveix)\fR. Note that during the
4302 popping, it's possible for perl to call destructors, call \f(CW\*(C`STORE\*(C'\fR to undo
4303 localisations of tied vars, and so on. Any of these can die or call
4304 \&\f(CWexit()\fR. In this case, \f(CWdounwind()\fR will be called, and the current
4305 context stack frame will be re-processed. Thus it is vital that all steps
4306 in popping a context are done in such a way to support reentrancy. The
4307 other alternative, of decrementing \f(CW\*(C`cxstack_ix\*(C'\fR \fIbefore\fR processing the
4308 frame, would lead to leaks and the like if something died halfway through,
4309 or overwriting of the current frame.
4310 .PP
4311 \&\f(CW\*(C`CX_LEAVE_SCOPE\*(C'\fR itself is safely re-entrant: if only half the savestack
4312 items have been popped before dying and getting trapped by eval, then the
4313 \&\f(CW\*(C`CX_LEAVE_SCOPE\*(C'\fRs in \f(CW\*(C`dounwind\*(C'\fR or \f(CW\*(C`pp_leaveeval\*(C'\fR will continue where
4314 the first one left off.
4315 .PP
4316 The next step is the type-specific context processing; in this case
4317 \&\f(CW\*(C`cx_popsub\*(C'\fR. In part, this looks like:
4318 .PP
4319 .Vb 4
4320 \& cv = cx\->blk_sub.cv;
4321 \& CvDEPTH(cv) = cx\->blk_sub.olddepth;
4322 \& cx\->blk_sub.cv = NULL;
4323 \& SvREFCNT_dec(cv);
4324 .Ve
4325 .PP
4326 where its processing the just-executed CV. Note that before it decrements
4327 the CV's reference count, it nulls the \f(CW\*(C`blk_sub.cv\*(C'\fR. This means that if
4328 it re-enters, the CV won't be freed twice. It also means that you can't
4329 rely on such type-specific fields having useful values after the return
4330 from \f(CW\*(C`cx_popfoo\*(C'\fR.
4331 .PP
4332 Next, \f(CW\*(C`cx_popblock\*(C'\fR restores all the various interpreter vars to their
4333 previous values or previous high water marks; it expands to:
4334 .PP
4335 .Vb 5
4336 \& PL_markstack_ptr = PL_markstack + cx\->blk_oldmarksp;
4337 \& PL_scopestack_ix = cx\->blk_oldscopesp;
4338 \& PL_curpm = cx\->blk_oldpm;
4339 \& PL_curcop = cx\->blk_oldcop;
4340 \& PL_tmps_floor = cx\->blk_old_tmpsfloor;
4341 .Ve
4342 .PP
4343 Note that it \fIdoesn't\fR restore \f(CW\*(C`PL_stack_sp\*(C'\fR; as mentioned earlier,
4344 which value to restore it to depends on the context type (specifically
4345 \&\f(CW\*(C`for (list) {}\*(C'\fR), and what args (if any) it returns; and that will
4346 already have been sorted out earlier by \f(CWleave_adjust_stacks()\fR.
4347 .PP
4348 Finally, the context stack pointer is actually decremented by \f(CWCX_POP(cx)\fR.
4349 After this point, it's possible that that the current context frame could
4350 be overwritten by other contexts being pushed. Although things like ties
4351 and \f(CW\*(C`DESTROY\*(C'\fR are supposed to work within a new context stack, it's best
4352 not to assume this. Indeed on debugging builds, \f(CWCX_POP(cx)\fR deliberately
4353 sets \f(CW\*(C`cx\*(C'\fR to null to detect code that is still relying on the field
4354 values in that context frame. Note in the \f(CWpp_leavesub()\fR example above,
4355 we grab \f(CW\*(C`blk_sub.retop\*(C'\fR \fIbefore\fR calling \f(CW\*(C`CX_POP\*(C'\fR.
4356 .SS "Redoing contexts"
4357 .IX Subsection "Redoing contexts"
4358 Finally, there is \f(CWcx_topblock(cx)\fR, which acts like a super\-\f(CW\*(C`nextstate\*(C'\fR
4359 as regards to resetting various vars to their base values. It is used in
4360 places like \f(CW\*(C`pp_next\*(C'\fR, \f(CW\*(C`pp_redo\*(C'\fR and \f(CW\*(C`pp_goto\*(C'\fR where rather than
4361 exiting a scope, we want to re-initialise the scope. As well as resetting
4362 \&\f(CW\*(C`PL_stack_sp\*(C'\fR like \f(CW\*(C`nextstate\*(C'\fR, it also resets \f(CW\*(C`PL_markstack_ptr\*(C'\fR,
4363 \&\f(CW\*(C`PL_scopestack_ix\*(C'\fR and \f(CW\*(C`PL_curpm\*(C'\fR. Note that it doesn't do a
4364 \&\f(CW\*(C`FREETMPS\*(C'\fR.
4365 .SH "Reference-counted argument stack"
4366 .IX Header "Reference-counted argument stack"
4367 .SS Introduction
4368 .IX Subsection "Introduction"
4369 As of perl 5.40, there is a build option, \f(CW\*(C`PERL_RC_STACK\*(C'\fR, not enabled by
4370 default, which requires that items pushed onto, or popped off the argument
4371 stack have their reference counts adjusted. It is intended that eventually
4372 this will be the default way (and finally the only way) to configure perl.
4373 .PP
4374 The macros which manipulate the stack such as \fBPUSHs()\fR and \fBPOPs()\fR don't
4375 adjust the reference count of the SV. Most of the time this is fine, since
4376 something else is keeping the SV alive while on the argument stack, such
4377 a pointer from the TEMPs stack, or from the pad (e.g. a lexical variable
4378 or a \f(CW\*(C`PADTMP\*(C'\fR). Occasionally this can go horribly wrong. For example,
4379 this code:
4380 .PP
4381 .Vb 3
4382 \& my @a = (1,2,3);
4383 \& sub f { @a = (); print "(@_)\en" };
4384 \& f(@a, 4);
4385 .Ve
4386 .PP
4387 may print undefined or random freed values, since some of the elements of
4388 \&\f(CW@_\fR, which have been aliased to the elements of \f(CW@a\fR, have been freed.
4389 \&\f(CW\*(C`PERL_RC_STACK\*(C'\fR is intended to fix this by making each SV pointer on the
4390 argument stack increment the reference count (RC) of the SV by one.
4391 .PP
4392 In this new environment, unmodified existing PP and XS functions, which
4393 have been written assuming a non reference-counted stack (non-RC for
4394 short), are called via special wrapper functions which adjust the stack
4395 before and after. At the moment there is no API to write an RC XS
4396 function, so all XS code will continue to be called via a wrapper (which
4397 makes them slightly slower), but means that in general, CPAN distributions
4398 containing XS code code continue to work without modification.
4399 .PP
4400 However, PP functions, either in perl core, or those in XS functions used
4401 to implement custom ops or to override the PP functions for built-in ops,
4402 need dealing with specially. For the latter, they can just be wrapped;
4403 this involves the least work, but has a performance impact. In the longer
4404 term, and for core PP functions, they need unwrapping and rewriting using
4405 a new API. With this, the old macros such as \fBPUSHs()\fR have been replaced
4406 with a new set of (mostly inline) functions with a common prefix, such as
4407 \&\fBrpp_push_1()\fR. "RPP" stands for "reference-counted push and pop functions".
4408 The new functions modify the reference count on \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds,
4409 while leaving them unadjusted otherwise. Thus in core they generally work
4410 in both cases, while in XS code they are portable to older perl versions
4411 via \f(CW\*(C`PPPort\*(C'\fR (XXX assuming that they get been added to \f(CW\*(C`PPPort\*(C'\fR).
4412 .PP
4413 The rest of this section is mainly concerned with how to convert existing
4414 PP functions, and how to write new PP functions to use the new \f(CW\*(C`rpp_\*(C'\fR
4415 API.
4416 .PP
4417 A reference-counted perl can be built using the PERL_RC_STACK define.
4418 For development and debugging purposes, it is best to enable leaking
4419 scalar debugging too, as that displays extra information about scalars
4420 that have leaked or been prematurely freed.
4421 .PP
4422 .Vb 2
4423 \& Configure \-DDEBUGGING \e
4424 \& \-Accflags=\*(Aq\-DPERL_RC_STACK \-DDEBUG_LEAKING_SCALARS\*(Aq
4425 .Ve
4426 .SS "Reference counted stack states"
4427 .IX Subsection "Reference counted stack states"
4428 In the new regime, the current argument stack can be in one of three
4429 states, which can be determined by the shown expression.
4430 .IP \(bu 4
4431 not reference-counted
4432 .Sp
4433 .Vb 1
4434 \& !AvREAL(PL_curstack)
4435 .Ve
4436 .Sp
4437 In this case, perl will assume when emptying the stack (such as during a
4438 \&\fBcroak()\fR) that the items on it don't need freeing. This is the traditional
4439 perl behaviour. On \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds, such stacks will be rarely
4440 encountered.
4441 .IP \(bu 4
4442 fully reference-counted
4443 .Sp
4444 .Vb 1
4445 \& AvREAL(PL_curstack) && !PL_curstackinfo\->si_stack_nonrc_base
4446 .Ve
4447 .Sp
4448 All the items on the stack are reference counted, and will be freed by
4449 functions like \fBrpp_popfree_1()\fR or if perl \fBcroak()\fRs. This is the normal
4450 state of the stack in \f(CW\*(C`PERL_RC_STACK\*(C'\fR builds.
4451 .IP \(bu 4
4452 partially reference-counted (split)
4453 .Sp
4454 .Vb 1
4455 \& AvREAL(PL_curstack) && PL_curstackinfo\->si_stack_nonrc_base > 0
4456 .Ve
4457 .Sp
4458 In this case, items on the stack from the index \f(CW\*(C`si_stack_nonrc_base\*(C'\fR
4459 upwards are non-RC; those below are RC. This state occurs when a PP or XS
4460 function has been wrapped. In this case, the wrapper function pushes a
4461 non-RC copy of the arg pointers above the cut then calls the real
4462 function. When that returns, the wrapper function bumps up the RC of any
4463 returned args. See below for more details.
4464 .PP
4465 Note that perl uses a stack-of-stacks, and the \fBAvREAL()\fR and
4466 \&\f(CW\*(C`si_stack_nonrc_base\*(C'\fR states are per stack. When perl starts up, the main
4467 stack is RC, but by default, new stacks pushed in XS code via \fBPUSHSTACKi()\fR
4468 are non-RC, so it is quite possible to get a mixture. The perl core itself
4469 uses the new \fBpush_stackinfo()\fR function which replaces \fBPUSHSTACKi()\fR and
4470 allows you to specify that the new stack should be RC by default.
4471 (XXX core mostly hasn't actually been updated yet to use \fBpush_stackinfo()\fR)
4472 .PP
4473 Most places in the core assume a particular RC environment. In particular,
4474 it is assumed that within a runops loop, all the PP functions are
4475 RC-aware, either because they have been (re)written to be aware, or
4476 because they have been wrapped. Whenever a runops loop is entered via
4477 \&\fBCALLRUNOPS()\fR, it will check the current state of the stack, and if it's
4478 not fully RC, will temporarily update its contents to be fully RC before
4479 entering the main runops loop. Then if necessary it will restore the stack
4480 to its old state on return. This means that functions like \fBcall_sv()\fR,
4481 which can be called from any environment (e.g. RC core or wrapped and
4482 temporarily non-RC XS code) will always do the Right Thing when invoking
4483 the runops loop, no matter what the current stack state is.
4484 .PP
4485 Similarly, croaks and the like (which can occur anywhere) have to be able
4486 to handle both stack types. So there are a few places in core \- \fBcall_sv()\fR,
4487 \&\fBeval_sv()\fR etc, \fBPerl_die_unwind()\fR and \fBS_my_exit_jump()\fR \- which have been
4488 specially crafted to handle both cases; everything else can assume a fixed
4489 environment.
4490 .SS Wrapping
4491 .IX Subsection "Wrapping"
4492 Normally a core PP function is declared like this:
4493 .PP
4494 .Vb 4
4495 \& PP(pp_foo)
4496 \& {
4497 \& ...
4498 \& }
4499 .Ve
4500 .PP
4501 This expands to something like:
4502 .PP
4503 .Vb 4
4504 \& OP* Perl_pp_foo(pTHX)
4505 \& {
4506 \& ...
4507 \& }
4508 .Ve
4509 .PP
4510 When such a function needs to be wrapped, it is instead declared as:
4511 .PP
4512 .Vb 4
4513 \& PP_wrapped(pp_foo, nargs, nlists)
4514 \& {
4515 \& ...
4516 \& }
4517 .Ve
4518 .PP
4519 which on non-RC builds, expands to the same as \fBPP()\fR (the extra args are
4520 ignored). On RC builds it expands to something like
4521 .PP
4522 .Vb 4
4523 \& OP* Perl_pp_foo(pTHX)
4524 \& {
4525 \& return Perl_pp_wrap(aTHX_ S_Perl_pp_foo_norc, nargs, nlists);
4526 \& }
4527 \&
4528 \& STATIC OP* S_Perl_pp_foo_norc(pTHX)
4529 \& {
4530 \& ...
4531 \& }
4532 .Ve
4533 .PP
4534 Here the externally visible PP function calls \fBpp_wrap()\fR, which adjusts
4535 the stack contents, then calls the hidden real body of the PP function,
4536 then on return, adjusts the stack back.
4537 .PP
4538 There is an API macro, \fBXSPP_wrapped()\fR, intended for use on PP functions
4539 declared in XS code, It is identical to \fBPP_wrapped()\fR, except that it
4540 doesn't prepend a \f(CW\*(C`Perl_\*(C'\fR prefix to the function name.
4541 .PP
4542 The \f(CW\*(C`nargs\*(C'\fR and \f(CW\*(C`nlists\*(C'\fR parameters to the macro are numeric constants
4543 or simple expressions which specify how many arguments the PP function
4544 expects, or how many lists it expects. For example,
4545 .PP
4546 .Vb 1
4547 \& PP_wrapped(pp_add, 2, 0); /* consumes two args off the stack */
4548 \&
4549 \& PP_wrapped(pp_readline, /* consumes one or two args */
4550 \& ((PL_op\->op_flags & OPf_STACKED) ? 2 : 1), 0);
4551 \&
4552 \& PP_wrapped(pp_push, 0, 1); /* consumes one list */
4553 \&
4554 \& PP_wrapped(pp_aassign, 0, 2); /* consumes two lists */
4555 .Ve
4556 .PP
4557 To understand what \fBpp_wrap()\fR does, consider calling \fBPerl_pp_foo()\fR which
4558 expects three arguments. On entry the stack may look like:
4559 .PP
4560 .Vb 1
4561 \& ... A+ B+ C+
4562 .Ve
4563 .PP
4564 (where the \f(CW\*(C`+\*(C'\fR indicates that the pointers to A, B and C are each
4565 reference counted). The wrapper function \fBpp_wrap()\fR marks a cut at the
4566 current stack position using \f(CW\*(C`si_stack_nonrc_base\*(C'\fR, then, based on the
4567 value of \f(CW\*(C`nargs\*(C'\fR, pushes a copy of those three pointers above the cut:
4568 .PP
4569 .Vb 1
4570 \& ... A+ B+ C+ | A0 B0 C0
4571 .Ve
4572 .PP
4573 (where the \f(CW0\fR indicates that the pointers aren't RC), then calls the
4574 real PP function, \fBS_Perl_pp_foo_norc()\fR. That function processes A, B and C,
4575 pops them off the stack, and pushes some result SVs. None of this
4576 manipulation adjusts any RCs. On return to \fBpp_wrap()\fR, the stack may look
4577 something like:
4578 .PP
4579 .Vb 1
4580 \& ... A+ B+ C+ | X0 Y0
4581 .Ve
4582 .PP
4583 The wrapper function bumps up the RCs of X and Y, decrements A B C,
4584 shifts the results down and sets \f(CW\*(C`si_stack_nonrc_base\*(C'\fR to zero, leaving
4585 the stack as:
4586 .PP
4587 .Vb 1
4588 \& ... X+ Y+
4589 .Ve
4590 .PP
4591 In places like \fBpp_entersub()\fR, a similar wrapping (via the functions
4592 \&\fBrpp_invoke_xs()\fR and then \fBxs_wrap()\fR) is done when calling XS subs.
4593 .PP
4594 When \f(CW\*(C`nlists\*(C'\fR is positive, a similar action takes place, except that the
4595 mark stack is examined (and adjusted) in order to determine the number of
4596 args that need copying.
4597 .PP
4598 A complex calling environment might have multiple nested stacks with
4599 different RC states. Perl starts off with an RC stack. Then for example,
4600 \&\fBpp_entersub()\fR is called, which (via \fBxs_wrap()\fR) splits the stack and
4601 executes the XS function in a non-RC environment. That function may call
4602 \&\fBPUSHSTACKi()\fR, which creates a new non-RC stack, then calls \fBcall_sv()\fR, which
4603 does \fBCALLRUNOPS()\fR, which causes the new stack to temporarily become RC.
4604 Then a tied method is called, which pushes a new RC stack, and so on. (XXX
4605 currently tied methods actually push a non-RC stack. To be fixed soon).
4606 .SS "(Re)writing a PP function using the \fBrpp_()\fP API"
4607 .IX Subsection "(Re)writing a PP function using the rpp_() API"
4608 Wrapping a PP function has a performance overhead, and is there mainly as
4609 a temporary crutch. Eventually, PP functions should be updated to use
4610 \&\fBrpp_()\fR functions, and any new PP functions should be written this way from
4611 scratch and thus not ever need wrapping.
4612 .PP
4613 A couple examples of core PP functions being converted can be seen in the
4614 commits \f(CW\*(C`v5.39.1\-304\-g205fcd8410\*(C'\fR and \f(CW\*(C`v5.39.1\-303\-g2fe263a83a\*(C'\fR, which
4615 demonstrate a unary and a binary op being converted (\fBpp_not()\fR and
4616 \&\fBpp_and()\fR).
4617 .PP
4618 The traditional PP stack API consisted of a \f(CW\*(C`dSP\*(C'\fR declaration, plus a
4619 number of macros to push, pop and extend the stack. A \fIvery simplified\fR
4620 \&\fBpp_add()\fR function might look something like:
4621 .PP
4622 .Vb 11
4623 \& PP(pp_add)
4624 \& {
4625 \& dSP;
4626 \& dTARGET;
4627 \& IV right = SvIV(POPs);
4628 \& IV left = SvIV(POPs);
4629 \& TARGi(left + right, 1);
4630 \& PUSHs(TARG);
4631 \& PUTBACK;
4632 \& return NORMAL;
4633 \& }
4634 .Ve
4635 .PP
4636 which expands to something like:
4637 .PP
4638 .Vb 10
4639 \& {
4640 \& SV **sp = PL_stack_sp;
4641 \& SV *targ = PAD_SV(PL_op\->op_targ);
4642 \& IV right = SvIV(*sp\-\-);
4643 \& IV left = SvIV(*sp\-\-);
4644 \& sv_setiv(targ, left + right);
4645 \& *++sp = targ;
4646 \& PL_stack_sp = sp;
4647 \& return PL_op\->op_next;
4648 \& }
4649 .Ve
4650 .PP
4651 The whole \f(CW\*(C`dSP\*(C'\fR thing harks back to the days before decent optimising
4652 compilers. It was always error-prone, e.g. if you forgot a \f(CW\*(C`PUTBACK\*(C'\fR or
4653 \&\f(CW\*(C`SPAGAIN\*(C'\fR. The new API always just accesses \f(CW\*(C`PL_stack_sp\*(C'\fR directly. In
4654 fact the first step of upgrading a PP function is always to remove the
4655 \&\f(CW\*(C`dSP\*(C'\fR declaration. This has the happy side effect that any old-style
4656 macros left in the pp function which implicitly use \f(CW\*(C`sp\*(C'\fR will become
4657 compile errors. The existence of a \f(CW\*(C`dSP\*(C'\fR somewhere in core is a good sign
4658 that that function still needs updating.
4659 .PP
4660 An obvious question is: why not just modify the definitions of the \fBPUSHs()\fR
4661 etc macros to modify reference counts on RC builds? The basic problem is
4662 that an SV may now be kept alive only by a single reference count from
4663 the stack (formerly, they tended to be on the TEMPs stack too). So in code
4664 like:
4665 .PP
4666 .Vb 2
4667 \& SV *sv = POPs;
4668 \& IV i = SvIV(sv);
4669 .Ve
4670 .PP
4671 including an \fBSvREFCNT_dec()\fR in the \f(CW\*(C`POPs\*(C'\fR macro definition would cause
4672 \&\f(CW\*(C`sv\*(C'\fR to be freed immediately, before its integer value can be read.
4673 .PP
4674 A potential issue with the new regime is that perl can croak at basically
4675 any point in execution (e.g. the \fBSvIV()\fR above might call \fBFETCH()\fR on a tied
4676 variable which then croaks). Thus at all times, the RC of each SV must be
4677 properly accounted for. In the example above, a naive approach to avoiding
4678 a premature free of \f(CW\*(C`sv\*(C'\fR might be:
4679 .PP
4680 .Vb 3
4681 \& SV *sv = *PL_stack_sp\-\-;
4682 \& IV i = SvIV(sv);
4683 \& SvREFCNT_dec(sv); // got i, so ok to free sv now
4684 .Ve
4685 .PP
4686 but that means that \f(CW\*(C`sv\*(C'\fR leaks if \fBSvIV()\fR triggers a croak.
4687 .PP
4688 To avoid that, the new regime has the general outline that arguments are
4689 left on the stack \fIuntil they are finished with\fR, then removed and their
4690 reference count adjusted at that point. With the new API, the \fBpp_add()\fR
4691 function looks something like:
4692 .PP
4693 .Vb 8
4694 \& {
4695 \& dTARGET;
4696 \& IV right = SvIV(PL_stack_sp[ 0]); // NB: arguments left on stack
4697 \& IV left = SvIV(PL_stack_sp[\-1]);
4698 \& TARGi(left + right, 1);
4699 \& rpp_replace_2_1(targ);
4700 \& return NORMAL;
4701 \& }
4702 .Ve
4703 .PP
4704 The \fBrpp_replace_2_1()\fR function pops two values off the stack and pushes
4705 one new value on, while adjusting reference counts as appropriate
4706 (depending on whether built with \f(CW\*(C`PERL_RC_STACK\*(C'\fR or not).
4707 .PP
4708 The \fBrpp_()\fR functions in the new API will be described in detail below, but
4709 in summary:
4710 .PP
4711 .Vb 2
4712 \& new function approximate old equivant
4713 \& \-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
4714 \&
4715 \& rpp_extend(n) EXTEND(SP, n)
4716 \&
4717 \& rpp_push_1(sv) PUSHs(sv)
4718 \& rpp_push_2(sv1, sv2)) PUSHs(sv1); PUSHs(sv2)
4719 \& rpp_xpush_1(sv) XPUSHs(sv)
4720 \& rpp_xpush_2(sv1, sv2)) EXTEND(SP,2); PUSHs(sv1); PUSHs(sv2);
4721 \&
4722 \& rpp_push_1_norc(sv) mPUSHs(sv) // on RC bulds, skips RC++;
4723 \& // on non\-RC builds, mortalises
4724 \& rpp_popfree_1() (void)POPs;
4725 \& rpp_popfree_2() (void)POPs; (void)POPs;
4726 \& rpp_popfree_to(svp) PL_stack_sp = svp;
4727 \& rpp_obliterate_stack_to(ix) // see description below
4728 \&
4729 \& sv = rpp_pop_1_norc() sv = SvREFCNT_inc(POPs)
4730 \&
4731 \& rpp_replace_1_1(sv) (void)POPs; PUSHs(sv);
4732 \& rpp_replace_2_1(sv) (void)POPs; (void)POPs; PUSHs(sv);
4733 \& rpp_replace_at(sp, sv) *sp = sv;
4734 \& rpp_replace_at_norc(sp, sv) *sp = sv_2mortal(sv);
4735 \&
4736 \& rpp_context(mark, gimme,
4737 \& extra) SP \-= extra;
4738 \& // impose void/scalar/list context on return args
4739 \& SP = (gimme == G_VOID) ? mark : ....
4740 \&
4741 \& rpp_try_AMAGIC_1() tryAMAGICun_MG()
4742 \& rpp_try_AMAGIC_2() tryAMAGICbin_MG()
4743 \&
4744 \& rpp_is_lone(sv) SvTEMP(sv) && SvREFCNT(sv) == 1
4745 \& rpp_stack_is_rc() no equivalent
4746 \&
4747 \& rpp_invoke_xs(cv) CvXSUB(cv)(aTHX_ cv);
4748 \&
4749 \&
4750 \& (no replacement) dATARGET // just write the macro body in full
4751 .Ve
4752 .PP
4753 There are also some \f(CW\*(C`_NN\*(C'\fR variants which assume that any items being
4754 removed from the stack are non-NULL, and so are slightly more efficient:
4755 .PP
4756 .Vb 3
4757 \& rpp_popfree_1_NN()
4758 \& rpp_popfree_2_NN()
4759 \& rpp_popfree_to_NN(svp)
4760 \&
4761 \& rpp_replace_1_1_NN(sv)
4762 \& rpp_replace_2_1_NN(sv)
4763 \& rpp_replace_at_NN(sp, sv)
4764 \& rpp_replace_at_norc_NN(sp, sv)
4765 .Ve
4766 .PP
4767 There are also a few \f(CW\*(C`_IMM\*(C'\fR variants, which expect the single pushed or
4768 replacement value to be an immortal, such as \f(CW&PL_sv_undef\fR \- this skips
4769 incrementing the ref count of the immortal SV. It doesn't matter if the
4770 ref count of the SV prematurely reaches zero, as \fBsv_free2()\fR will just
4771 resurrect it. Not every variant is provided; if a suitable one
4772 doesn't exist, just using a standard \f(CW\*(C`_1\*(C'\fR version is fine, albeit
4773 slightly slower.
4774 .PP
4775 .Vb 4
4776 \& rpp_push_IMM(&PL_sv_undef)
4777 \& rpp_xpush_IMM(&PL_sv_zero)
4778 \& rpp_replace_1_IMM_NN(&PL_sv_yes)
4779 \& rpp_replace_2_IMM_NN(&PL_sv_no)
4780 .Ve
4781 .PP
4782 Other new C and perl functions related to reference-counted stacks are:
4783 .PP
4784 .Vb 3
4785 \& push_stackinfo(type,rc) PUSHSTACKi(type)
4786 \& pop_stackinfo() POPSTACK()
4787 \& switch_argstack(to) SWITCHSTACK(from,to)
4788 \&
4789 \& (Internals::stack_refcounted() & 1) # perl built with PERL_RC_STACK
4790 .Ve
4791 .PP
4792 Some of these new functions are trivial, but should be used in preference
4793 to writing direct code because they will work on both RC and non-RC
4794 builds, and may do extra checks and assertions on \f(CW\*(C`DEBUGGING\*(C'\fR builds.
4795 .PP
4796 Note that \fBrpp_popfree_1()\fR etc aren't direct replacements for \f(CW\*(C`POPs\*(C'\fR. The
4797 \&\fBrpp_()\fR variants don't return a value and are intended to be called when
4798 the SV is finished with. So
4799 .PP
4800 .Vb 2
4801 \& SV *sv = POPs;
4802 \& ... do stuff with sv ...
4803 .Ve
4804 .PP
4805 becomes
4806 .PP
4807 .Vb 3
4808 \& SV *sv = *PL_stack_sp;
4809 \& ... do stuff with sv ...
4810 \& rpp_popfree_1(); /* does SvREFCNT_dec(*PL_stack_sp\-\-) */
4811 .Ve
4812 .PP
4813 The \fBrpp_replace_M_N()\fR functions are shortcuts for popping and freeing \f(CW\*(C`M\*(C'\fR
4814 items then pushing and bumping up the RCs of \f(CW\*(C`N\*(C'\fR items. Note that they
4815 handle edge cases such as an old and new SV being the same.
4816 .PP
4817 rpp_replace_at(sp, sv) is similar to \fBrpp_replace_1_1()\fR, except that
4818 it replaces an SV at an address in the stack rather than at the top.
4819 .PP
4820 rpp_replace_at_norc(sp, sv) is similar to \fBrpp_replace_at()\fR, except that
4821 it assumes that \f(CW\*(C`sv\*(C'\fR already has a bumped reference count. So, a bit
4822 like \fBrpp_push_1_norc()\fR (see below), it doesn't bother increasing \f(CW\*(C`sv\*(C'\fR's
4823 reference count, or on non-RC builds it mortalises it instead.
4824 .PP
4825 rpp_popfree_to(svp) is designed to replace code like
4826 .PP
4827 .Vb 1
4828 \& PL_stack_sp = PL_stack_base + cx\->blk_oldsp;
4829 .Ve
4830 .PP
4831 which typically appears in list ops or scope exits when the arguments are
4832 finished with. Left unaltered, all the SVs above \f(CW\*(C`oldsp\*(C'\fR would leak. The
4833 new approach is
4834 .PP
4835 .Vb 1
4836 \& rpp_popfree_to(PL_stack_base + cx\->blk_oldsp);
4837 .Ve
4838 .PP
4839 There is a rarely-used variant of this, \fBrpp_obliterate_stack_to()\fR, which
4840 pops the stack back to the specified index regardless of the current RC
4841 state of the stack. So for example if the stack is split, it will only
4842 adjust the RCs of any SVs which are below the split point, while
4843 \&\fBrpp_popfree_to()\fR would mindlessly free \fIall\fR SVs (on RC builds anyway).
4844 For normal PP functions you should only ever use \fBrpp_popfree_to()\fR, which
4845 is faster.
4846 .PP
4847 There are no new equivalents for all the convenience macros like \fBPOPi()\fR
4848 and (shudder) \fBdPOPPOPiirl()\fR. These should be replaced with the \fBrpp_()\fR
4849 functions above and with the conversions and variable declarations being
4850 made explicit, e.g. \fBdPOPPOPiirl()\fR becomes:
4851 .PP
4852 .Vb 3
4853 \& IV right = SvIV(PL_stack_sp[ 0]);
4854 \& IV left = SvIV(PL_stack_sp[\-1]);
4855 \& rpp_popfree_2();
4856 .Ve
4857 .PP
4858 A couple of the \fBrpp_()\fR functions with \f(CW\*(C`norc\*(C'\fR in their names don't adjust
4859 the reference count on RC builds (but, conversely, do on non-RC builds).
4860 .PP
4861 rpp_push_1_norc(sv) does a simple \f(CW\*(C`*++PL_stack_sp = sv\*(C'\fR on RC builds. It
4862 is typically used to "root" a newly-created SV, which already has an RC of
4863 1. On non-RC builds it mortalises the SV instead. So for example, code
4864 which used to look like
4865 .PP
4866 .Vb 1
4867 \& mPUSHs(newSViv(i));
4868 .Ve
4869 .PP
4870 and which expanded to the equivalent of:
4871 .PP
4872 .Vb 1
4873 \& PUSHs(sv_2mortal(newSViv(i));
4874 .Ve
4875 .PP
4876 should be rewritten as:
4877 .PP
4878 .Vb 1
4879 \& rpp_push_1_norc(newSViv(i));
4880 .Ve
4881 .PP
4882 This is because \fBnewSViv()\fR and similar create a new SV with a reference
4883 count one too high (1 rather than 0). This count is then "donated" to the
4884 stack by pushing it. Conversely on non-RC builds, the count is donated to
4885 the TEMPs stack.
4886 .PP
4887 Similarly, on RC builds, \f(CW\*(C`sv = rpp_pop_1_norc()\*(C'\fR does a simple
4888 \&\f(CW\*(C`sv = *PL_stack_sv\-\-\*(C'\fR without adjusting the reference count, while on
4889 non-RC builds it actually increments the SV's reference count. It is
4890 intended for cases where you immediately want to increment the reference
4891 count again after popping, e.g. where the SV is to be immediately embedded
4892 somewhere. For example this code:
4893 .PP
4894 .Vb 4
4895 \& SV *sv = PL_stack_sp[0];
4896 \& SvREFCNT_inc(sv);
4897 \& av_store(av, i, sv); /* in real life should check return value */
4898 \& rpp_popfree_1();
4899 .Ve
4900 .PP
4901 can be more efficiently written as
4902 .PP
4903 .Vb 1
4904 \& av_store(av, i, rpp_pop_1_norc());
4905 .Ve
4906 .PP
4907 By using this function, the code works correctly on both RC and non-RC
4908 builds.
4909 .PP
4910 A common operation on list ops is to impose void, scalar or list context
4911 on the return arguments, possibly discarding all, or all except one, of
4912 them. rpp_context(mark, gimme, extra) does this. As a first step (for
4913 convenience and efficiency) it notionally pops \f(CW\*(C`extra\*(C'\fR args off the
4914 stack. Then for list context, leaves things as is. For void context, the
4915 stack pointer is reset to mark, and everything above is popped. For
4916 scalar, the top argument (or &PL_sv_undef) is moved from the top to
4917 mark+1 and everything above is discarded.
4918 .PP
4919 The macros which appear at the start of many PP functions to check for
4920 unary or binary op overloading (among other things) have been replaced
4921 with \fBrpp_try_AMAGIC_1()\fR and \fB_2()\fR inline functions, which now rely on the
4922 calling PP function to choose whether to return immediately rather than
4923 the return being hidden away in the macro.
4924 .PP
4925 The \fBrpp_invoke_xs()\fR function calls the XS function associated with the CV,
4926 but may do so via a wrapper function to adjust the stack as necessary.
4927 .PP
4928 In the spirit of hiding away less in macros, \f(CW\*(C`dATARGET\*(C'\fR hasn't been given
4929 a replacement; where its effect is needed, it is now written out in full;
4930 see \fBpp_add()\fR for an example.
4931 .PP
4932 Finally, a couple of \fBrpp()\fR functions provide information rather than
4933 manipulate the stack.
4934 .PP
4935 rpp_is_lone(sv) indicates whether \f(CW\*(C`sv\*(C'\fR, assumed to be still on the stack,
4936 it kept alive only by a single reference-counted pointer from the argument
4937 and/or temps stacks, and thus is a candidate for some optimisations (like
4938 skipping the copying of return arguments from a subroutine call).
4939 .PP
4940 \&\fBrpp_stack_is_rc()\fR indicates whether the current stack is currently
4941 reference-counted. It's used mainly in a few places like \fBcall_sv()\fR which
4942 can be called from anywhere, and thus have to deal with both cases.
4943 .PP
4944 So for example, rather than using \fBrpp_xpush_1()\fR, \fBcall_sv()\fR has lines like:
4945 .PP
4946 .Vb 6
4947 \& rpp_extend(1);
4948 \& *++PL_stack_sp = sv;
4949 \& #ifdef PERL_RC_STACK
4950 \& if (rpp_stack_is_rc())
4951 \& SvREFCNT_inc_simple_void_NN(sv);
4952 \& #endif
4953 .Ve
4954 .PP
4955 which works on both standard builds and RC builds, and works whether
4956 \&\fBcall_sv()\fR is called from a standard PP function (\fBrpp_stack_is_rc()\fR is
4957 true) or from a wrapped PP or XS function (\fBrpp_stack_is_rc()\fR is false).
4958 Note that you're unlikely to need to use this function, as in most places,
4959 such as PP or XS functions, it is always RC or non-RC respectively. In
4960 fact on debugging builds under \f(CW\*(C`PERL_RC_STACK\*(C'\fR, \fBPUSHs()\fR and similar
4961 macros include an \f(CW\*(C`assert(!rpp_stack_is_rc())\*(C'\fR, while \fBrpp_push_1()\fR and
4962 similar functions have \f(CW\*(C`assert(rpp_stack_is_rc())\*(C'\fR.
4963 .PP
4964 The macros for pushing new stackinfos have been replaced with inline
4965 functions which don't rely on \f(CW\*(C`dSP\*(C'\fR being in scope, and which have less
4966 ambiguous names: they make it clear that a new \fIstackinfo\fR is being
4967 pushed, rather than just some sort of \fIstack\fR. \fBpush_stackinfo()\fR also has
4968 a boolean argument indicating whether the new argument stack should be
4969 reference-counted or not. For backwards compatibility, PUSHSTACKi(type) is
4970 defined to be push_stackinfo(type, 0).
4971 .PP
4972 Some test scripts check for things like leaks by testing that the
4973 reference count of a particular variable has an expected value. If this
4974 is different on a perl built with \f(CW\*(C`PERL_RC_STACK\*(C'\fR, then the perl
4975 function \fBInternals::stack_refcounted()\fR can be used. This returns an
4976 integer, the lowest bit of which indicates that perl was built with
4977 \&\f(CW\*(C`PERL_RC_STACK\*(C'\fR. Other bits are reserved for future use and should be
4978 masked out.
4979 .SH "Slab-based operator allocation"
4980 .IX Header "Slab-based operator allocation"
4981 \&\fBNote:\fR this section describes a non-public internal API that is subject
4982 to change without notice.
4983 .PP
4984 Perl's internal error-handling mechanisms implement \f(CW\*(C`die\*(C'\fR (and its internal
4985 equivalents) using longjmp. If this occurs during lexing, parsing or
4986 compilation, we must ensure that any ops allocated as part of the compilation
4987 process are freed. (Older Perl versions did not adequately handle this
4988 situation: when failing a parse, they would leak ops that were stored in
4989 C \f(CW\*(C`auto\*(C'\fR variables and not linked anywhere else.)
4990 .PP
4991 To handle this situation, Perl uses \fIop slabs\fR that are attached to the
4992 currently-compiling CV. A slab is a chunk of allocated memory. New ops are
4993 allocated as regions of the slab. If the slab fills up, a new one is created
4994 (and linked from the previous one). When an error occurs and the CV is freed,
4995 any ops remaining are freed.
4996 .PP
4997 Each op is preceded by two pointers: one points to the next op in the slab, and
4998 the other points to the slab that owns it. The next-op pointer is needed so
4999 that Perl can iterate over a slab and free all its ops. (Op structures are of
5000 different sizes, so the slab's ops can't merely be treated as a dense array.)
5001 The slab pointer is needed for accessing a reference count on the slab: when
5002 the last op on a slab is freed, the slab itself is freed.
5003 .PP
5004 The slab allocator puts the ops at the end of the slab first. This will tend to
5005 allocate the leaves of the op tree first, and the layout will therefore
5006 hopefully be cache-friendly. In addition, this means that there's no need to
5007 store the size of the slab (see below on why slabs vary in size), because Perl
5008 can follow pointers to find the last op.
5009 .PP
5010 It might seem possible to eliminate slab reference counts altogether, by having
5011 all ops implicitly attached to \f(CW\*(C`PL_compcv\*(C'\fR when allocated and freed when the
5012 CV is freed. That would also allow \f(CW\*(C`op_free\*(C'\fR to skip \f(CW\*(C`FreeOp\*(C'\fR altogether, and
5013 thus free ops faster. But that doesn't work in those cases where ops need to
5014 survive beyond their CVs, such as re-evals.
5015 .PP
5016 The CV also has to have a reference count on the slab. Sometimes the first op
5017 created is immediately freed. If the reference count of the slab reaches 0,
5018 then it will be freed with the CV still pointing to it.
5019 .PP
5020 CVs use the \f(CW\*(C`CVf_SLABBED\*(C'\fR flag to indicate that the CV has a reference count
5021 on the slab. When this flag is set, the slab is accessible via \f(CW\*(C`CvSTART\*(C'\fR when
5022 \&\f(CW\*(C`CvROOT\*(C'\fR is not set, or by subtracting two pointers \f(CW\*(C`(2*sizeof(I32 *))\*(C'\fR from
5023 \&\f(CW\*(C`CvROOT\*(C'\fR when it is set. The alternative to this approach of sneaking the slab
5024 into \f(CW\*(C`CvSTART\*(C'\fR during compilation would be to enlarge the \f(CW\*(C`xpvcv\*(C'\fR struct by
5025 another pointer. But that would make all CVs larger, even though slab-based op
5026 freeing is typically of benefit only for programs that make significant use of
5027 string eval.
5028 .PP
5029 When the \f(CW\*(C`CVf_SLABBED\*(C'\fR flag is set, the CV takes responsibility for freeing
5030 the slab. If \f(CW\*(C`CvROOT\*(C'\fR is not set when the CV is freed or undeffed, it is
5031 assumed that a compilation error has occurred, so the op slab is traversed and
5032 all the ops are freed.
5033 .PP
5034 Under normal circumstances, the CV forgets about its slab (decrementing the
5035 reference count) when the root is attached. So the slab reference counting that
5036 happens when ops are freed takes care of freeing the slab. In some cases, the
5037 CV is told to forget about the slab (\f(CW\*(C`cv_forget_slab\*(C'\fR) precisely so that the
5038 ops can survive after the CV is done away with.
5039 .PP
5040 Forgetting the slab when the root is attached is not strictly necessary, but
5041 avoids potential problems with \f(CW\*(C`CvROOT\*(C'\fR being written over. There is code all
5042 over the place, both in core and on CPAN, that does things with \f(CW\*(C`CvROOT\*(C'\fR, so
5043 forgetting the slab makes things more robust and avoids potential problems.
5044 .PP
5045 Since the CV takes ownership of its slab when flagged, that flag is never
5046 copied when a CV is cloned, as one CV could free a slab that another CV still
5047 points to, since forced freeing of ops ignores the reference count (but asserts
5048 that it looks right).
5049 .PP
5050 To avoid slab fragmentation, freed ops are marked as freed and attached to the
5051 slab's freed chain (an idea stolen from DBM::Deep). Those freed ops are reused
5052 when possible. Not reusing freed ops would be simpler, but it would result in
5053 significantly higher memory usage for programs with large \f(CW\*(C`if (DEBUG) {...}\*(C'\fR
5054 blocks.
5055 .PP
5056 \&\f(CW\*(C`SAVEFREEOP\*(C'\fR is slightly problematic under this scheme. Sometimes it can cause
5057 an op to be freed after its CV. If the CV has forcibly freed the ops on its
5058 slab and the slab itself, then we will be fiddling with a freed slab. Making
5059 \&\f(CW\*(C`SAVEFREEOP\*(C'\fR a no-op doesn't help, as sometimes an op can be savefreed when
5060 there is no compilation error, so the op would never be freed. It holds
5061 a reference count on the slab, so the whole slab would leak. So \f(CW\*(C`SAVEFREEOP\*(C'\fR
5062 now sets a special flag on the op (\f(CW\*(C`\->op_savefree\*(C'\fR). The forced freeing of
5063 ops after a compilation error won't free any ops thus marked.
5064 .PP
5065 Since many pieces of code create tiny subroutines consisting of only a few ops,
5066 and since a huge slab would be quite a bit of baggage for those to carry
5067 around, the first slab is always very small. To avoid allocating too many
5068 slabs for a single CV, each subsequent slab is twice the size of the previous.
5069 .PP
5070 Smartmatch expects to be able to allocate an op at run time, run it, and then
5071 throw it away. For that to work the op is simply malloced when \f(CW\*(C`PL_compcv\*(C'\fR hasn't
5072 been set up. So all slab-allocated ops are marked as such (\f(CW\*(C`\->op_slabbed\*(C'\fR),
5073 to distinguish them from malloced ops.
5074 .SH AUTHORS
5075 .IX Header "AUTHORS"
5076 Until May 1997, this document was maintained by Jeff Okamoto
5077 <[email protected]>. It is now maintained as part of Perl
5078 itself by the Perl 5 Porters <perl5\[email protected]>.
5079 .PP
5080 With lots of help and suggestions from Dean Roehrich, Malcolm Beattie,
5081 Andreas Koenig, Paul Hudson, Ilya Zakharevich, Paul Marquess, Neil
5082 Bowers, Matthew Green, Tim Bunce, Spider Boardman, Ulrich Pfeifer,
5083 Stephen McCamant, and Gurusamy Sarathy.
5084 .SH "SEE ALSO"
5085 .IX Header "SEE ALSO"
5086 perlapi, perlintern, perlxs, perlembed