Debian Codegraph @

upstream/fedora-44/man7/pipe.7

raw
1 .\" Copyright (C) 2005 Michael Kerrisk <[email protected]>
2 .\"
3 .\" SPDX-License-Identifier: Linux-man-pages-copyleft
4 .\"
5 .TH pipe 7 2024-08-29 "Linux man-pages 6.13"
6 .SH NAME
7 pipe \- overview of pipes and FIFOs
8 .SH DESCRIPTION
9 Pipes and FIFOs (also known as named pipes)
10 provide a unidirectional interprocess communication channel.
11 A pipe has a
12 .I read end
13 and a
14 .IR "write end" .
15 Data written to the write end of a pipe can be read
16 from the read end of the pipe.
17 .P
18 A pipe is created using
19 .BR pipe (2),
20 which creates a new pipe and returns two file descriptors,
21 one referring to the read end of the pipe,
22 the other referring to the write end.
23 Pipes can be used to create a communication channel between related
24 processes; see
25 .BR pipe (2)
26 for an example.
27 .P
28 A FIFO (short for First In First Out) has a name within the filesystem
29 (created using
30 .BR mkfifo (3)),
31 and is opened using
32 .BR open (2).
33 Any process may open a FIFO, assuming the file permissions allow it.
34 The read end is opened using the
35 .B O_RDONLY
36 flag; the write end is opened using the
37 .B O_WRONLY
38 flag.
39 See
40 .BR fifo (7)
41 for further details.
42 .IR Note :
43 although FIFOs have a pathname in the filesystem,
44 I/O on FIFOs does not involve operations on the underlying device
45 (if there is one).
46 .SS I/O on pipes and FIFOs
47 The only difference between pipes and FIFOs is the manner in which
48 they are created and opened.
49 Once these tasks have been accomplished,
50 I/O on pipes and FIFOs has exactly the same semantics.
51 .P
52 If a process attempts to read from an empty pipe, then
53 .BR read (2)
54 will block until data is available.
55 If a process attempts to write to a full pipe (see below), then
56 .BR write (2)
57 blocks until sufficient data has been read from the pipe
58 to allow the write to complete.
59 .P
60 Nonblocking I/O is possible by using the
61 .BR fcntl (2)
62 .B F_SETFL
63 operation to enable the
64 .B O_NONBLOCK
65 open file status flag or by opening a
66 .BR fifo (7)
67 with
68 .BR O_NONBLOCK .
69 If any process has the pipe open for writing, reads fail with
70 .BR EAGAIN ;
71 otherwise\[em]with no potential writers\[em]reads succeed and return empty.
72 .P
73 The communication channel provided by a pipe is a
74 .IR "byte stream" :
75 there is no concept of message boundaries.
76 .P
77 If all file descriptors referring to the write end of a pipe
78 have been closed, then an attempt to
79 .BR read (2)
80 from the pipe will see end-of-file
81 .RB ( read (2)
82 will return 0).
83 If all file descriptors referring to the read end of a pipe
84 have been closed, then a
85 .BR write (2)
86 will cause a
87 .B SIGPIPE
88 signal to be generated for the calling process.
89 If the calling process is ignoring this signal, then
90 .BR write (2)
91 fails with the error
92 .BR EPIPE .
93 An application that uses
94 .BR pipe (2)
95 and
96 .BR fork (2)
97 should use suitable
98 .BR close (2)
99 calls to close unnecessary duplicate file descriptors;
100 this ensures that end-of-file and
101 .BR SIGPIPE / EPIPE
102 are delivered when appropriate.
103 .P
104 It is not possible to apply
105 .BR lseek (2)
106 to a pipe.
107 .SS Pipe capacity
108 A pipe has a limited capacity.
109 If the pipe is full, then a
110 .BR write (2)
111 will block or fail, depending on whether the
112 .B O_NONBLOCK
113 flag is set (see below).
114 Different implementations have different limits for the pipe capacity.
115 Applications should not rely on a particular capacity:
116 an application should be designed so that a reading process consumes data
117 as soon as it is available,
118 so that a writing process does not remain blocked.
119 .P
120 Before Linux 2.6.11, the capacity of a pipe was the same as
121 the system page size (e.g., 4096 bytes on i386).
122 Since Linux 2.6.11, the pipe capacity is 16 pages
123 (i.e., 65,536 bytes in a system with a page size of 4096 bytes).
124 Since Linux 2.6.35, the default pipe capacity is 16 pages,
125 but the capacity can be queried and set using the
126 .BR fcntl (2)
127 .B F_GETPIPE_SZ
128 and
129 .B F_SETPIPE_SZ
130 operations.
131 See
132 .BR fcntl (2)
133 for more information.
134 Since Linux 4.5, the default pipe capacity is lower than 16 pages when the
135 .I pipe\-user\-pages\-soft
136 limit is exceeded.
137 .P
138 The following
139 .BR ioctl (2)
140 operation, which can be applied to a file descriptor
141 that refers to either end of a pipe,
142 places a count of the number of unread bytes in the pipe in the
143 .I int
144 buffer pointed to by the final argument of the call:
145 .P
146 .in +4n
147 .EX
148 ioctl(fd, FIONREAD, &nbytes);
149 .EE
150 .in
151 .P
152 The
153 .B FIONREAD
154 operation is not specified in any standard,
155 but is provided on many implementations.
156 .\"
157 .SS /proc files
158 On Linux, the following files control how much memory can be used for pipes:
159 .TP
160 .IR /proc/sys/fs/pipe\-max\-pages " (only in Linux 2.6.34)"
161 .\" commit b492e95be0ae672922f4734acf3f5d35c30be948
162 An upper limit, in pages, on the capacity that an unprivileged user
163 (one without the
164 .B CAP_SYS_RESOURCE
165 capability)
166 can set for a pipe.
167 .IP
168 The default value for this limit is 16 times the default pipe capacity
169 (see above); the lower limit is two pages.
170 .IP
171 This interface was removed in Linux 2.6.35, in favor of
172 .IR /proc/sys/fs/pipe\-max\-size .
173 .TP
174 .IR /proc/sys/fs/pipe\-max\-size " (since Linux 2.6.35)"
175 .\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac
176 The maximum size (in bytes) of individual pipes that can be set
177 .\" This limit is not checked on pipe creation, where the capacity is
178 .\" always PIPE_DEF_BUFS, regardless of pipe-max-size
179 by users without the
180 .B CAP_SYS_RESOURCE
181 capability.
182 The value assigned to this file may be rounded upward,
183 to reflect the value actually employed for a convenient implementation.
184 To determine the rounded-up value,
185 display the contents of this file after assigning a value to it.
186 .IP
187 The default value for this file is 1048576 (1\ MiB).
188 The minimum value that can be assigned to this file is the system page size.
189 Attempts to set a limit less than the page size cause
190 .BR write (2)
191 to fail with the error
192 .BR EINVAL .
193 .IP
194 Since Linux 4.9,
195 .\" commit 086e774a57fba4695f14383c0818994c0b31da7c
196 the value on this file also acts as a ceiling on the default capacity
197 of a new pipe or newly opened FIFO.
198 .TP
199 .IR /proc/sys/fs/pipe\-user\-pages\-hard " (since Linux 4.5)"
200 .\" commit 759c01142a5d0f364a462346168a56de28a80f52
201 The hard limit on the total size (in pages) of all pipes created or set by
202 a single unprivileged user (i.e., one with neither the
203 .B CAP_SYS_RESOURCE
204 nor the
205 .B CAP_SYS_ADMIN
206 capability).
207 So long as the total number of pages allocated to pipe buffers
208 for this user is at this limit,
209 attempts to create new pipes will be denied,
210 and attempts to increase a pipe's capacity will be denied.
211 .IP
212 When the value of this limit is zero (which is the default),
213 no hard limit is applied.
214 .\" The default was chosen to avoid breaking existing applications that
215 .\" make intensive use of pipes (e.g., for splicing).
216 .TP
217 .IR /proc/sys/fs/pipe\-user\-pages\-soft " (since Linux 4.5)"
218 .\" commit 759c01142a5d0f364a462346168a56de28a80f52
219 The soft limit on the total size (in pages) of all pipes created or set by
220 a single unprivileged user (i.e., one with neither the
221 .B CAP_SYS_RESOURCE
222 nor the
223 .B CAP_SYS_ADMIN
224 capability).
225 So long as the total number of pages allocated to pipe buffers
226 for this user is at this limit,
227 individual pipes created by a user will be
228 limited to two pages
229 (one page before Linux 5.14),
230 and attempts to increase a pipe's capacity will be denied.
231 .IP
232 When the value of this limit is zero, no soft limit is applied.
233 The default value for this file is 16384,
234 which permits creating up to 1024 pipes with the default capacity.
235 .P
236 Before Linux 4.9, some bugs affected the handling of the
237 .I pipe\-user\-pages\-soft
238 and
239 .I pipe\-user\-pages\-hard
240 limits; see BUGS.
241 .\"
242 .SS PIPE_BUF
243 POSIX.1 says that writes of less than
244 .B PIPE_BUF
245 bytes must be atomic: the output data is written to the pipe as a
246 contiguous sequence.
247 Writes of more than
248 .B PIPE_BUF
249 bytes may be nonatomic: the kernel may interleave the data
250 with data written by other processes.
251 POSIX.1 requires
252 .B PIPE_BUF
253 to be at least 512 bytes.
254 (On Linux,
255 .B PIPE_BUF
256 is 4096 bytes.)
257 The precise semantics depend on whether the file descriptor is nonblocking
258 .RB ( O_NONBLOCK ),
259 whether there are multiple writers to the pipe, and on
260 .IR n ,
261 the number of bytes to be written:
262 .TP
263 \fBO_NONBLOCK\fP disabled, \fIn\fP <= \fBPIPE_BUF\fP
264 All
265 .I n
266 bytes are written atomically;
267 .BR write (2)
268 may block if there is not room for
269 .I n
270 bytes to be written immediately
271 .TP
272 \fBO_NONBLOCK\fP enabled, \fIn\fP <= \fBPIPE_BUF\fP
273 If there is room to write
274 .I n
275 bytes to the pipe, then
276 .BR write (2)
277 succeeds immediately, writing all
278 .I n
279 bytes; otherwise
280 .BR write (2)
281 fails, with
282 .I errno
283 set to
284 .BR EAGAIN .
285 .TP
286 \fBO_NONBLOCK\fP disabled, \fIn\fP > \fBPIPE_BUF\fP
287 The write is nonatomic: the data given to
288 .BR write (2)
289 may be interleaved with
290 .BR write (2)s
291 by other process;
292 the
293 .BR write (2)
294 blocks until
295 .I n
296 bytes have been written.
297 .TP
298 \fBO_NONBLOCK\fP enabled, \fIn\fP > \fBPIPE_BUF\fP
299 If the pipe is full, then
300 .BR write (2)
301 fails, with
302 .I errno
303 set to
304 .BR EAGAIN .
305 Otherwise, from 1 to
306 .I n
307 bytes may be written (i.e., a "partial write" may occur;
308 the caller should check the return value from
309 .BR write (2)
310 to see how many bytes were actually written),
311 and these bytes may be interleaved with writes by other processes.
312 .SS Open file status flags
313 The only open file status flags that can be meaningfully applied to
314 a pipe or FIFO are
315 .B O_NONBLOCK
316 and
317 .BR O_ASYNC .
318 .P
319 Setting the
320 .B O_ASYNC
321 flag for the read end of a pipe causes a signal
322 .RB ( SIGIO
323 by default) to be generated when new input becomes available on the pipe.
324 The target for delivery of signals must be set using the
325 .BR fcntl (2)
326 .B F_SETOWN
327 command.
328 On Linux,
329 .B O_ASYNC
330 is supported for pipes and FIFOs only since Linux 2.6.
331 .SS Portability notes
332 On some systems (but not Linux), pipes are bidirectional:
333 data can be transmitted in both directions between the pipe ends.
334 POSIX.1 requires only unidirectional pipes.
335 Portable applications should avoid reliance on
336 bidirectional pipe semantics.
337 .SS BUGS
338 Before Linux 4.9, some bugs affected the handling of the
339 .I pipe\-user\-pages\-soft
340 and
341 .I pipe\-user\-pages\-hard
342 limits when using the
343 .BR fcntl (2)
344 .B F_SETPIPE_SZ
345 operation to change a pipe's capacity:
346 .\" These bugs where remedied by a series of patches, in particular,
347 .\" commit b0b91d18e2e97b741b294af9333824ecc3fadfd8 and
348 .\" commit a005ca0e6813e1d796a7422a7e31d8b8d6555df1
349 .IP (a) 5
350 When increasing the pipe capacity, the checks against the soft and
351 hard limits were made against existing consumption,
352 and excluded the memory required for the increased pipe capacity.
353 The new increase in pipe capacity could then push the total
354 memory used by the user for pipes (possibly far) over a limit.
355 (This could also trigger the problem described next.)
356 .IP
357 Starting with Linux 4.9,
358 the limit checking includes the memory required for the new pipe capacity.
359 .IP (b)
360 The limit checks were performed even when the new pipe capacity was
361 less than the existing pipe capacity.
362 This could lead to problems if a user set a large pipe capacity,
363 and then the limits were lowered, with the result that the user could
364 no longer decrease the pipe capacity.
365 .IP
366 Starting with Linux 4.9, checks against the limits
367 are performed only when increasing a pipe's capacity;
368 an unprivileged user can always decrease a pipe's capacity.
369 .IP (c)
370 The accounting and checking against the limits were done as follows:
371 .RS
372 .IP (1) 5
373 .PD 0
374 Test whether the user has exceeded the limit.
375 .IP (2)
376 Make the new pipe buffer allocation.
377 .IP (3)
378 Account new allocation against the limits.
379 .PD
380 .RE
381 .IP
382 This was racey.
383 Multiple processes could pass point (1) simultaneously,
384 and then allocate pipe buffers that were accounted for only in step (3),
385 with the result that the user's pipe buffer
386 allocation could be pushed over the limit.
387 .IP
388 Starting with Linux 4.9,
389 the accounting step is performed before doing the allocation,
390 and the operation fails if the limit would be exceeded.
391 .P
392 Before Linux 4.9, bugs similar to points (a) and (c) could also occur
393 when the kernel allocated memory for a new pipe buffer;
394 that is, when calling
395 .BR pipe (2)
396 and when opening a previously unopened FIFO.
397 .SH SEE ALSO
398 .BR mkfifo (1),
399 .BR dup (2),
400 .BR fcntl (2),
401 .BR open (2),
402 .BR pipe (2),
403 .BR poll (2),
404 .BR select (2),
405 .BR socketpair (2),
406 .BR splice (2),
407 .BR stat (2),
408 .BR tee (2),
409 .BR vmsplice (2),
410 .BR mkfifo (3),
411 .BR epoll (7),
412 .BR fifo (7)