Debian Codegraph @

upstream/debian-unstable/man8/tc-bpf.8

raw
1 .TH "BPF classifier and actions in tc" 8 "18 May 2015" "iproute2" "Linux"
2 .SH NAME
3 BPF \- BPF programmable classifier and actions for ingress/egress
4 queueing disciplines
5 .SH SYNOPSIS
6 .SS eBPF classifier (filter) or action:
7 .B tc filter ... bpf
8 [
9 .B object-file
10 OBJ_FILE ] [
11 .B section
12 CLS_NAME ] [
13 .B export
14 UDS_FILE ] [
15 .B verbose
16 ] [
17 .B direct-action
18 |
19 .B da
20 ] [
21 .B skip_hw
22 |
23 .B skip_sw
24 ] [
25 .B police
26 POLICE_SPEC ] [
27 .B action
28 ACTION_SPEC ] [
29 .B classid
30 CLASSID ]
31 .br
32 .B tc action ... bpf
33 [
34 .B object-file
35 OBJ_FILE ] [
36 .B section
37 CLS_NAME ] [
38 .B export
39 UDS_FILE ] [
40 .B verbose
41 ]
42
43 .SS cBPF classifier (filter) or action:
44 .B tc filter ... bpf
45 [
46 .B bytecode-file
47 BPF_FILE |
48 .B bytecode
49 BPF_BYTECODE ] [
50 .B police
51 POLICE_SPEC ] [
52 .B action
53 ACTION_SPEC ] [
54 .B classid
55 CLASSID ]
56 .br
57 .B tc action ... bpf
58 [
59 .B bytecode-file
60 BPF_FILE |
61 .B bytecode
62 BPF_BYTECODE ]
63
64 .SH DESCRIPTION
65
66 Extended Berkeley Packet Filter (
67 .B eBPF
68 ) and classic Berkeley Packet Filter
69 (originally known as BPF, for better distinction referred to as
70 .B cBPF
71 here) are both available as a fully programmable and highly efficient
72 classifier and actions. They both offer a minimal instruction set for
73 implementing small programs which can safely be loaded into the kernel
74 and thus executed in a tiny virtual machine from kernel space. An in-kernel
75 verifier guarantees that a specified program always terminates and neither
76 crashes nor leaks data from the kernel.
77
78 In Linux, it's generally considered that eBPF is the successor of cBPF.
79 The kernel internally transforms cBPF expressions into eBPF expressions and
80 executes the latter. Execution of them can be performed in an interpreter
81 or at setup time, they can be just-in-time compiled (JIT'ed) to run as
82 native machine code.
83 .PP
84 Currently, the eBPF JIT compiler is available for the following architectures:
85 .IP * 4
86 x86_64 (since Linux 3.18)
87 .PD 0
88 .IP *
89 arm64 (since Linux 3.18)
90 .IP *
91 s390 (since Linux 4.1)
92 .IP *
93 ppc64 (since Linux 4.8)
94 .IP *
95 sparc64 (since Linux 4.12)
96 .IP *
97 mips64 (since Linux 4.13)
98 .IP *
99 arm32 (since Linux 4.14)
100 .IP *
101 x86_32 (since Linux 4.18)
102 .PD
103 .PP
104 Whereas the following architectures have cBPF, but did not (yet) switch to eBPF
105 JIT support:
106 .IP * 4
107 ppc32
108 .PD 0
109 .IP *
110 sparc32
111 .IP *
112 mips32
113 .PD
114 .PP
115 eBPF's instruction set has similar underlying principles as the cBPF
116 instruction set, it however is modelled closer to the underlying
117 architecture to better mimic native instruction sets with the aim to
118 achieve a better run-time performance. It is designed to be JIT'ed with
119 a one to one mapping, which can also open up the possibility for compilers
120 to generate optimized eBPF code through an eBPF backend that performs
121 almost as fast as natively compiled code. Given that LLVM provides such
122 an eBPF backend, eBPF programs can therefore easily be programmed in a
123 subset of the C language. Other than that, eBPF infrastructure also comes
124 with a construct called "maps". eBPF maps are key/value stores that are
125 shared between multiple eBPF programs, but also between eBPF programs and
126 user space applications.
127
128 For the traffic control subsystem, classifier and actions that can be
129 attached to ingress and egress qdiscs can be written in eBPF or cBPF. The
130 advantage over other classifier and actions is that eBPF/cBPF provides the
131 generic framework, while users can implement their highly specialized use
132 cases efficiently. This means that the classifier or action written that
133 way will not suffer from feature bloat, and can therefore execute its task
134 highly efficient. It allows for non-linear classification and even merging
135 the action part into the classification. Combined with efficient eBPF map
136 data structures, user space can push new policies like classids into the
137 kernel without reloading a classifier, or it can gather statistics that
138 are pushed into one map and use another one for dynamically load balancing
139 traffic based on the determined load, just to provide a few examples.
140
141 .SH PARAMETERS
142 .SS object-file
143 points to an object file that has an executable and linkable format (ELF)
144 and contains eBPF opcodes and eBPF map definitions. The LLVM compiler
145 infrastructure with
146 .B clang(1)
147 as a C language front end is one project that supports emitting eBPF object
148 files that can be passed to the eBPF classifier (more details in the
149 .B EXAMPLES
150 section). This option is mandatory when an eBPF classifier or action is
151 to be loaded.
152
153 .SS section
154 is the name of the ELF section from the object file, where the eBPF
155 classifier or action resides. By default the section name for the
156 classifier is called "classifier", and for the action "action". Given
157 that a single object file can contain multiple classifier and actions,
158 the corresponding section name needs to be specified, if it differs
159 from the defaults.
160
161 .SS export
162 points to a Unix domain socket file. In case the eBPF object file also
163 contains a section named "maps" with eBPF map specifications, then the
164 map file descriptors can be handed off via the Unix domain socket to
165 an eBPF "agent" herding all descriptors after tc lifetime. This can be
166 some third party application implementing the IPC counterpart for the
167 import, that uses them for calling into
168 .B bpf(2)
169 system call to read out or update eBPF map data from user space, for
170 example, for monitoring purposes or to push down new policies.
171
172 .SS verbose
173 if set, it will dump the eBPF verifier output, even if loading the eBPF
174 program was successful. By default, only on error, the verifier log is
175 being emitted to the user.
176
177 .SS direct-action | da
178 instructs eBPF classifier to not invoke external TC actions, instead use the
179 TC actions return codes (\fBTC_ACT_OK\fR, \fBTC_ACT_SHOT\fR etc.) for
180 classifiers.
181
182 .SS skip_hw | skip_sw
183 hardware offload control flags. By default TC will try to offload
184 filters to hardware if possible.
185 .B skip_hw
186 explicitly disables the attempt to offload.
187 .B skip_sw
188 forces the offload and disables running the eBPF program in the kernel.
189 If hardware offload is not possible and this flag was set kernel will
190 report an error and filter will not be installed at all.
191
192 .SS police
193 is an optional parameter for an eBPF/cBPF classifier that specifies a
194 police in
195 .B tc(1)
196 which is attached to the classifier, for example, on an ingress qdisc.
197
198 .SS action
199 is an optional parameter for an eBPF/cBPF classifier that specifies a
200 subsequent action in
201 .B tc(1)
202 which is attached to a classifier.
203
204 .SS classid
205 .SS flowid
206 provides the default traffic control class identifier for this eBPF/cBPF
207 classifier. The default class identifier can also be overwritten by the
208 return code of the eBPF/cBPF program. A default return code of
209 .B -1
210 specifies the here provided default class identifier to be used. A return
211 code of the eBPF/cBPF program of 0 implies that no match took place, and
212 a return code other than these two will override the default classid. This
213 allows for efficient, non-linear classification with only a single eBPF/cBPF
214 program as opposed to having multiple individual programs for various class
215 identifiers which would need to reparse packet contents.
216
217 .SS bytecode
218 is being used for loading cBPF classifier and actions only. The cBPF bytecode
219 is directly passed as a text string in the form of
220 .B \(aqs,c t f k,c t f k,c t f k,...'
221 , where
222 .B s
223 denotes the number of subsequent 4-tuples. One such 4-tuple consists of
224 .B c t f k
225 decimals, where
226 .B c
227 represents the cBPF opcode,
228 .B t
229 the jump true offset target,
230 .B f
231 the jump false offset target and
232 .B k
233 the immediate constant/literal. There are various tools that generate code
234 in this loadable format, for example,
235 .B bpf_asm
236 that ships with the Linux kernel source tree under
237 .B tools/net/
238 , so it is certainly not expected to hack this by hand. The
239 .B bytecode
240 or
241 .B bytecode-file
242 option is mandatory when a cBPF classifier or action is to be loaded.
243
244 .SS bytecode-file
245 also being used to load a cBPF classifier or action. It's effectively the
246 same as
247 .B bytecode
248 only that the cBPF bytecode is not passed directly via command line, but
249 rather resides in a text file.
250
251 .SH EXAMPLES
252 .SS eBPF TOOLING
253 A full blown example including eBPF agent code can be found inside the
254 iproute2 source package under:
255 .B examples/bpf/
256
257 As prerequisites, the kernel needs to have the eBPF system call namely
258 .B bpf(2)
259 enabled and ships with
260 .B cls_bpf
261 and
262 .B act_bpf
263 kernel modules for the traffic control subsystem. To enable eBPF/eBPF JIT
264 support, depending which of the two the given architecture supports:
265
266 .in +4n
267 .B echo 1 > /proc/sys/net/core/bpf_jit_enable
268 .in
269
270 A given restricted C file can be compiled via LLVM as:
271
272 .in +4n
273 .B clang -O2 -emit-llvm -c bpf.c -o - | llc -march=bpf -filetype=obj -o bpf.o
274 .in
275
276 The compiler invocation might still simplify in future, so for now,
277 it's quite handy to alias this construct in one way or another, for
278 example:
279 .in +4n
280 .nf
281 .sp
282 __bcc() {
283 clang -O2 -emit-llvm -c $1 -o - | \\
284 llc -march=bpf -filetype=obj -o "`basename $1 .c`.o"
285 }
286
287 alias bcc=__bcc
288 .fi
289 .in
290
291 A minimal, stand-alone unit, which matches on all traffic with the
292 default classid (return code of -1) looks like:
293
294 .in +4n
295 .nf
296 .sp
297 #include <linux/bpf.h>
298
299 #ifndef __section
300 # define __section(x) __attribute__((section(x), used))
301 #endif
302
303 __section("classifier") int cls_main(struct __sk_buff *skb)
304 {
305 return -1;
306 }
307
308 char __license[] __section("license") = "GPL";
309 .fi
310 .in
311
312 More examples can be found further below in subsection
313 .B eBPF PROGRAMMING
314 as focus here will be on tooling.
315
316 There can be various other sections, for example, also for actions.
317 Thus, an object file in eBPF can contain multiple entrance points.
318 Always a specific entrance point, however, must be specified when
319 configuring with tc. A license must be part of the restricted C code
320 and the license string syntax is the same as with Linux kernel modules.
321 The kernel reserves its right that some eBPF helper functions can be
322 restricted to GPL compatible licenses only, and thus may reject a program
323 from loading into the kernel when such a license mismatch occurs.
324
325 The resulting object file from the compilation can be inspected with
326 the usual set of tools that also operate on normal object files, for
327 example
328 .B objdump(1)
329 for inspecting ELF section headers:
330
331 .in +4n
332 .nf
333 .sp
334 objdump -h bpf.o
335 [...]
336 3 classifier 000007f8 0000000000000000 0000000000000000 00000040 2**3
337 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
338 4 action-mark 00000088 0000000000000000 0000000000000000 00000838 2**3
339 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
340 5 action-rand 00000098 0000000000000000 0000000000000000 000008c0 2**3
341 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
342 6 maps 00000030 0000000000000000 0000000000000000 00000958 2**2
343 CONTENTS, ALLOC, LOAD, DATA
344 7 license 00000004 0000000000000000 0000000000000000 00000988 2**0
345 CONTENTS, ALLOC, LOAD, DATA
346 [...]
347 .fi
348 .in
349
350 Adding an eBPF classifier from an object file that contains a classifier
351 in the default ELF section is trivial (note that instead of "object-file"
352 also shortcuts such as "obj" can be used):
353
354 .in +4n
355 .B bcc bpf.c
356 .br
357 .B tc filter add dev em1 parent 1: bpf obj bpf.o flowid 1:1
358 .in
359
360 In case the classifier resides in ELF section "mycls", then that same
361 command needs to be invoked as:
362
363 .in +4n
364 .B tc filter add dev em1 parent 1: bpf obj bpf.o sec mycls flowid 1:1
365 .in
366
367 Dumping the classifier configuration will tell the location of the
368 classifier, in other words that it's from object file "bpf.o" under
369 section "mycls":
370
371 .in +4n
372 .B tc filter show dev em1
373 .br
374 .B filter parent 1: protocol all pref 49152 bpf
375 .br
376 .B filter parent 1: protocol all pref 49152 bpf handle 0x1 flowid 1:1 bpf.o:[mycls]
377 .in
378
379 The same program can also be installed on ingress qdisc side as opposed
380 to egress ...
381
382 .in +4n
383 .B tc qdisc add dev em1 handle ffff: ingress
384 .br
385 .B tc filter add dev em1 parent ffff: bpf obj bpf.o sec mycls flowid ffff:1
386 .in
387
388 \&... and again dumped from there:
389
390 .in +4n
391 .B tc filter show dev em1 parent ffff:
392 .br
393 .B filter protocol all pref 49152 bpf
394 .br
395 .B filter protocol all pref 49152 bpf handle 0x1 flowid ffff:1 bpf.o:[mycls]
396 .in
397
398 Attaching a classifier and action on ingress has the restriction that
399 it doesn't have an actual underlying queueing discipline. What ingress
400 can do is to classify, mangle, redirect or drop packets. When queueing
401 is required on ingress side, then ingress must redirect packets to the
402 .B ifb
403 device, otherwise policing can be used. Moreover, ingress can be used to
404 have an early drop point of unwanted packets before they hit upper layers
405 of the networking stack, perform network accounting with eBPF maps that
406 could be shared with egress, or have an early mangle and/or redirection
407 point to different networking devices.
408
409 Multiple eBPF actions and classifier can be placed into a single
410 object file within various sections. In that case, non-default section
411 names must be provided, which is the case for both actions in this
412 example:
413
414 .in +4n
415 .B tc filter add dev em1 parent 1: bpf obj bpf.o flowid 1:1 \e
416 .br
417 .in +25n
418 .B action bpf obj bpf.o sec action-mark \e
419 .br
420 .B action bpf obj bpf.o sec action-rand ok
421 .in -25n
422 .in -4n
423
424 The advantage of this is that the classifier and the two actions can
425 then share eBPF maps with each other, if implemented in the programs.
426
427 In order to access eBPF maps from user space beyond
428 .B tc(8)
429 setup lifetime, the ownership can be transferred to an eBPF agent via
430 Unix domain sockets. There are two possibilities for implementing this:
431
432 .B 1)
433 implementation of an own eBPF agent that takes care of setting up
434 the Unix domain socket and implementing the protocol that
435 .B tc(8)
436 dictates. A code example of this can be found inside the iproute2
437 source package under:
438 .B examples/bpf/
439
440 .B 2)
441 use
442 .B tc exec
443 for transferring the eBPF map file descriptors through a Unix domain
444 socket, and spawning an application such as
445 .B sh(1)
446 \&. This approach's advantage is that tc will place the file descriptors
447 into the environment and thus make them available just like stdin, stdout,
448 stderr file descriptors, meaning, in case user applications run from within
449 this fd-owner shell, they can terminate and restart without losing eBPF
450 maps file descriptors. Example invocation with the previous classifier and
451 action mixture:
452
453 .in +4n
454 .B tc exec bpf imp /tmp/bpf
455 .br
456 .B tc filter add dev em1 parent 1: bpf obj bpf.o exp /tmp/bpf flowid 1:1 \e
457 .br
458 .in +25n
459 .B action bpf obj bpf.o sec action-mark \e
460 .br
461 .B action bpf obj bpf.o sec action-rand ok
462 .in -25n
463 .in -4n
464
465 Assuming that eBPF maps are shared with classifier and actions, it's
466 enough to export them once, for example, from within the classifier
467 or action command. tc will setup all eBPF map file descriptors at the
468 time when the object file is first parsed.
469
470 When a shell has been spawned, the environment will have a couple of
471 eBPF related variables. BPF_NUM_MAPS provides the total number of maps
472 that have been transferred over the Unix domain socket. BPF_MAP<X>'s
473 value is the file descriptor number that can be accessed in eBPF agent
474 applications, in other words, it can directly be used as the file
475 descriptor value for the
476 .B bpf(2)
477 system call to retrieve or alter eBPF map values. <X> denotes the
478 identifier of the eBPF map. It corresponds to the
479 .B id
480 member of
481 .B struct bpf_elf_map
482 \& from the tc eBPF map specification.
483
484 The environment in this example looks as follows:
485
486 .in +4n
487 .nf
488 .sp
489 sh# env | grep BPF
490 BPF_NUM_MAPS=3
491 BPF_MAP1=6
492 BPF_MAP0=5
493 BPF_MAP2=7
494 sh# ls -la /proc/self/fd
495 [...]
496 lrwx------. 1 root root 64 Apr 14 16:46 5 -> anon_inode:bpf-map
497 lrwx------. 1 root root 64 Apr 14 16:46 6 -> anon_inode:bpf-map
498 lrwx------. 1 root root 64 Apr 14 16:46 7 -> anon_inode:bpf-map
499 sh# my_bpf_agent
500 .fi
501 .in
502
503 eBPF agents are very useful in that they can prepopulate eBPF maps from
504 user space, monitor statistics via maps and based on that feedback, for
505 example, rewrite classids in eBPF map values during runtime. Given that eBPF
506 agents are implemented as normal applications, they can also dynamically
507 receive traffic control policies from external controllers and thus push
508 them down into eBPF maps to dynamically adapt to network conditions. Moreover,
509 eBPF maps can also be shared with other eBPF program types (e.g. tracing),
510 thus very powerful combination can therefore be implemented.
511
512 .SS eBPF PROGRAMMING
513
514 eBPF classifier and actions are being implemented in restricted C syntax
515 (in future, there could additionally be new language frontends supported).
516
517 The header file
518 .B linux/bpf.h
519 provides eBPF helper functions that can be called from an eBPF program.
520 This man page will only provide two minimal, stand-alone examples, have a
521 look at
522 .B examples/bpf
523 from the iproute2 source package for a fully fledged flow dissector
524 example to better demonstrate some of the possibilities with eBPF.
525
526 Supported 32 bit classifier return codes from the C program and their meanings:
527 .in +4n
528 .B 0
529 , denotes a mismatch
530 .br
531 .B -1
532 , denotes the default classid configured from the command line
533 .br
534 .B else
535 , everything else will override the default classid to provide a facility for
536 non-linear matching
537 .in
538
539 Supported 32 bit action return codes from the C program and their meanings (
540 .B linux/pkt_cls.h
541 ):
542 .in +4n
543 .B TC_ACT_OK (0)
544 , will terminate the packet processing pipeline and allows the packet to
545 proceed
546 .br
547 .B TC_ACT_SHOT (2)
548 , will terminate the packet processing pipeline and drops the packet
549 .br
550 .B TC_ACT_UNSPEC (-1)
551 , will use the default action configured from tc (similarly as returning
552 .B -1
553 from a classifier)
554 .br
555 .B TC_ACT_PIPE (3)
556 , will iterate to the next action, if available
557 .br
558 .B TC_ACT_RECLASSIFY (1)
559 , will terminate the packet processing pipeline and start classification
560 from the beginning
561 .br
562 .B else
563 , everything else is an unspecified return code
564 .in
565
566 Both classifier and action return codes are supported in eBPF and cBPF
567 programs.
568
569 To demonstrate restricted C syntax, a minimal toy classifier example is
570 provided, which assumes that egress packets, for instance originating
571 from a container, have previously been marked in interval [0, 255]. The
572 program keeps statistics on different marks for user space and maps the
573 classid to the root qdisc with the marking itself as the minor handle:
574
575 .in +4n
576 .nf
577 .sp
578 #include <stdint.h>
579 #include <asm/types.h>
580
581 #include <linux/bpf.h>
582 #include <linux/pkt_sched.h>
583
584 #include "helpers.h"
585
586 struct tuple {
587 long packets;
588 long bytes;
589 };
590
591 #define BPF_MAP_ID_STATS 1 /* agent's map identifier */
592 #define BPF_MAX_MARK 256
593
594 struct bpf_elf_map __section("maps") map_stats = {
595 .type = BPF_MAP_TYPE_ARRAY,
596 .id = BPF_MAP_ID_STATS,
597 .size_key = sizeof(uint32_t),
598 .size_value = sizeof(struct tuple),
599 .max_elem = BPF_MAX_MARK,
600 .pinning = PIN_GLOBAL_NS,
601 };
602
603 static inline void cls_update_stats(const struct __sk_buff *skb,
604 uint32_t mark)
605 {
606 struct tuple *tu;
607
608 tu = bpf_map_lookup_elem(&map_stats, &mark);
609 if (likely(tu)) {
610 __sync_fetch_and_add(&tu->packets, 1);
611 __sync_fetch_and_add(&tu->bytes, skb->len);
612 }
613 }
614
615 __section("cls") int cls_main(struct __sk_buff *skb)
616 {
617 uint32_t mark = skb->mark;
618
619 if (unlikely(mark >= BPF_MAX_MARK))
620 return 0;
621
622 cls_update_stats(skb, mark);
623
624 return TC_H_MAKE(TC_H_ROOT, mark);
625 }
626
627 char __license[] __section("license") = "GPL";
628 .fi
629 .in
630
631 Another small example is a port redirector which demuxes destination port
632 80 into the interval [8080, 8087] steered by RSS, that can then be attached
633 to ingress qdisc. The exercise of adding the egress counterpart and IPv6
634 support is left to the reader:
635
636 .in +4n
637 .nf
638 .sp
639 #include <asm/types.h>
640 #include <asm/byteorder.h>
641
642 #include <linux/bpf.h>
643 #include <linux/filter.h>
644 #include <linux/in.h>
645 #include <linux/if_ether.h>
646 #include <linux/ip.h>
647 #include <linux/tcp.h>
648
649 #include "helpers.h"
650
651 static inline void set_tcp_dport(struct __sk_buff *skb, int nh_off,
652 __u16 old_port, __u16 new_port)
653 {
654 bpf_l4_csum_replace(skb, nh_off + offsetof(struct tcphdr, check),
655 old_port, new_port, sizeof(new_port));
656 bpf_skb_store_bytes(skb, nh_off + offsetof(struct tcphdr, dest),
657 &new_port, sizeof(new_port), 0);
658 }
659
660 static inline int lb_do_ipv4(struct __sk_buff *skb, int nh_off)
661 {
662 __u16 dport, dport_new = 8080, off;
663 __u8 ip_proto, ip_vl;
664
665 ip_proto = load_byte(skb, nh_off +
666 offsetof(struct iphdr, protocol));
667 if (ip_proto != IPPROTO_TCP)
668 return 0;
669
670 ip_vl = load_byte(skb, nh_off);
671 if (likely(ip_vl == 0x45))
672 nh_off += sizeof(struct iphdr);
673 else
674 nh_off += (ip_vl & 0xF) << 2;
675
676 dport = load_half(skb, nh_off + offsetof(struct tcphdr, dest));
677 if (dport != 80)
678 return 0;
679
680 off = skb->queue_mapping & 7;
681 set_tcp_dport(skb, nh_off - BPF_LL_OFF, __constant_htons(80),
682 __cpu_to_be16(dport_new + off));
683 return -1;
684 }
685
686 __section("lb") int lb_main(struct __sk_buff *skb)
687 {
688 int ret = 0, nh_off = BPF_LL_OFF + ETH_HLEN;
689
690 if (likely(skb->protocol == __constant_htons(ETH_P_IP)))
691 ret = lb_do_ipv4(skb, nh_off);
692
693 return ret;
694 }
695
696 char __license[] __section("license") = "GPL";
697 .fi
698 .in
699
700 The related helper header file
701 .B helpers.h
702 in both examples was:
703
704 .in +4n
705 .nf
706 .sp
707 /* Misc helper macros. */
708 #define __section(x) __attribute__((section(x), used))
709 #define offsetof(x, y) __builtin_offsetof(x, y)
710 #define likely(x) __builtin_expect(!!(x), 1)
711 #define unlikely(x) __builtin_expect(!!(x), 0)
712
713 /* Object pinning settings */
714 #define PIN_NONE 0
715 #define PIN_OBJECT_NS 1
716 #define PIN_GLOBAL_NS 2
717
718 /* ELF map definition */
719 struct bpf_elf_map {
720 __u32 type;
721 __u32 size_key;
722 __u32 size_value;
723 __u32 max_elem;
724 __u32 flags;
725 __u32 id;
726 __u32 pinning;
727 __u32 inner_id;
728 __u32 inner_idx;
729 };
730
731 /* Some used BPF function calls. */
732 static int (*bpf_skb_store_bytes)(void *ctx, int off, void *from,
733 int len, int flags) =
734 (void *) BPF_FUNC_skb_store_bytes;
735 static int (*bpf_l4_csum_replace)(void *ctx, int off, int from,
736 int to, int flags) =
737 (void *) BPF_FUNC_l4_csum_replace;
738 static void *(*bpf_map_lookup_elem)(void *map, void *key) =
739 (void *) BPF_FUNC_map_lookup_elem;
740
741 /* Some used BPF intrinsics. */
742 unsigned long long load_byte(void *skb, unsigned long long off)
743 asm ("llvm.bpf.load.byte");
744 unsigned long long load_half(void *skb, unsigned long long off)
745 asm ("llvm.bpf.load.half");
746 .fi
747 .in
748
749 Best practice, we recommend to only have a single eBPF classifier loaded
750 in tc and perform
751 .B all
752 necessary matching and mangling from there instead of a list of individual
753 classifier and separate actions. Just a single classifier tailored for a
754 given use-case will be most efficient to run.
755
756 .SS eBPF DEBUGGING
757
758 Both tc
759 .B filter
760 and
761 .B action
762 commands for
763 .B bpf
764 support an optional
765 .B verbose
766 parameter that can be used to inspect the eBPF verifier log. It is dumped
767 by default in case of an error.
768
769 In case the eBPF/cBPF JIT compiler has been enabled, it can also be
770 instructed to emit a debug output of the resulting opcode image into
771 the kernel log, which can be read via
772 .B dmesg(1)
773 :
774
775 .in +4n
776 .B echo 2 > /proc/sys/net/core/bpf_jit_enable
777 .in
778
779 The Linux kernel source tree ships additionally under
780 .B tools/net/
781 a small helper called
782 .B bpf_jit_disasm
783 that reads out the opcode image dump from the kernel log and dumps the
784 resulting disassembly:
785
786 .in +4n
787 .B bpf_jit_disasm -o
788 .in
789
790 Other than that, the Linux kernel also contains an extensive eBPF/cBPF
791 test suite module called
792 .B test_bpf
793 \&. Upon ...
794
795 .in +4n
796 .B modprobe test_bpf
797 .in
798
799 \&... it performs a diversity of test cases and dumps the results into
800 the kernel log that can be inspected with
801 .B dmesg(1)
802 \&. The results can differ depending on whether the JIT compiler is enabled
803 or not. In case of failed test cases, the module will fail to load. In
804 such cases, we urge you to file a bug report to the related JIT authors,
805 Linux kernel and networking mailing lists.
806
807 .SS cBPF
808
809 Although we generally recommend switching to implementing
810 .B eBPF
811 classifier and actions, for the sake of completeness, a few words on how to
812 program in cBPF will be lost here.
813
814 Likewise, the
815 .B bpf_jit_enable
816 switch can be enabled as mentioned already. Tooling such as
817 .B bpf_jit_disasm
818 is also independent whether eBPF or cBPF code is being loaded.
819
820 Unlike in eBPF, classifier and action are not implemented in restricted C,
821 but rather in a minimal assembler-like language or with the help of other
822 tooling.
823
824 The raw interface with tc takes opcodes directly. For example, the most
825 minimal classifier matching on every packet resulting in the default
826 classid of 1:1 looks like:
827
828 .in +4n
829 .B tc filter add dev em1 parent 1: bpf bytecode '1,6 0 0 4294967295,' flowid 1:1
830 .in
831
832 The first decimal of the bytecode sequence denotes the number of subsequent
833 4-tuples of cBPF opcodes. As mentioned, such a 4-tuple consists of
834 .B c t f k
835 decimals, where
836 .B c
837 represents the cBPF opcode,
838 .B t
839 the jump true offset target,
840 .B f
841 the jump false offset target and
842 .B k
843 the immediate constant/literal. Here, this denotes an unconditional return
844 from the program with immediate value of -1.
845
846 Thus, for egress classification, Willem de Bruijn implemented a minimal stand-alone
847 helper tool under the GNU General Public License version 2 for
848 .B iptables(8)
849 BPF extension, which abuses the
850 .B libpcap
851 internal classic BPF compiler, his code derived here for usage with
852 .B tc(8)
853 :
854
855 .in +4n
856 .nf
857 .sp
858 #include <pcap.h>
859 #include <stdio.h>
860
861 int main(int argc, char **argv)
862 {
863 struct bpf_program prog;
864 struct bpf_insn *ins;
865 int i, ret, dlt = DLT_RAW;
866
867 if (argc < 2 || argc > 3)
868 return 1;
869 if (argc == 3) {
870 dlt = pcap_datalink_name_to_val(argv[1]);
871 if (dlt == -1)
872 return 1;
873 }
874
875 ret = pcap_compile_nopcap(-1, dlt, &prog, argv[argc - 1],
876 1, PCAP_NETMASK_UNKNOWN);
877 if (ret)
878 return 1;
879
880 printf("%d,", prog.bf_len);
881 ins = prog.bf_insns;
882
883 for (i = 0; i < prog.bf_len - 1; ++ins, ++i)
884 printf("%u %u %u %u,", ins->code,
885 ins->jt, ins->jf, ins->k);
886 printf("%u %u %u %u",
887 ins->code, ins->jt, ins->jf, ins->k);
888
889 pcap_freecode(&prog);
890 return 0;
891 }
892 .fi
893 .in
894
895 Given this small helper, any
896 .B tcpdump(8)
897 filter expression can be abused as a classifier where a match will
898 result in the default classid:
899
900 .in +4n
901 .B bpftool EN10MB 'tcp[tcpflags] & tcp-syn != 0' > /var/bpf/tcp-syn
902 .br
903 .B tc filter add dev em1 parent 1: bpf bytecode-file /var/bpf/tcp-syn flowid 1:1
904 .in
905
906 Basically, such a minimal generator is equivalent to:
907
908 .in +4n
909 .B tcpdump -iem1 -ddd 'tcp[tcpflags] & tcp-syn != 0' | tr '\\\\n' ',' > /var/bpf/tcp-syn
910 .in
911
912 Since
913 .B libpcap
914 does not support all Linux' specific cBPF extensions in its compiler, the
915 Linux kernel also ships under
916 .B tools/net/
917 a minimal BPF assembler called
918 .B bpf_asm
919 for providing full control. For detailed syntax and semantics on implementing
920 such programs by hand, see references under
921 .B FURTHER READING
922 \&.
923
924 Trivial toy example in
925 .B bpf_asm
926 for classifying IPv4/TCP packets, saved in a text file called
927 .B foobar
928 :
929
930 .in +4n
931 .nf
932 .sp
933 ldh [12]
934 jne #0x800, drop
935 ldb [23]
936 jneq #6, drop
937 ret #-1
938 drop: ret #0
939 .fi
940 .in
941
942 Similarly, such a classifier can be loaded as:
943
944 .in +4n
945 .B bpf_asm foobar > /var/bpf/tcp-syn
946 .br
947 .B tc filter add dev em1 parent 1: bpf bytecode-file /var/bpf/tcp-syn flowid 1:1
948 .in
949
950 For BPF classifiers, the Linux kernel provides additionally under
951 .B tools/net/
952 a small BPF debugger called
953 .B bpf_dbg
954 , which can be used to test a classifier against pcap files, single-step
955 or add various breakpoints into the classifier program and dump register
956 contents during runtime.
957
958 Implementing an action in classic BPF is rather limited in the sense that
959 packet mangling is not supported. Therefore, it's generally recommended to
960 make the switch to eBPF, whenever possible.
961
962 .SH FURTHER READING
963 Further and more technical details about the BPF architecture can be found
964 in the Linux kernel source tree under
965 .B Documentation/networking/filter.txt
966 \&.
967
968 Further details on eBPF
969 .B tc(8)
970 examples can be found in the iproute2 source
971 tree under
972 .B examples/bpf/
973 \&.
974
975 .SH SEE ALSO
976 .BR tc (8),
977 .BR tc-ematch (8)
978 .BR bpf (2)
979 .BR bpf (4)
980
981 .SH AUTHORS
982 Manpage written by Daniel Borkmann.
983
984 Please report corrections or improvements to the Linux kernel networking
985 mailing list:
986 .B <[email protected]>