Monitoring Network Packets Drops in Kernel
script examples
network profiling
examples of SystemTap scripts
network profiling
network profiling
examples of SystemTap scripts
profiling the network
examples of SystemTap scripts
network traffic, monitoring
examples of SystemTap scripts
tracepoint
The network stack in Linux
can discard packets for various reasons. Some Linux kernels include a
tracepoint, kernel.trace("kfree_skb"), which easily tracks where packets are discarded.
uses
kernel.trace("kfree_skb") to trace
packet discards; the script summarizes which locations
discard packets every five-second interval.
dropwatch.stp
The kernel.trace("kfree_skb") traces which places
in the kernel drop network packets.
The kernel.trace("kfree_skb") has two arguments:
a pointer to the buffer being freed ($skb)
and the location in kernel code the
buffer is being freed ($location).
The script provides the function containing
$location where possible.
The information to map $location back to the
function is not in the instrumentation by default.
On SystemTap 1.4 the --all-modules option will include the
required mapping information and the following command can be used to run
the script:
stap --all-modules dropwatch.stp
On older versions of SystemTap you can use the following command to emulate the
--all-modules option:
stap -dkernel \
`cat /proc/modules | awk 'BEGIN { ORS = " " } {print "-d"$1}'` \
dropwatch.stp
Running the dropwatch.stp script 15 seconds would result in output similar in
. The output lists the number of misses for
each tracepoint location with either the function name or the address.
Sample Output
Monitoring for dropped packets
Tue Nov 17 00:26:51 2020
1762 packets dropped at unix_stream_recvmsg
4 packets dropped at tun_do_read
2 packets dropped at nf_hook_slow
Tue Nov 17 00:26:56 2020
467 packets dropped at unix_stream_recvmsg
20 packets dropped at nf_hook_slow
6 packets dropped at tun_do_read
Tue Nov 17 00:27:01 2020
446 packets dropped at unix_stream_recvmsg
4 packets dropped at tun_do_read
4 packets dropped at nf_hook_slow
Stopping dropped packet monitor
When the script is being compiled on one machine and run on
another the --all-modules and
/proc/modules directory are not available; the
symname function will just print out the raw address.
To make the raw address of packet drops more meaningful, refer to the
/boot/System.map-`uname -r` file. This file lists the
starting addresses for each function, allowing you to map the
addresses in the output of to a specific
function name.
Given the following snippet of the
/boot/System.map-`uname -r` file,
the address 0xffffffff8149a8ed maps to the function
unix_stream_recvmsg:
[...]
ffffffff8149a420 t unix_dgram_poll
ffffffff8149a5e0 t unix_stream_recvmsg
ffffffff8149ad00 t unix_find_other
[...]