| 1 |
<?xml version='1.0'?> |
| 2 |
<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ |
| 3 |
]> |
| 4 |
|
| 5 |
|
| 6 |
<section id="dropwatchsect"> |
| 7 |
<title>Monitoring Network Packets Drops in Kernel</title> |
| 8 |
<indexterm> |
| 9 |
<primary>script examples</primary> |
| 10 |
<secondary>network profiling</secondary> |
| 11 |
</indexterm> |
| 12 |
|
| 13 |
<indexterm> |
| 14 |
<primary>examples of SystemTap scripts</primary> |
| 15 |
<secondary>network profiling</secondary> |
| 16 |
</indexterm> |
| 17 |
|
| 18 |
<indexterm> |
| 19 |
<primary>network profiling</primary> |
| 20 |
<secondary>examples of SystemTap scripts</secondary> |
| 21 |
</indexterm> |
| 22 |
|
| 23 |
<indexterm> |
| 24 |
<primary>profiling the network</primary> |
| 25 |
<secondary>examples of SystemTap scripts</secondary> |
| 26 |
</indexterm> |
| 27 |
|
| 28 |
<indexterm> |
| 29 |
<primary>network traffic, monitoring</primary> |
| 30 |
<secondary>examples of SystemTap scripts</secondary> |
| 31 |
</indexterm> |
| 32 |
|
| 33 |
<para> |
| 34 |
<indexterm><primary>tracepoint</primary></indexterm> |
| 35 |
The network stack in Linux |
| 36 |
can discard packets for various reasons. Some Linux kernels include a |
| 37 |
tracepoint, <command>kernel.trace("kfree_skb")</command>, which easily tracks where packets are discarded. |
| 38 |
<xref linkend="dropwatch"/> uses |
| 39 |
<command>kernel.trace("kfree_skb")</command> to trace |
| 40 |
packet discards; the script summarizes which locations |
| 41 |
discard packets every five-second interval. |
| 42 |
</para> |
| 43 |
|
| 44 |
<formalpara id="dropwatch"> |
| 45 |
<title>dropwatch.stp</title> |
| 46 |
<para> |
| 47 |
<programlisting><xi:include parse="text" href="../testsuite/systemtap.examples/network/dropwatch.stp" xmlns:xi="http://www.w3.org/2001/XInclude" /></programlisting> |
| 48 |
</para> |
| 49 |
</formalpara> |
| 50 |
|
| 51 |
<para> |
| 52 |
The <command>kernel.trace("kfree_skb")</command> traces which places |
| 53 |
in the kernel drop network packets. |
| 54 |
The <command>kernel.trace("kfree_skb")</command> has two arguments: |
| 55 |
a pointer to the buffer being freed (<command>$skb</command>) |
| 56 |
and the location in kernel code the |
| 57 |
buffer is being freed (<command>$location</command>). |
| 58 |
The <xref linkend="dropwatch"/> script provides the function containing |
| 59 |
<command>$location</command> where possible. |
| 60 |
The information to map <command>$location</command> back to the |
| 61 |
function is not in the instrumentation by default. |
| 62 |
On SystemTap 1.4 the <command>--all-modules</command> option will include the |
| 63 |
required mapping information and the following command can be used to run |
| 64 |
the script: |
| 65 |
</para> |
| 66 |
|
| 67 |
<programlisting>stap --all-modules dropwatch.stp</programlisting> |
| 68 |
|
| 69 |
<para> |
| 70 |
On older versions of SystemTap you can use the following command to emulate the |
| 71 |
<command>--all-modules</command> option: |
| 72 |
</para> |
| 73 |
|
| 74 |
<programlisting>stap -dkernel \ |
| 75 |
`cat /proc/modules | awk 'BEGIN { ORS = " " } {print "-d"$1}'` \ |
| 76 |
dropwatch.stp</programlisting> |
| 77 |
|
| 78 |
<para> |
| 79 |
Running the dropwatch.stp script 15 seconds would result in output similar in |
| 80 |
<xref linkend="dropwatchoutput"/>. The output lists the number of misses for |
| 81 |
each tracepoint location with either the function name or the address. |
| 82 |
</para> |
| 83 |
|
| 84 |
<example id="dropwatchoutput"> |
| 85 |
<title><xref linkend="dropwatch"/> Sample Output</title> |
| 86 |
<screen>Monitoring for dropped packets |
| 87 |
|
| 88 |
Tue Nov 17 00:26:51 2020 |
| 89 |
1762 packets dropped at unix_stream_recvmsg |
| 90 |
4 packets dropped at tun_do_read |
| 91 |
2 packets dropped at nf_hook_slow |
| 92 |
|
| 93 |
Tue Nov 17 00:26:56 2020 |
| 94 |
467 packets dropped at unix_stream_recvmsg |
| 95 |
20 packets dropped at nf_hook_slow |
| 96 |
6 packets dropped at tun_do_read |
| 97 |
|
| 98 |
Tue Nov 17 00:27:01 2020 |
| 99 |
446 packets dropped at unix_stream_recvmsg |
| 100 |
4 packets dropped at tun_do_read |
| 101 |
4 packets dropped at nf_hook_slow |
| 102 |
Stopping dropped packet monitor |
| 103 |
</screen> |
| 104 |
</example> |
| 105 |
|
| 106 |
<para> |
| 107 |
When the script is being compiled on one machine and run on |
| 108 |
another the <command>--all-modules</command> and |
| 109 |
<filename>/proc/modules</filename> directory are not available; the |
| 110 |
<command>symname</command> function will just print out the raw address. |
| 111 |
To make the raw address of packet drops more meaningful, refer to the |
| 112 |
<filename>/boot/System.map-`uname -r`</filename> file. This file lists the |
| 113 |
starting addresses for each function, allowing you to map the |
| 114 |
addresses in the output of <xref linkend="dropwatchoutput"/> to a specific |
| 115 |
function name. |
| 116 |
Given the following snippet of the |
| 117 |
<filename>/boot/System.map-`uname -r`</filename> file, |
| 118 |
the address 0xffffffff8149a8ed maps to the function |
| 119 |
<command>unix_stream_recvmsg</command>: |
| 120 |
</para> |
| 121 |
<screen>[...] |
| 122 |
ffffffff8149a420 t unix_dgram_poll |
| 123 |
ffffffff8149a5e0 t unix_stream_recvmsg |
| 124 |
ffffffff8149ad00 t unix_find_other |
| 125 |
[...]</screen> |
| 126 |
</section> |
| 127 |
|