| 1 |
========================== |
| 2 |
Coresight CPU Debug Module |
| 3 |
========================== |
| 4 |
|
| 5 |
:Author: Leo Yan <[email protected]> |
| 6 |
:Date: April 5th, 2017 |
| 7 |
|
| 8 |
Introduction |
| 9 |
------------ |
| 10 |
|
| 11 |
Coresight CPU debug module is defined in ARMv8-a architecture reference manual |
| 12 |
(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate |
| 13 |
debug module and it is mainly used for two modes: self-hosted debug and |
| 14 |
external debug. Usually the external debug mode is well known as the external |
| 15 |
debugger connects with SoC from JTAG port; on the other hand the program can |
| 16 |
explore debugging method which rely on self-hosted debug mode, this document |
| 17 |
is to focus on this part. |
| 18 |
|
| 19 |
The debug module provides sample-based profiling extension, which can be used |
| 20 |
to sample CPU program counter, secure state and exception level, etc; usually |
| 21 |
every CPU has one dedicated debug module to be connected. Based on self-hosted |
| 22 |
debug mechanism, Linux kernel can access these related registers from mmio |
| 23 |
region when the kernel panic happens. The callback notifier for kernel panic |
| 24 |
will dump related registers for every CPU; finally this is good for assistant |
| 25 |
analysis for panic. |
| 26 |
|
| 27 |
|
| 28 |
Implementation |
| 29 |
-------------- |
| 30 |
|
| 31 |
- During driver registration, it uses EDDEVID and EDDEVID1 - two device ID |
| 32 |
registers to decide if sample-based profiling is implemented or not. On some |
| 33 |
platforms this hardware feature is fully or partially implemented; and if |
| 34 |
this feature is not supported then registration will fail. |
| 35 |
|
| 36 |
- At the time this documentation was written, the debug driver mainly relies on |
| 37 |
information gathered by the kernel panic callback notifier from three |
| 38 |
sampling registers: EDPCSR, EDVIDSR and EDCIDSR: from EDPCSR we can get |
| 39 |
program counter; EDVIDSR has information for secure state, exception level, |
| 40 |
bit width, etc; EDCIDSR is context ID value which contains the sampled value |
| 41 |
of CONTEXTIDR_EL1. |
| 42 |
|
| 43 |
- The driver supports a CPU running in either AArch64 or AArch32 mode. The |
| 44 |
registers naming convention is a bit different between them, AArch64 uses |
| 45 |
'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses |
| 46 |
'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to |
| 47 |
use AArch64 naming convention. |
| 48 |
|
| 49 |
- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different |
| 50 |
register bits definition. So the driver consolidates two difference: |
| 51 |
|
| 52 |
If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented; |
| 53 |
but ARMv7-a defines "PCSR samples are offset by a value that depends on the |
| 54 |
instruction set state". For ARMv7-a, the driver checks furthermore if CPU |
| 55 |
runs with ARM or thumb instruction set and calibrate PCSR value, the |
| 56 |
detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter |
| 57 |
C11.11.34 "DBGPCSR, Program Counter Sampling Register". |
| 58 |
|
| 59 |
If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have |
| 60 |
no offset applied and do not sample the instruction set state in AArch32 |
| 61 |
state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates |
| 62 |
in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64 |
| 63 |
state EDPCSR is sampled and no offset are applied. |
| 64 |
|
| 65 |
|
| 66 |
Clock and power domain |
| 67 |
---------------------- |
| 68 |
|
| 69 |
Before accessing debug registers, we should ensure the clock and power domain |
| 70 |
have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1 |
| 71 |
Debug registers', the debug registers are spread into two domains: the debug |
| 72 |
domain and the CPU domain. |
| 73 |
:: |
| 74 |
|
| 75 |
+---------------+ |
| 76 |
| | |
| 77 |
| | |
| 78 |
+----------+--+ | |
| 79 |
dbg_clock -->| |**| |<-- cpu_clock |
| 80 |
| Debug |**| CPU | |
| 81 |
dbg_power_domain -->| |**| |<-- cpu_power_domain |
| 82 |
+----------+--+ | |
| 83 |
| | |
| 84 |
| | |
| 85 |
+---------------+ |
| 86 |
|
| 87 |
For debug domain, the user uses DT binding "clocks" and "power-domains" to |
| 88 |
specify the corresponding clock source and power supply for the debug logic. |
| 89 |
The driver calls the pm_runtime_{put|get} operations as needed to handle the |
| 90 |
debug power domain. |
| 91 |
|
| 92 |
For CPU domain, the different SoC designs have different power management |
| 93 |
schemes and finally this heavily impacts external debug module. So we can |
| 94 |
divide into below cases: |
| 95 |
|
| 96 |
- On systems with a sane power controller which can behave correctly with |
| 97 |
respect to CPU power domain, the CPU power domain can be controlled by |
| 98 |
register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ |
| 99 |
to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation |
| 100 |
of CPU power down. As result, this can ensure the CPU power domain is |
| 101 |
powered on properly during the period when access debug related registers; |
| 102 |
|
| 103 |
- Some designs will power down an entire cluster if all CPUs on the cluster |
| 104 |
are powered down - including the parts of the debug registers that should |
| 105 |
remain powered in the debug power domain. The bits in EDPRCR are not |
| 106 |
respected in these cases, so these designs do not support debug over |
| 107 |
power down in the way that the CoreSight / Debug designers anticipated. |
| 108 |
This means that even checking EDPRSR has the potential to cause a bus hang |
| 109 |
if the target register is unpowered. |
| 110 |
|
| 111 |
In this case, accessing to the debug registers while they are not powered |
| 112 |
is a recipe for disaster; so we need preventing CPU low power states at boot |
| 113 |
time or when user enable module at the run time. Please see chapter |
| 114 |
"How to use the module" for detailed usage info for this. |
| 115 |
|
| 116 |
|
| 117 |
Device Tree Bindings |
| 118 |
-------------------- |
| 119 |
|
| 120 |
See Documentation/devicetree/bindings/arm/arm,coresight-cpu-debug.yaml for |
| 121 |
details. |
| 122 |
|
| 123 |
|
| 124 |
How to use the module |
| 125 |
--------------------- |
| 126 |
|
| 127 |
If you want to enable debugging functionality at boot time, you can add |
| 128 |
"coresight_cpu_debug.enable=1" to the kernel command line parameter. |
| 129 |
|
| 130 |
The driver also can work as module, so can enable the debugging when insmod |
| 131 |
module:: |
| 132 |
|
| 133 |
# insmod coresight_cpu_debug.ko debug=1 |
| 134 |
|
| 135 |
When boot time or insmod module you have not enabled the debugging, the driver |
| 136 |
uses the debugfs file system to provide a knob to dynamically enable or disable |
| 137 |
debugging: |
| 138 |
|
| 139 |
To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable:: |
| 140 |
|
| 141 |
# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable |
| 142 |
|
| 143 |
To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable:: |
| 144 |
|
| 145 |
# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable |
| 146 |
|
| 147 |
As explained in chapter "Clock and power domain", if you are working on one |
| 148 |
platform which has idle states to power off debug logic and the power |
| 149 |
controller cannot work well for the request from EDPRCR, then you should |
| 150 |
firstly constraint CPU idle states before enable CPU debugging feature; so can |
| 151 |
ensure the accessing to debug logic. |
| 152 |
|
| 153 |
If you want to limit idle states at boot time, you can use "nohlt" or |
| 154 |
"cpuidle.off=1" in the kernel command line. |
| 155 |
|
| 156 |
At the runtime you can disable idle states with below methods: |
| 157 |
|
| 158 |
It is possible to disable CPU idle states by way of the PM QoS |
| 159 |
subsystem, more specifically by using the "/dev/cpu_dma_latency" |
| 160 |
interface (see Documentation/power/pm_qos_interface.rst for more |
| 161 |
details). As specified in the PM QoS documentation the requested |
| 162 |
parameter will stay in effect until the file descriptor is released. |
| 163 |
For example:: |
| 164 |
|
| 165 |
# exec 3<> /dev/cpu_dma_latency; echo 0 >&3 |
| 166 |
... |
| 167 |
Do some work... |
| 168 |
... |
| 169 |
# exec 3<>- |
| 170 |
|
| 171 |
The same can also be done from an application program. |
| 172 |
|
| 173 |
Disable specific CPU's specific idle state from cpuidle sysfs (see |
| 174 |
Documentation/admin-guide/pm/cpuidle.rst):: |
| 175 |
|
| 176 |
# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable |
| 177 |
|
| 178 |
Output format |
| 179 |
------------- |
| 180 |
|
| 181 |
Here is an example of the debugging output format:: |
| 182 |
|
| 183 |
ARM external debug module: |
| 184 |
coresight-cpu-debug 850000.debug: CPU[0]: |
| 185 |
coresight-cpu-debug 850000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock) |
| 186 |
coresight-cpu-debug 850000.debug: EDPCSR: handle_IPI+0x174/0x1d8 |
| 187 |
coresight-cpu-debug 850000.debug: EDCIDSR: 00000000 |
| 188 |
coresight-cpu-debug 850000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0) |
| 189 |
coresight-cpu-debug 852000.debug: CPU[1]: |
| 190 |
coresight-cpu-debug 852000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock) |
| 191 |
coresight-cpu-debug 852000.debug: EDPCSR: debug_notifier_call+0x23c/0x358 |
| 192 |
coresight-cpu-debug 852000.debug: EDCIDSR: 00000000 |
| 193 |
coresight-cpu-debug 852000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0) |