Debian Codegraph @

Documentation/core-api/real-time/differences.rst

raw
1 .. SPDX-License-Identifier: GPL-2.0
2
3 ===========================
4 How realtime kernels differ
5 ===========================
6
7 :Author: Sebastian Andrzej Siewior <[email protected]>
8
9 Preface
10 =======
11
12 With forced-threaded interrupts and sleeping spin locks, code paths that
13 previously caused long scheduling latencies have been made preemptible and
14 moved into process context. This allows the scheduler to manage them more
15 effectively and respond to higher-priority tasks with reduced latency.
16
17 The following chapters provide an overview of key differences between a
18 PREEMPT_RT kernel and a standard, non-PREEMPT_RT kernel.
19
20 Locking
21 =======
22
23 Spinning locks such as spinlock_t are used to provide synchronization for data
24 structures accessed from both interrupt context and process context. For this
25 reason, locking functions are also available with the _irq() or _irqsave()
26 suffixes, which disable interrupts before acquiring the lock. This ensures that
27 the lock can be safely acquired in process context when interrupts are enabled.
28
29 However, on a PREEMPT_RT system, interrupts are forced-threaded and no longer
30 run in hard IRQ context. As a result, there is no need to disable interrupts as
31 part of the locking procedure when using spinlock_t.
32
33 For low-level core components such as interrupt handling, the scheduler, or the
34 timer subsystem the kernel uses raw_spinlock_t. This lock type preserves
35 traditional semantics: it disables preemption and, when used with _irq() or
36 _irqsave(), also disables interrupts. This ensures proper synchronization in
37 critical sections that must remain non-preemptible or with interrupts disabled.
38
39 Execution context
40 =================
41
42 Interrupt handling in a PREEMPT_RT system is invoked in process context through
43 the use of threaded interrupts. Other parts of the kernel also shift their
44 execution into threaded context by different mechanisms. The goal is to keep
45 execution paths preemptible, allowing the scheduler to interrupt them when a
46 higher-priority task needs to run.
47
48 Below is an overview of the kernel subsystems involved in this transition to
49 threaded, preemptible execution.
50
51 Interrupt handling
52 ------------------
53
54 All interrupts are forced-threaded in a PREEMPT_RT system. The exceptions are
55 interrupts that are requested with the IRQF_NO_THREAD, IRQF_PERCPU, or
56 IRQF_ONESHOT flags.
57
58 The IRQF_ONESHOT flag is used together with threaded interrupts, meaning those
59 registered using request_threaded_irq() and providing only a threaded handler.
60 Its purpose is to keep the interrupt line masked until the threaded handler has
61 completed.
62
63 If a primary handler is also provided in this case, it is essential that the
64 handler does not acquire any sleeping locks, as it will not be threaded. The
65 handler should be minimal and must avoid introducing delays, such as
66 busy-waiting on hardware registers.
67
68
69 Soft interrupts, bottom half handling
70 -------------------------------------
71
72 Soft interrupts are raised by the interrupt handler and are executed after the
73 handler returns. Since they run in thread context, they can be preempted by
74 other threads. Do not assume that softirq context runs with preemption
75 disabled. This means you must not rely on mechanisms like local_bh_disable() in
76 process context to protect per-CPU variables. Because softirq handlers are
77 preemptible under PREEMPT_RT, this approach does not provide reliable
78 synchronization.
79
80 If this kind of protection is required for performance reasons, consider using
81 local_lock_nested_bh(). On non-PREEMPT_RT kernels, this allows lockdep to
82 verify that bottom halves are disabled. On PREEMPT_RT systems, it adds the
83 necessary locking to ensure proper protection.
84
85 Using local_lock_nested_bh() also makes the locking scope explicit and easier
86 for readers and maintainers to understand.
87
88
89 per-CPU variables
90 -----------------
91
92 Protecting access to per-CPU variables solely by using preempt_disable() should
93 be avoided, especially if the critical section has unbounded runtime or may
94 call APIs that can sleep.
95
96 If using a spinlock_t is considered too costly for performance reasons,
97 consider using local_lock_t. On non-PREEMPT_RT configurations, this introduces
98 no runtime overhead when lockdep is disabled. With lockdep enabled, it verifies
99 that the lock is only acquired in process context and never from softirq or
100 hard IRQ context.
101
102 On a PREEMPT_RT kernel, local_lock_t is implemented using a per-CPU spinlock_t,
103 which provides safe local protection for per-CPU data while keeping the system
104 preemptible.
105
106 Because spinlock_t on PREEMPT_RT does not disable preemption, it cannot be used
107 to protect per-CPU data by relying on implicit preemption disabling. If this
108 inherited preemption disabling is essential and if local_lock_t cannot be used
109 due to performance constraints, brevity of the code, or abstraction boundaries
110 within an API then preempt_disable_nested() may be a suitable alternative. On
111 non-PREEMPT_RT kernels, it verifies with lockdep that preemption is already
112 disabled. On PREEMPT_RT, it explicitly disables preemption.
113
114 Timers
115 ------
116
117 By default, an hrtimer is executed in hard interrupt context. The exception is
118 timers initialized with the HRTIMER_MODE_SOFT flag, which are executed in
119 softirq context.
120
121 On a PREEMPT_RT kernel, this behavior is reversed: hrtimers are executed in
122 softirq context by default, typically within the ktimersd thread. This thread
123 runs at the lowest real-time priority, ensuring it executes before any
124 SCHED_OTHER tasks but does not interfere with higher-priority real-time
125 threads. To explicitly request execution in hard interrupt context on
126 PREEMPT_RT, the timer must be marked with the HRTIMER_MODE_HARD flag.
127
128 Memory allocation
129 -----------------
130
131 The memory allocation APIs, such as kmalloc() and alloc_pages(), require a
132 gfp_t flag to indicate the allocation context. On non-PREEMPT_RT kernels, it is
133 necessary to use GFP_ATOMIC when allocating memory from interrupt context or
134 from sections where preemption is disabled. This is because the allocator must
135 not sleep in these contexts waiting for memory to become available.
136
137 However, this approach does not work on PREEMPT_RT kernels. The memory
138 allocator in PREEMPT_RT uses sleeping locks internally, which cannot be
139 acquired when preemption is disabled. Fortunately, this is generally not a
140 problem, because PREEMPT_RT moves most contexts that would traditionally run
141 with preemption or interrupts disabled into threaded context, where sleeping is
142 allowed.
143
144 What remains problematic is code that explicitly disables preemption or
145 interrupts. In such cases, memory allocation must be performed outside the
146 critical section.
147
148 This restriction also applies to memory deallocation routines such as kfree()
149 and free_pages(), which may also involve internal locking and must not be
150 called from non-preemptible contexts.
151
152 IRQ work
153 --------
154
155 The irq_work API provides a mechanism to schedule a callback in interrupt
156 context. It is designed for use in contexts where traditional scheduling is not
157 possible, such as from within NMI handlers or from inside the scheduler, where
158 using a workqueue would be unsafe.
159
160 On non-PREEMPT_RT systems, all irq_work items are executed immediately in
161 interrupt context. Items marked with IRQ_WORK_LAZY are deferred until the next
162 timer tick but are still executed in interrupt context.
163
164 On PREEMPT_RT systems, the execution model changes. Because irq_work callbacks
165 may acquire sleeping locks or have unbounded execution time, they are handled
166 in thread context by a per-CPU irq_work kernel thread. This thread runs at the
167 lowest real-time priority, ensuring it executes before any SCHED_OTHER tasks
168 but does not interfere with higher-priority real-time threads.
169
170 The exception are work items marked with IRQ_WORK_HARD_IRQ, which are still
171 executed in hard interrupt context. Lazy items (IRQ_WORK_LAZY) continue to be
172 deferred until the next timer tick and are also executed by the irq_work/
173 thread.
174
175 RCU callbacks
176 -------------
177
178 RCU callbacks are invoked by default in softirq context. Their execution is
179 important because, depending on the use case, they either free memory or ensure
180 progress in state transitions. Running these callbacks as part of the softirq
181 chain can lead to undesired situations, such as contention for CPU resources
182 with other SCHED_OTHER tasks when executed within ksoftirqd.
183
184 To avoid running callbacks in softirq context, the RCU subsystem provides a
185 mechanism to execute them in process context instead. This behavior can be
186 enabled by setting the boot command-line parameter rcutree.use_softirq=0. This
187 setting is enforced in kernels configured with PREEMPT_RT.
188
189 Spin until ready
190 ================
191
192 The "spin until ready" pattern involves repeatedly checking (spinning on) the
193 state of a data structure until it becomes available. This pattern assumes that
194 preemption, soft interrupts, or interrupts are disabled. If the data structure
195 is marked busy, it is presumed to be in use by another CPU, and spinning should
196 eventually succeed as that CPU makes progress.
197
198 Some examples are hrtimer_cancel() or timer_delete_sync(). These functions
199 cancel timers that execute with interrupts or soft interrupts disabled. If a
200 thread attempts to cancel a timer and finds it active, spinning until the
201 callback completes is safe because the callback can only run on another CPU and
202 will eventually finish.
203
204 On PREEMPT_RT kernels, however, timer callbacks run in thread context. This
205 introduces a challenge: a higher-priority thread attempting to cancel the timer
206 may preempt the timer callback thread. Since the scheduler cannot migrate the
207 callback thread to another CPU due to affinity constraints, spinning can result
208 in livelock even on multiprocessor systems.
209
210 To avoid this, both the canceling and callback sides must use a handshake
211 mechanism that supports priority inheritance. This allows the canceling thread
212 to suspend until the callback completes, ensuring forward progress without
213 risking livelock.
214
215 In order to solve the problem at the API level, the sequence locks were extended
216 to allow a proper handover between the the spinning reader and the maybe
217 blocked writer.
218
219 Sequence locks
220 --------------
221
222 Sequence counters and sequential locks are documented in
223 Documentation/locking/seqlock.rst.
224
225 The interface has been extended to ensure proper preemption states for the
226 writer and spinning reader contexts. This is achieved by embedding the writer
227 serialization lock directly into the sequence counter type, resulting in
228 composite types such as seqcount_spinlock_t or seqcount_mutex_t.
229
230 These composite types allow readers to detect an ongoing write and actively
231 boost the writer’s priority to help it complete its update instead of spinning
232 and waiting for its completion.
233
234 If the plain seqcount_t is used, extra care must be taken to synchronize the
235 reader with the writer during updates. The writer must ensure its update is
236 serialized and non-preemptible relative to the reader. This cannot be achieved
237 using a regular spinlock_t because spinlock_t on PREEMPT_RT does not disable
238 preemption. In such cases, using seqcount_spinlock_t is the preferred solution.
239
240 However, if there is no spinning involved i.e., if the reader only needs to
241 detect whether a write has started and not serialize against it then using
242 seqcount_t is reasonable.