| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
============================================= |
| 4 |
Porting an architecture to support PREEMPT_RT |
| 5 |
============================================= |
| 6 |
|
| 7 |
:Author: Sebastian Andrzej Siewior <[email protected]> |
| 8 |
|
| 9 |
This list outlines the architecture specific requirements that must be |
| 10 |
implemented in order to enable PREEMPT_RT. Once all required features are |
| 11 |
implemented, ARCH_SUPPORTS_RT can be selected in architecture’s Kconfig to make |
| 12 |
PREEMPT_RT selectable. |
| 13 |
Many prerequisites (genirq support for example) are enforced by the common code |
| 14 |
and are omitted here. |
| 15 |
|
| 16 |
The optional features are not strictly required but it is worth to consider |
| 17 |
them. |
| 18 |
|
| 19 |
Requirements |
| 20 |
------------ |
| 21 |
|
| 22 |
Forced threaded interrupts |
| 23 |
CONFIG_IRQ_FORCED_THREADING must be selected. Any interrupts that must |
| 24 |
remain in hard-IRQ context must be marked with IRQF_NO_THREAD. This |
| 25 |
requirement applies for instance to clocksource event interrupts, |
| 26 |
perf interrupts and cascading interrupt-controller handlers. |
| 27 |
|
| 28 |
PREEMPTION support |
| 29 |
Kernel preemption must be supported and requires that |
| 30 |
CONFIG_ARCH_NO_PREEMPT remain unselected. Scheduling requests, such as those |
| 31 |
issued from an interrupt or other exception handler, must be processed |
| 32 |
immediately. |
| 33 |
|
| 34 |
POSIX CPU timers and KVM |
| 35 |
POSIX CPU timers must expire from thread context rather than directly within |
| 36 |
the timer interrupt. This behavior is enabled by setting the configuration |
| 37 |
option CONFIG_HAVE_POSIX_CPU_TIMERS_TASK_WORK. |
| 38 |
When virtualization support, such as KVM, is enabled, |
| 39 |
CONFIG_VIRT_XFER_TO_GUEST_WORK must also be set to ensure |
| 40 |
that any pending work, such as POSIX timer expiration, is handled before |
| 41 |
transitioning into guest mode. |
| 42 |
|
| 43 |
Hard-IRQ and Soft-IRQ stacks |
| 44 |
Soft interrupts are handled in the thread context in which they are raised. If |
| 45 |
a soft interrupt is triggered from hard-IRQ context, its execution is deferred |
| 46 |
to the ksoftirqd thread. Preemption is never disabled during soft interrupt |
| 47 |
handling, which makes soft interrupts preemptible. |
| 48 |
If an architecture provides a custom __do_softirq() implementation that uses a |
| 49 |
separate stack, it must select CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK. The |
| 50 |
functionality should only be enabled when CONFIG_SOFTIRQ_ON_OWN_STACK is set. |
| 51 |
|
| 52 |
FPU and SIMD access in kernel mode |
| 53 |
FPU and SIMD registers are typically not used in kernel mode and are therefore |
| 54 |
not saved during kernel preemption. As a result, any kernel code that uses |
| 55 |
these registers must be enclosed within a kernel_fpu_begin() and |
| 56 |
kernel_fpu_end() section. |
| 57 |
The kernel_fpu_begin() function usually invokes local_bh_disable() to prevent |
| 58 |
interruptions from softirqs and to disable regular preemption. This allows the |
| 59 |
protected code to run safely in both thread and softirq contexts. |
| 60 |
On PREEMPT_RT kernels, however, kernel_fpu_begin() must not call |
| 61 |
local_bh_disable(). Instead, it should use preempt_disable(), since softirqs |
| 62 |
are always handled in thread context under PREEMPT_RT. In this case, disabling |
| 63 |
preemption alone is sufficient. |
| 64 |
The crypto subsystem operates on memory pages and requires users to "walk and |
| 65 |
map" these pages while processing a request. This operation must occur outside |
| 66 |
the kernel_fpu_begin()/ kernel_fpu_end() section because it requires preemption |
| 67 |
to be enabled. These preemption points are generally sufficient to avoid |
| 68 |
excessive scheduling latency. |
| 69 |
|
| 70 |
Exception handlers |
| 71 |
Exception handlers, such as the page fault handler, typically enable interrupts |
| 72 |
early, before invoking any generic code to process the exception. This is |
| 73 |
necessary because handling a page fault may involve operations that can sleep. |
| 74 |
Enabling interrupts is especially important on PREEMPT_RT, where certain |
| 75 |
locks, such as spinlock_t, become sleepable. For example, handling an |
| 76 |
invalid opcode may result in sending a SIGILL signal to the user task. A |
| 77 |
debug excpetion will send a SIGTRAP signal. |
| 78 |
In both cases, if the exception occurred in user space, it is safe to enable |
| 79 |
interrupts early. Sending a signal requires both interrupts and kernel |
| 80 |
preemption to be enabled. |
| 81 |
|
| 82 |
Optional features |
| 83 |
----------------- |
| 84 |
|
| 85 |
Timer and clocksource |
| 86 |
A high-resolution clocksource and clockevents device are recommended. The |
| 87 |
clockevents device should support the CLOCK_EVT_FEAT_ONESHOT feature for |
| 88 |
optimal timer behavior. In most cases, microsecond-level accuracy is |
| 89 |
sufficient |
| 90 |
|
| 91 |
Lazy preemption |
| 92 |
This mechanism allows an in-kernel scheduling request for non-real-time tasks |
| 93 |
to be delayed until the task is about to return to user space. It helps avoid |
| 94 |
preempting a task that holds a sleeping lock at the time of the scheduling |
| 95 |
request. |
| 96 |
With CONFIG_GENERIC_IRQ_ENTRY enabled, supporting this feature requires |
| 97 |
defining a bit for TIF_NEED_RESCHED_LAZY, preferably near TIF_NEED_RESCHED. |
| 98 |
|
| 99 |
Serial console with NBCON |
| 100 |
With PREEMPT_RT enabled, all console output is handled by a dedicated thread |
| 101 |
rather than directly from the context in which printk() is invoked. This design |
| 102 |
allows printk() to be safely used in atomic contexts. |
| 103 |
However, this also means that if the kernel crashes and cannot switch to the |
| 104 |
printing thread, no output will be visible preventing the system from printing |
| 105 |
its final messages. |
| 106 |
There are exceptions for immediate output, such as during panic() handling. To |
| 107 |
support this, the console driver must implement new-style lock handling. This |
| 108 |
involves setting the CON_NBCON flag in console::flags and providing |
| 109 |
implementations for the write_atomic, write_thread, device_lock, and |
| 110 |
device_unlock callbacks. |