| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
The Virtual Stateless Decoder Driver (visl) |
| 4 |
=========================================== |
| 5 |
|
| 6 |
A virtual stateless decoder device for stateless uAPI development |
| 7 |
purposes. |
| 8 |
|
| 9 |
This tool's objective is to help the development and testing of |
| 10 |
userspace applications that use the V4L2 stateless API to decode media. |
| 11 |
|
| 12 |
A userspace implementation can use visl to run a decoding loop even when |
| 13 |
no hardware is available or when the kernel uAPI for the codec has not |
| 14 |
been upstreamed yet. This can reveal bugs at an early stage. |
| 15 |
|
| 16 |
This driver can also trace the contents of the V4L2 controls submitted |
| 17 |
to it. It can also dump the contents of the vb2 buffers through a |
| 18 |
debugfs interface. This is in many ways similar to the tracing |
| 19 |
infrastructure available for other popular encode/decode APIs out there |
| 20 |
and can help develop a userspace application by using another (working) |
| 21 |
one as a reference. |
| 22 |
|
| 23 |
.. note:: |
| 24 |
|
| 25 |
No actual decoding of video frames is performed by visl. The |
| 26 |
V4L2 test pattern generator is used to write various debug information |
| 27 |
to the capture buffers instead. |
| 28 |
|
| 29 |
Module parameters |
| 30 |
----------------- |
| 31 |
|
| 32 |
- visl_debug: Activates debug info, printing various debug messages through |
| 33 |
dprintk. Also controls whether per-frame debug info is shown. Defaults to off. |
| 34 |
Note that enabling this feature can result in slow performance through serial. |
| 35 |
|
| 36 |
- visl_transtime_ms: Simulated process time in milliseconds. Slowing down the |
| 37 |
decoding speed can be useful for debugging. |
| 38 |
|
| 39 |
- visl_dprintk_frame_start, visl_dprintk_frame_nframes: Dictates a range of |
| 40 |
frames where dprintk is activated. This only controls the dprintk tracing on a |
| 41 |
per-frame basis. Note that printing a lot of data can be slow through serial. |
| 42 |
|
| 43 |
- keep_bitstream_buffers: Controls whether bitstream (i.e. OUTPUT) buffers are |
| 44 |
kept after a decoding session. Defaults to false so as to reduce the amount of |
| 45 |
clutter. keep_bitstream_buffers == false works well when live debugging the |
| 46 |
client program with GDB. |
| 47 |
|
| 48 |
- bitstream_trace_frame_start, bitstream_trace_nframes: Similar to |
| 49 |
visl_dprintk_frame_start, visl_dprintk_nframes, but controls the dumping of |
| 50 |
buffer data through debugfs instead. |
| 51 |
|
| 52 |
- tpg_verbose: Write extra information on each output frame to ease debugging |
| 53 |
the API. When set to true, the output frames are not stable for a given input |
| 54 |
as some information like pointers or queue status will be added to them. |
| 55 |
|
| 56 |
What is the default use case for this driver? |
| 57 |
--------------------------------------------- |
| 58 |
|
| 59 |
This driver can be used as a way to compare different userspace implementations. |
| 60 |
This assumes that a working client is run against visl and that the ftrace and |
| 61 |
OUTPUT buffer data is subsequently used to debug a work-in-progress |
| 62 |
implementation. |
| 63 |
|
| 64 |
Even though no video decoding is actually done, the output frames can be used |
| 65 |
against a reference for a given input, except if tpg_verbose is set to true. |
| 66 |
|
| 67 |
Depending on the tpg_verbose parameter value, information on reference frames, |
| 68 |
their timestamps, the status of the OUTPUT and CAPTURE queues and more can be |
| 69 |
read directly from the CAPTURE buffers. |
| 70 |
|
| 71 |
Supported codecs |
| 72 |
---------------- |
| 73 |
|
| 74 |
The following codecs are supported: |
| 75 |
|
| 76 |
- FWHT |
| 77 |
- MPEG2 |
| 78 |
- VP8 |
| 79 |
- VP9 |
| 80 |
- H.264 |
| 81 |
- HEVC |
| 82 |
- AV1 |
| 83 |
|
| 84 |
visl trace events |
| 85 |
----------------- |
| 86 |
The trace events are defined on a per-codec basis, e.g.: |
| 87 |
|
| 88 |
.. code-block:: bash |
| 89 |
|
| 90 |
$ ls /sys/kernel/tracing/events/ | grep visl |
| 91 |
visl_av1_controls |
| 92 |
visl_fwht_controls |
| 93 |
visl_h264_controls |
| 94 |
visl_hevc_controls |
| 95 |
visl_mpeg2_controls |
| 96 |
visl_vp8_controls |
| 97 |
visl_vp9_controls |
| 98 |
|
| 99 |
For example, in order to dump HEVC SPS data: |
| 100 |
|
| 101 |
.. code-block:: bash |
| 102 |
|
| 103 |
$ echo 1 > /sys/kernel/tracing/events/visl_hevc_controls/v4l2_ctrl_hevc_sps/enable |
| 104 |
|
| 105 |
The SPS data will be dumped to the trace buffer, i.e.: |
| 106 |
|
| 107 |
.. code-block:: bash |
| 108 |
|
| 109 |
$ cat /sys/kernel/tracing/trace |
| 110 |
video_parameter_set_id 0 |
| 111 |
seq_parameter_set_id 0 |
| 112 |
pic_width_in_luma_samples 1920 |
| 113 |
pic_height_in_luma_samples 1080 |
| 114 |
bit_depth_luma_minus8 0 |
| 115 |
bit_depth_chroma_minus8 0 |
| 116 |
log2_max_pic_order_cnt_lsb_minus4 4 |
| 117 |
sps_max_dec_pic_buffering_minus1 6 |
| 118 |
sps_max_num_reorder_pics 2 |
| 119 |
sps_max_latency_increase_plus1 0 |
| 120 |
log2_min_luma_coding_block_size_minus3 0 |
| 121 |
log2_diff_max_min_luma_coding_block_size 3 |
| 122 |
log2_min_luma_transform_block_size_minus2 0 |
| 123 |
log2_diff_max_min_luma_transform_block_size 3 |
| 124 |
max_transform_hierarchy_depth_inter 2 |
| 125 |
max_transform_hierarchy_depth_intra 2 |
| 126 |
pcm_sample_bit_depth_luma_minus1 0 |
| 127 |
pcm_sample_bit_depth_chroma_minus1 0 |
| 128 |
log2_min_pcm_luma_coding_block_size_minus3 0 |
| 129 |
log2_diff_max_min_pcm_luma_coding_block_size 0 |
| 130 |
num_short_term_ref_pic_sets 0 |
| 131 |
num_long_term_ref_pics_sps 0 |
| 132 |
chroma_format_idc 1 |
| 133 |
sps_max_sub_layers_minus1 0 |
| 134 |
flags AMP_ENABLED|SAMPLE_ADAPTIVE_OFFSET|TEMPORAL_MVP_ENABLED|STRONG_INTRA_SMOOTHING_ENABLED |
| 135 |
|
| 136 |
|
| 137 |
Dumping OUTPUT buffer data through debugfs |
| 138 |
------------------------------------------ |
| 139 |
|
| 140 |
If the **VISL_DEBUGFS** Kconfig is enabled, visl will populate |
| 141 |
**/sys/kernel/debug/visl/bitstream** with OUTPUT buffer data according to the |
| 142 |
values of bitstream_trace_frame_start and bitstream_trace_nframes. This can |
| 143 |
highlight errors as broken clients may fail to fill the buffers properly. |
| 144 |
|
| 145 |
A single file is created for each processed OUTPUT buffer. Its name contains an |
| 146 |
integer that denotes the buffer sequence, i.e.: |
| 147 |
|
| 148 |
.. code-block:: c |
| 149 |
|
| 150 |
snprintf(name, 32, "bitstream%d", run->src->sequence); |
| 151 |
|
| 152 |
Dumping the values is simply a matter of reading from the file, i.e.: |
| 153 |
|
| 154 |
For the buffer with sequence == 0: |
| 155 |
|
| 156 |
.. code-block:: bash |
| 157 |
|
| 158 |
$ xxd /sys/kernel/debug/visl/bitstream/bitstream0 |
| 159 |
00000000: 2601 af04 d088 bc25 a173 0e41 a4f2 3274 &......%.s.A..2t |
| 160 |
00000010: c668 cb28 e775 b4ac f53a ba60 f8fd 3aa1 .h.(.u...:.`..:. |
| 161 |
00000020: 46b4 bcfc 506c e227 2372 e5f5 d7ea 579f F...Pl.'#r....W. |
| 162 |
00000030: 6371 5eb5 0eb8 23b5 ca6a 5de5 983a 19e4 cq^...#..j]..:.. |
| 163 |
00000040: e8c3 4320 b4ba a226 cbc1 4138 3a12 32d6 ..C ...&..A8:.2. |
| 164 |
00000050: fef3 247b 3523 4e90 9682 ac8e eb0c a389 ..${5#N......... |
| 165 |
00000060: ddd0 6cfc 0187 0e20 7aae b15b 1812 3d33 ..l.... z..[..=3 |
| 166 |
00000070: e1c5 f425 a83a 00b7 4f18 8127 3c4c aefb ...%.:..O..'<L.. |
| 167 |
|
| 168 |
For the buffer with sequence == 1: |
| 169 |
|
| 170 |
.. code-block:: bash |
| 171 |
|
| 172 |
$ xxd /sys/kernel/debug/visl/bitstream/bitstream1 |
| 173 |
00000000: 0201 d021 49e1 0c40 aa11 1449 14a6 01dc [email protected].... |
| 174 |
00000010: 7023 889a c8cd 2cd0 13b4 dab0 e8ca 21fe p#....,.......!. |
| 175 |
00000020: c4c8 ab4c 486e 4e2f b0df 96cc c74e 8dde ...LHnN/.....N.. |
| 176 |
00000030: 8ce7 ee36 d880 4095 4d64 30a0 ff4f 0c5e [email protected].^ |
| 177 |
00000040: f16b a6a1 d806 ca2a 0ece a673 7bea 1f37 .k.....*...s{..7 |
| 178 |
00000050: 370f 5bb9 1dc4 ba21 6434 bc53 0173 cba0 7.[....!d4.S.s.. |
| 179 |
00000060: dfe6 bc99 01ea b6e0 346b 92b5 c8de 9f5d ........4k.....] |
| 180 |
00000070: e7cc 3484 1769 fef2 a693 a945 2c8b 31da ..4..i.....E,.1. |
| 181 |
|
| 182 |
And so on. |
| 183 |
|
| 184 |
By default, the files are removed during STREAMOFF. This is to reduce the amount |
| 185 |
of clutter. |