Debian Codegraph @

doc/READERS

raw
1 The libmpg123 web of reader abstractions
2 ========================================
3
4 Somehow the differing ways of getting compressed data into libmpg123 reached
5 unholy numbers with the years. As keeper of the legacy, I got quite some of that
6 to keep. There are intersectional layers ... however you might call it.
7
8 An attempt to get an overview and be able to refactor that for the glorious
9 portable API of mpg123 1.32.0.
10
11 The frame struct has two parts concerned with input streams.
12
13 struct reader *rd; /* pointer to the reading functions */
14 struct reader_data rdat; /* reader data and state info */
15
16 The distinction is blurred a bit: Over time, I added function pointers to
17 the latter.
18
19 1. Basic methods of data input for (seekable) streams
20 -----------------------------------------------------
21
22 These reside in the struct reader_data member of the frame struct (mpg123_handle), so
23 normally fr->rdat. This is an assortment of (user-supplied) function pointers
24 and stores the stream position and length. The latter was the initial purpose.
25
26 1.1 Reader based around POSIX I/O
27 ---------------------------------
28
29 This one relies on some read() and lseek() to give the input bytes. With an
30 innocent mpg123_open(), you trigger compat_open() and further work on the
31 resulting fd.
32
33 But you can also do mpg123_open_fd() to provide the descriptor yourself. Same
34 further code path, just no closing.
35
36 1.1.1 Timeout read
37 ------------------
38
39 The timeout reader is a variant that squeezes into the internal POSIX I/O with
40 some fcntl() and select() on the file descriptor, and a separate reader callback
41 stored as fdread().
42
43 1.2 Replaced I/O on file descriptor
44 -----------------------------------
45
46 After calling mpg123_replace_reader(), you have your callbacks (or the respective
47 fallback callback) operating on the file descriptor that could have resulted
48 from internal opening or been handed over to libmpg123.
49
50 1.3 Replaced I/O on custom handle
51 ---------------------------------
52
53 This replaces both read() and lseek() with your callbacks and opening is
54 full external, just the handle being handed over via mpg123_open_handle().
55
56 1.4 Replaced I/O on custom handle using 64 bit offsets
57 ------------------------------------------------------
58
59 This is to come, the above just with a differing style of callbacks that avoid
60 off_t. I intend to pack all the above into wrapper code and have this whole
61 first aspect of differing callbacks removed.
62
63
64 2. Abstractions
65 ---------------
66
67 The actual interface to the parser is given by instances of struct reader. This
68 is usually accessed as fr->rd and contains function pointers to specific routines
69 like fullread() and head_read(). These access the basic methods behind the scenes.
70
71 There is overlap in the functions. The main differentiator is the fullread() call,
72 which is the next layer of read(). I guess code sharing could be one excuse not
73 to have each of these as a wholly separate I/O layer implementation.
74
75 2.1 READER_STREAM
76 -----------------
77
78 This reader handles a plain possibly seekable input stream. It introduces the
79 plain_fullread() function which loops over fr->rdat.fdread() until the desired
80 bytes are aquired or EOF occurs. There is no signal handling. A return value
81 less than zero is an error and the end, the function returning a short byte
82 count. This function also advances fr->rdat.filepos if the reader is not
83 buffered.
84
85 2.2 READER_ICY_STREAM
86 ---------------------
87
88 This replaces plain_fullread() with icy_fullread(), which looks out for ICY
89 metadata at the configured interval. It resorts to fr->rdat.fdread() and
90 plain_fullread() to do its chores.
91
92 2.3 READER_FEED
93 ---------------
94
95 The reader handling libmpg123 feeder mode. It stuffs data into an internal buffer
96 and extracts from that, handing out READER_MORE as error to be recovered from
97 when the client provides more data. It provides a bit of seeking within the
98 buffer for parsing (look-ahead) purposes, before read data is purged from the
99 buffer chain.
100
101 The actual mid-level reader here is feed_read(), wrapping over the bufferchain data
102 structre with its methods.
103
104 2.4 READER_BUF_STREAM
105 ---------------------
106
107 For some reason, I had to add a mode for a stream with some buffering
108 (MPG123_SEEKBUFFER). Well ... yes, MPEG parsing is just more fun if you can peek
109 ahead and have a little window of input data to work with. This used to employ
110 buffered_fullread(), which in turn called fr->rdat.fullread(), which was
111 plain_fullread() for this variant, and wraped a bufferchain around it.
112
113 Now this got its separate buffered_plain_fullread() without the extra function
114 pointer in fr->rdat.
115
116 2.5 READER_BUF_ICY_STREAM
117 -------------------------
118
119 This is the same buffered reader but with buffered_icy_fullread() instead.
120
121
122 3. Control flow for setting up a stream
123 =======================================
124
125 3.1 mpg123_open()
126 -----------------
127
128 Client code just opens a track or possibly called mpg123_replace_reader()
129 beforehand, which does not change the opening behaviour.
130
131 This accesses the given file path via compat_open(), stores a file descriptor,
132 calls reader stream init which checks seekability (also fetching ID3v1) and
133 stores file size. This results in one of the READER_*STREAM family.
134
135 The lfs_wrap machinery gets triggrered and inserts its callbacks, working
136 on the prepared wrapper data.
137
138 3.2 mpg123_open_fd()
139 --------------------
140
141 This just skips the compat_open() and stores the given file descriptor, assuming
142 that it works with the configured callbacks. The same dance with stream setup.
143
144 The lfs_wrap machinery gets triggrered and inserts its callbacks, working
145 on the prepared wrapper data.
146
147 3.3 mpg123_open_handle()
148 ------------------------
149
150 Also skips the opening, stores the handle and does the stream setup.
151
152 This shall not trigger callback insertion. The idea is that the user did
153 call mpg123_reader64() or a wrapper variant of it before. The wrapper code
154 itself finalizes its work with a call to mpg123_reader64().
155
156 Oh, wait. What about the other wrapper calls? Client code calls
157 mpg123_replace_reader_handle() with its callbacks, be it off_t or off64_t.
158 This needs to trigger preparation of wrapperdata and installment of wrapper
159 callbacks via mpg123_reader64(). A subsequent mpg123_open_handle() needs
160 to store the actual client handle inside the wrapperdata structure and use
161 the latter as iohandle for stream operation. I need tell apart internal and
162 external use of mpg123_reader64().
163
164 So store a flag for that? Is there another way without introducing yet another
165 function? Well, the wrapperdata can have two states that fit this scenario:
166
167 - not present at all
168 - present with a respective state set
169
170 I want to avoid unnecessary allocation of the wrapperdata (just because I am that
171 kind of person). So I need to ensure that INT123_wrap_open() when called with
172 an external handle and not encountering an existing wrapperdata instance, does not
173 allocate one, but just silently does nothing, as there is nothing to do. Well,
174 it can check if callbacks are in place. At least that.
175
176
177 3.4 mpg123_open_feed()
178 ----------------------
179
180 Prepares for the non-seekable feeder mode, limited stream setup because peeking
181 at the end won't work.
182
183 This does not trigger the wrapper ... except ... should it unregister its
184 callbacks? No. The code path of the feeder is separate enough that it does
185 not interfere.
186
187
188 4. Plan
189 =======
190
191 Keep the abstractions in readers.c, move all variants of POSIX-like callback stuff
192 into the wrapper section (lfs_wrap.c for now). In theory, the buffer chain for
193 the feeder could also be moved into a variant hidden behind mpg123_reader64(). Maybe
194 in the future. At some point.
195
196 I don't want any mpg123 internals regarding the frame struct in the wrapper
197 implementation. So maybe lfs_wrap.c only offers wrapper setup as such and
198 libmpg123.c does the actual stream opening? No. The explicit largefile stuff
199 needs to be handled (O_LARGEFILE). But the further stream setup ... that
200 should happen in mpg123_open() and friends, after reader handle setup.
201
202 I still need fr->rdat.flags for selecting buffered etc., but not for
203 READER_FD_OPENEND or READER_NONBLOCK.
204
205 Both fr->rdat.fullread and fr->rdat.fdread are gone now. The picture is getting
206 clearer.
207
208 I made the static functions fdread() and fdseek() robust against missing
209 callbacks. It's a question if we'd rather want to catch those earlier, though.