Debian Codegraph @

docs/random/thomasvs/metadata

raw
1 I'll use this doc to describe how I think media info should work from the
2 perspective of the application developer and end user, and from that
3 extrapolate what we need to provide that.
4
5 RATIONALE
6 ---------
7 One of the strong points of GStreamer is that it abstracts library dependencies
8 away. A user is free to choose whatever plug-ins he has, and a developer
9 can code to the general API that GStreamer provides without having to deal
10 with the underlying codecs.
11
12 It is important that GStreamer also handles media info well and efficiently,
13 since more often than not the same libraries are needed to do this. So
14 to avoid applications depending on these libs just to do the media info,
15 we should make sure GStreamer provides a reasonable and fast abstraction
16 for this as well.
17
18 GOALS
19 -----
20 - quickly read and write "tags"
21 - quickly read stream metadata (technical properties, length, audio props, ...)
22 - cache both kinds of data transparently
23 - (possibly) provide bins that do this
24 - provide a simple API to do this
25
26 DEFINITION OF TERMS
27 -------------------
28 The user or developer using GStreamer is interested in all information that
29 describes the stream. The library handles these two types differently
30 however, so I will use the following terms to describe this :
31
32 - metadata :
33 every kind of information that is tied to the "concept" of the stream,
34 and not tied to the actual encoding or representation of the stream.
35 - it can be altered without transcoding the stream
36 - it would stay the same for different encodings of the file
37 - describes properties of the information encoded into the stream
38 - examples:
39 - artist, title, author
40 - year, track order, album
41 - comments
42
43 - mediainfo
44 every kind of information that is tied to the "codec" used.
45 - cannot be altered without transcoding
46 - is the set of parameters the stream has been encoded with
47 - describes properties of the encoded stream itself
48 - examples:
49 - bitrate targets (e.g. nominal), encoding mode (e.g. joint stereo)
50 - to this we also add "bitrate", but we query this through the pad_query
51 interface
52
53 - format
54 every kind of information that is tied to the "raw" bitstream
55 - cannot be altered without decoding and changing the raw bitstream
56 - examples:
57 - samplerate, bit depth/width, channels
58 - length in time
59 - video size, frames per second, colorspace used
60 - the format is queried by getting the GstCaps of the pad that sources
61 the buffers
62
63 - length in time and tracks for the whole stream
64 - gotten through pad queries
65 - stored in variables in the struct
66
67 - immediate info
68 - examples:
69 - position in time
70 - current bitrate
71
72 - tracks :
73 a media file or stream can contain multiple consecutive streams, which
74 we will call "tracks". GStreamer has a format for track used in querying
75 and seeking as well.
76 A track should be thought of as the whole of one single piece of media
77 inside a physical stream.
78 A track can have at most one set of tags, and has fixed "raw" properties.
79
80 EXAMPLE PIPELINES
81 -----------------
82 reading metadata : filesrc ! id3v1
83 - would read metadata from file
84 - id3v1 immediately causes filesrc to seek until it has found
85 - the (first) metadata
86 - that there is no metadata present
87 - id3v1 sends out a property notification with name "metadata" and
88 a GstCaps structure
89
90 resetting and writing content metadata :
91 id3v1 reset=true artist="Arid" ! filesink
92
93 - effect: clear the current tag and reset it to only have Arid as artist
94 - id3v1 seeks to the right location, clears the tag, and writes the new one
95
96 COST
97 ----
98 Querying media info can be expensive.
99 Any application querying for media info should take this into account and
100 make sure that it doesn't block the app unnecessarily while the querying
101 happens.
102
103 The app should create an object, hand it a bunch of locations to query,
104 and connect to the signal the app is going to send out.
105
106 In most cases, querying content data should be fast since it doesn't involve
107 decoding
108
109 Technical data could be harder and thus might be better done only when needed.
110
111 CACHE
112 -----
113 Getting media info can be an expensive operation. It makes sense to cache
114 the dia info queried on-disk to provide rapid access to this data.
115 It is important however that this is done transparently - the system should
116 be able to keep working without it, or keep working when you delete this cache.
117
118 The API would provide a function like
119 gst_media_info_read_cached (media_info, location,
120 GST_MEDIA_INFO_METADATA,
121 GST_MEDIA_INFO_READ_CACHED);
122
123 to try and get the cached metadata using the media info object.
124
125 - check if the file is cached in the media info cache
126 - if no, then read the media info and store it in the cache
127 - if yes, then check the file against it's timestamp (or (part of) md5sum ?)
128 - if it was changed, force a new read and store it in the cache
129 - if it wasn't changed, just return the cached media info
130
131
132 For optimizations, it might also make sense to do
133 GList * gst_metadata_read_many (media_info, GList *locations, ...)
134
135 which would allow the back-end to implement this more efficiently.
136 Suppose an application loads a playlist, for example, then this playlist
137 could be handed to this function, and a GList of metadata types could
138 be returned.
139
140 Possible implementations :
141 - one large XML file : would end up being too heavy
142 - one XML file per dir on system : good compromise; would still make sense
143 to keep this in memory instead of reading and writing it all the time
144 Also, odds are good that users mostly use files from same dir in one app
145 (but not necessarily)
146
147 Possible extra niceties :
148 - matching of moved files, and a similar move of metadata (through user-space
149 tool ?)
150
151 !!! For speed reasons, it might make sense to somehow keep the cache in memory
152 instead of reparsing the same cache file each time.
153
154 !!! For disk space reasons, it might make sense to have a system cache.
155 Not sure if the complexity added is worth it though.
156
157 !!! For disk space reasons, we might want to add an upper limit on the size of
158 the cache. For that we might need a timestamp on last retrieval of metadata,
159 so that we can drop the old ones.
160
161 The cache should use standard glibc.
162 FIXME: is it worth it to use gnome-vfs for this ?
163
164 STANDARDIZATION OF MEDIAINFO
165 ----------------------------
166 Different file formats have different "tags". It is not always possible
167 to map metadata to tags. Some level of agreement on metadata names is also
168 required.
169
170 For media info, the names or properties should be fairly standard.
171 We also use the same names as used for properties and capabilities in
172 GStreamer.
173
174 This means we use
175 - encoded audio
176 - "bitrate" (which is bits per second - use the most correct one,
177 ie. average bitrate for VBR for example)
178
179 - raw audio
180 - "samplerate" - sampling frequency
181 - "channels"
182 - "bitwidth" - how wide is the audio in bits
183
184 - encoded video
185 - "bitrate"
186
187 - raw video
188 (FIXME: I don't know enough about video, are these correct)
189 - "width"
190 - "height"
191 - "colordepth"
192 - "colorspace"
193 - "fps"
194 - "aspectratio"
195
196 We must find a way to avoid collision. A system stream can contain both
197 audio and video (-> bitrate) or multiple audio or video streams. One way
198 to do this might be to make a metadata set for a stream a GList of metadata
199 for elementary streams.
200
201 For metadata and tags, the standards are less clear.
202 Some nice ones to standardize on might be
203 - artist
204 - title
205 - author
206 - year
207 - genre (messy though)
208 - RMS, inpoint, outpoint (calculated through some formula, used for mixing)
209
210 TESTING
211 -------
212 It is important to write a decent testsuite for this and do speed comparisons
213 between the library used and the GStreamer implementation.
214
215
216 API
217 ---
218 struct GstMetadata
219 {
220 gchar *location;
221 GstMetadataType type;
222
223 GList *streams;
224 GHashtable *values;
225 };
226
227 (streams would be a GList of (again) GstMetadata's.
228 "location" would then be reused to indicate an identifier in the stream.
229 FIXME: is that evil ?)
230
231 GstMetadataType - technical, content
232 GstMetadataReadType - cached, raw
233
234 GstMetadata *
235 gst_metadata_read (const char *location,
236 GstMetadataType type,
237 GstMetadataReadType read_type);
238 GstMetadata *
239 gst_metadata_read_props (const char *location,
240 GList *names,
241 GstMetadataType type,
242 GstMetadataReadType read_type);
243 GstMetadata *
244 gst_metadata_read_cached (const char *location,
245 GstMetadataType type,
246 GstMetadataReadType read_type);
247
248 GstMetadata *
249 gst_metadata_read_props_cached (...)
250
251 GList *
252 gst_metadata_read_cached_many (GList *locations,
253 GstMetadataType type,
254 GstMetadataReadType read_type);
255
256 GList *
257 gst_metadata_read_props_cached_many (GList *locations,
258 GList *names,
259 GstMetadataType type,
260 GstMetadataReadType read_type);
261
262 GList *
263 gst_metadata_content_write (const char *location,
264 GstMetadata *metadata);
265
266
267 SOME USEFUL RESOURCES
268 ---------------------
269 http://www.chin.gc.ca/English/Standards/metadata_multimedia.html
270 - describes multimedia data for images
271 distinction between content (descriptive), technical and
272 administrative metadata