| 1 |
In this document we describe how we can obtain various properties |
| 2 |
of the pipeline we are running. |
| 3 |
|
| 4 |
we have 5 possible ways to get information, each one of these |
| 5 |
methods focus on one particular property of the pipeline. |
| 6 |
|
| 7 |
- caps: this is a description of the media type that flows between |
| 8 |
2 pads. |
| 9 |
- metadata: non essential extra (human readable) information about |
| 10 |
the stream, like author, copyright, name etc.. |
| 11 |
- streaminfo: information about the stream as encoded into the |
| 12 |
stream itself. |
| 13 |
- pad/element queries: information about the stream as it is being |
| 14 |
processed. |
| 15 |
- pad/element convert: information about the relation between formats |
| 16 |
in a streams as it is being processed. |
| 17 |
|
| 18 |
note that element properties are not included in this list. Element |
| 19 |
properties are only used to configure the codec/element. |
| 20 |
|
| 21 |
caps, metadata, streaminfo are supposed to remain fairly static during |
| 22 |
the stream. The queries/converts can be done on demand. The reason |
| 23 |
for this is that caps/metadata/streaminfo is quite expensive and |
| 24 |
could degrade pipeline performance. |
| 25 |
|
| 26 |
|
| 27 |
Caps. |
| 28 |
----- |
| 29 |
|
| 30 |
Caps are automatically set on pads by the core when two pads agree |
| 31 |
on a media type. This automatically means that the caps are fixed. |
| 32 |
|
| 33 |
Since caps is a property of the pad and the g_object_notify() mechanism |
| 34 |
is used to signal a change, the user can either connect a "notify" |
| 35 |
signal handler to the pad or connect to the "deep_notify" signal on |
| 36 |
a parent pipeline. |
| 37 |
|
| 38 |
The caps notifications are useful if you want to know what kind of |
| 39 |
media is passing through pads. You can, for example, report to the |
| 40 |
user how the video or audiosink is configured or what kind of |
| 41 |
media some plugin is producing/accepting. |
| 42 |
|
| 43 |
So, always use the caps to find out the channels/samplerate/size |
| 44 |
of the media. |
| 45 |
|
| 46 |
|
| 47 |
Metadata |
| 48 |
-------- |
| 49 |
|
| 50 |
Metadata is a GstCaps element property (named "metadata") that contains |
| 51 |
additional information encoded into the stream that doesn't say anything |
| 52 |
about the media type of the stream itself. |
| 53 |
|
| 54 |
Metadata are typically human readable information like author, copyright, |
| 55 |
title, ... and can be displayed by the application as-is. |
| 56 |
|
| 57 |
An element with a "metadata" property is supposed to g_object_notify that |
| 58 |
property when the metadata changes so that the app can connect a signal |
| 59 |
handler to the property or use the "deep_notify" signal to get the |
| 60 |
notification. |
| 61 |
|
| 62 |
|
| 63 |
Streaminfo |
| 64 |
---------- |
| 65 |
|
| 66 |
Streaminfo is a GstCaps element property (named "streaminfo") that contains |
| 67 |
additional information about the stream that is not stricly required to |
| 68 |
specify the media type of the stream. |
| 69 |
|
| 70 |
Streaminfo is typically the length, bitrate, framerate, flags, ... of the |
| 71 |
stream. |
| 72 |
|
| 73 |
It is important to note that this information should be derived from the |
| 74 |
stream itself and might not be correct for the pipeline being processed. |
| 75 |
Let's illustrate this with an example: |
| 76 |
|
| 77 |
- an mp3 stream has an id3 tags that contains the length of the stream |
| 78 |
(TLEN). |
| 79 |
- we cut the mp3 stream in half |
| 80 |
|
| 81 |
The actual length doesn't match the stated length as encoded in the id3 |
| 82 |
tags, so the TLEN tag should be put in the streaminfo and the actual |
| 83 |
length should be queried with a pad_query. |
| 84 |
|
| 85 |
So, be careful when showing streaminfo as-is in an app. |
| 86 |
|
| 87 |
Queries |
| 88 |
------- |
| 89 |
|
| 90 |
Queries can be performed on pads and elements and are used to get |
| 91 |
information about the current stream. The value is a single gint64 |
| 92 |
in a specific format. |
| 93 |
|
| 94 |
example: |
| 95 |
|
| 96 |
- the query (GST_QUERY_TOTAL, GST_FORMAT_TIME) will return the total |
| 97 |
amount of time this pad/element will run. |
| 98 |
|
| 99 |
- the query (GST_QUERY_POSITION, GST_FORMAT_UNITS) will return the |
| 100 |
current position of the stream expressed in units (units are samples, |
| 101 |
frames, bytes, ... depending on the media type of the stream) |
| 102 |
|
| 103 |
two methods exist to perform a query: |
| 104 |
|
| 105 |
- gboolean gst_pad_query (GstPad *pad, GstQueryType type, |
| 106 |
GstFormat *format, gint64 *value); |
| 107 |
|
| 108 |
and: |
| 109 |
|
| 110 |
- gboolean gst_element_query (GstElement *element, GstQueryType type, |
| 111 |
GstFormat *format, gint64 *value); |
| 112 |
|
| 113 |
if you want to get the total duration of a stream or the current position, |
| 114 |
you need to use a pad query. A pad query can fail (method returns FALSE), |
| 115 |
this usually means that the duration is not known or that not enough data |
| 116 |
has been processed to report the correct value. |
| 117 |
|
| 118 |
the possible queries that can be performed on a pad/element can be obtained |
| 119 |
with |
| 120 |
|
| 121 |
- const GstQueryType* gst_pad_get_query_types (GstPad *pad); |
| 122 |
|
| 123 |
and |
| 124 |
|
| 125 |
- const GstQueryType* gst_element_get_query_types (GstElement *element); |
| 126 |
|
| 127 |
These functions return an array of GstQueryTypes (last element == 0). you |
| 128 |
can loop over the array to see what is supported or do |
| 129 |
|
| 130 |
- gboolean gst_queries_contains (const GstQueryType *types, GstQueryType type); |
| 131 |
|
| 132 |
to see if a specific format is contained in the list. |
| 133 |
|
| 134 |
The value reported by a pad query can be considered correct or |
| 135 |
as-good-as-can-be. Elements that support pad queries must do the best |
| 136 |
they can to report correct values. |
| 137 |
|
| 138 |
|
| 139 |
Convert |
| 140 |
------- |
| 141 |
|
| 142 |
The convert functions are used to query an element/pad for the relationship |
| 143 |
between two formats that it supports. For example: |
| 144 |
|
| 145 |
suppose we want to know how many frames a particular video decoder will |
| 146 |
produce in one second, we ask it to convert its concept of 1 SECOND into |
| 147 |
1 UNIT (frames in the context of video). so, |
| 148 |
|
| 149 |
GstFormat format = GST_FORMAT_UNITS; |
| 150 |
|
| 151 |
res = gst_pad_convert (pad, GST_FORMAT_TIME, GST_SECOND, |
| 152 |
&format, &value); |
| 153 |
|
| 154 |
if res == TRUE, value will contain the framerate of the video, of course |
| 155 |
this framerate will only contain the integral part. If you want more |
| 156 |
accuracy, use 1000 * GST_SECOND and divide the result by 1000 to get |
| 157 |
a fractional part. |
| 158 |
|
| 159 |
All other neat things can be done too, look at the typical cdplayer |
| 160 |
plugin for example. It defines a new format "track". Now you can |
| 161 |
ask it to convert a TRACK to TIME like this: |
| 162 |
|
| 163 |
|
| 164 |
GstFormat format = GST_FORMAT_TIME; |
| 165 |
track_format = gst_format_get_by_nick ("track"); |
| 166 |
res = gst_pad_convert (pad, track_format, 1, |
| 167 |
&format, &value); |
| 168 |
|
| 169 |
|
| 170 |
This will convert 1 track to a time. of course we didn't (couldn't) specify |
| 171 |
which track, but that's not a problem if you understand how the stream is |
| 172 |
divided into different formats: |
| 173 |
|
| 174 |
Take the total stream as containing bytes (indicated with + in figure) we |
| 175 |
can subdivide the (byte)stream in different formats. |
| 176 |
|
| 177 |
|
| 178 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ... bytes |
| 179 |
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ... samples (units) |
| 180 |
! ! ! ! ... buffers |
| 181 |
! ! ! ... time |
| 182 |
! ! ... track |
| 183 |
|
| 184 |
The raw bytestream can be grouped in samples for example (for 16 bit stereo |
| 185 |
int audio, 4 bytes == 1 sample) or we can divide it into time |
| 186 |
(44100 samples == 176400 bytes == 1 second) or into tracks |
| 187 |
(1 track could be 17640000 bytes or 100 seconds or 4410000 samples) |
| 188 |
|
| 189 |
It is important to know that the stream starts at position 0 (for all formats) |
| 190 |
and ends at position X (expressed in a specific format). now take this stream |
| 191 |
divided into the track format: |
| 192 |
|
| 193 |
0 200 500 700 1000 |
| 194 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. bytes |
| 195 |
! track0 ! track1 ! track2 ! track3 ! ... |
| 196 |
|
| 197 |
if we now perform a pad convert from 100 bytes to the track format, we |
| 198 |
get the value 0, as the region 0-100 bytes is contained in track0. |
| 199 |
if we perform a pad convert from 600 bytes to track, we get the value |
| 200 |
2, as track0->track2 contains the bytes 0-500. |
| 201 |
|
| 202 |
We can also do: convert track1 to bytes, then we get 200. If we do |
| 203 |
convert track2 to bytes, we get 500. Note that the conversions are |
| 204 |
always performed relative to 0, so if we convert track2 to bytes, we |
| 205 |
always get the number of bytes from track0->track2. |
| 206 |
|
| 207 |
If we want to get the number of bytes of one particular track, we have |
| 208 |
to subtract two convert values. Look at the following figure to understand |
| 209 |
this. The --- defines the region we want to convert. |
| 210 |
|
| 211 |
|
| 212 |
0 200 500 700 1000 |
| 213 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. bytes |
| 214 |
! track0 ! track1 ! track2 ! track3 ! ... |
| 215 |
|
| 216 |
----------- track1 -> bytes (1) (size of track0) |
| 217 |
|
| 218 |
---------------------- track2 -> bytes (2) (size of track0 and track1) |
| 219 |
|
| 220 |
----------- (2) - (1) = total bytes for track 1 |
| 221 |
|
| 222 |
Another example would be to get the bitrate of a decoder plugin, consider the |
| 223 |
following example: |
| 224 |
|
| 225 |
(------------) |
| 226 |
! mad ! |
| 227 |
- sink src - |
| 228 |
(------------) |
| 229 |
|
| 230 |
The element has a sinkpad that will take N bytes as input to produce M |
| 231 |
samples (units) on its srcpad. |
| 232 |
The rate at which it takes bytes is defined as the byterate of the |
| 233 |
stream (bitrate == byterate * 8). So, we do: |
| 234 |
|
| 235 |
GstFormat format = GST_FORMAT_BYTES; |
| 236 |
gint64 value; |
| 237 |
|
| 238 |
gst_pad_convert (mad->sinkpad, GST_FORMAT_TIME, GST_SECOND, |
| 239 |
&format, &value); |
| 240 |
|
| 241 |
..and we get the number of bytes this plugin takes in each second. |
| 242 |
Again, note that this value is relative to 0, you can get an average |
| 243 |
of a specific period by using the same subtract trick as above. |
| 244 |
|
| 245 |
|
| 246 |
Element Properties |
| 247 |
------------------ |
| 248 |
|
| 249 |
Element properties are used to configure an element. They should not be |
| 250 |
used to describe media info, metadata fields, streaminfo fields or anything |
| 251 |
other that doesn't involve codec configuration. |
| 252 |
|
| 253 |
Several reasons: |
| 254 |
|
| 255 |
- metadata requires dynamic properties (one for each tag). This cannot be done |
| 256 |
with GObject properties. |
| 257 |
- you cannot signal a logical group of related properties (exposing stuff like |
| 258 |
samplerate/channels/encoding/... in different element properties is not a |
| 259 |
good idea. |
| 260 |
- stuff like length an position depend on the pads of the element, you cannot |
| 261 |
sanely expose a property for each pad to describe this. |
| 262 |
- element properties can only report stuff with one type. If your property |
| 263 |
exposes something like "total_length", you cannot make it both report this |
| 264 |
in time/bytes/samples/frames... |
| 265 |
- impossible to sanely implement convert with element properties. |
| 266 |
|