| 1 |
purpose |
| 2 |
------- |
| 3 |
|
| 4 |
GStreamer has a very flexible mechanism to describe media types using |
| 5 |
mime types and key/value pairs (GstCaps). The definition of media types |
| 6 |
is entirely done by the plugins which can set the media type to one or |
| 7 |
more of the plugins GstPads. |
| 8 |
|
| 9 |
Pads with the same mime type and 'compatible' properties are allowed to |
| 10 |
connect. It is possible that a pad can accept or produce many different |
| 11 |
media types. |
| 12 |
|
| 13 |
The purpose of caps negotiation is to provide a framework for plugins so |
| 14 |
that they can agree on a common media type for their pads. |
| 15 |
|
| 16 |
|
| 17 |
Capabilities (GstCaps) |
| 18 |
---------------------- |
| 19 |
|
| 20 |
The core component in the caps negotiation system are GstCaps. They consist |
| 21 |
of: |
| 22 |
|
| 23 |
- a name (ex. "my_audio_capabilities") |
| 24 |
- a mime-type (ex. audio/raw, audio/mp3, ...) |
| 25 |
- a list of key/value pairs (ex. channels=2, rate=44100, ...) |
| 26 |
|
| 27 |
The list of key/value pairs is maintained by the GstProps object. |
| 28 |
|
| 29 |
The GstProps object maintains a GList of GstPropsEntries. An entry has |
| 30 |
a key, which is always a string constant (internally converted to a GQuark) |
| 31 |
and a value, which can be one of the following: |
| 32 |
|
| 33 |
- an integer constant (ex. 5) |
| 34 |
- a float constant (ex. 1.0) |
| 35 |
- a string constant (ex. "int") |
| 36 |
- a boolean constant (ex. FALSE) |
| 37 |
- a fourcc constant (ex. I420) |
| 38 |
* fourcc codes are usually used to describe a video format |
| 39 |
|
| 40 |
In addition to these constant values, the following variable values are |
| 41 |
supported too: |
| 42 |
|
| 43 |
- an integer range (ex. 0-200) |
| 44 |
- a float range (ex. 1.0-3.0) |
| 45 |
- a list of values (ex. 1, 2, 5). |
| 46 |
* A List cannot contain another list and the |
| 47 |
entries in the list should all have the |
| 48 |
same type (int, float, string, fourcc). It is |
| 49 |
allowed to mix integers/floats and |
| 50 |
integer/float ranges in a list. |
| 51 |
|
| 52 |
A capability is usually described as follows: |
| 53 |
|
| 54 |
GST_CAPS_NEW ( |
| 55 |
capability name ---> "my_audio_capabilities", |
| 56 |
mime-type ---------> "audio/raw", |
| 57 |
( "format", GST_PROPS_STRING ("int"), |
| 58 |
GstProps ---------> ( "channels", GST_PROPS_INT_RANGE (1, 2), |
| 59 |
(list of entries) ( "rate", GST_PROPS_INT (44100) |
| 60 |
) |
| 61 |
|
| 62 |
(-----------) (--------------------------) |
| 63 |
entry key entry value |
| 64 |
|
| 65 |
Two capabilities can be chained together to form a larger capability: |
| 66 |
|
| 67 |
( GST_CAPS_NEW ( |
| 68 |
( "my_mp3_capability", |
| 69 |
( "audio/mp3", |
| 70 |
one capability ----> ( NULL |
| 71 |
created by chaining ( ), |
| 72 |
two capabilities. ( GST_CAPS_NEW ( |
| 73 |
( "my_vorbis_capability", |
| 74 |
( "audio/vorbis", |
| 75 |
( NULL |
| 76 |
( ) |
| 77 |
|
| 78 |
Capabilities always work as constraints, this means that they constrain the |
| 79 |
media type to the given mime-type and properties. By this definition a NULL |
| 80 |
GstCaps or a NULL GstProps means: no constraints. |
| 81 |
|
| 82 |
|
| 83 |
Variable capabilities vs. fixed capabilities |
| 84 |
-------------------------------------------- |
| 85 |
|
| 86 |
Definition: |
| 87 |
|
| 88 |
A GstProps structure is said to be fixed if it doesn't contain lists or |
| 89 |
ranges, otherwise it is a variable GstProps. A variable GstProps, by definitin |
| 90 |
does not constrain the media type to a set of fixed properties. |
| 91 |
|
| 92 |
A GstCaps is said to be fixed if it is not chained and it doesn't contain |
| 93 |
a variable GstProps component. |
| 94 |
|
| 95 |
|
| 96 |
GstCaps compatibility |
| 97 |
--------------------- |
| 98 |
|
| 99 |
<write me> |
| 100 |
|
| 101 |
GstCaps intersection |
| 102 |
-------------------- |
| 103 |
|
| 104 |
<write me> |
| 105 |
|
| 106 |
|
| 107 |
GstCaps usage |
| 108 |
------------- |
| 109 |
|
| 110 |
GstCaps are used in the following data structures: |
| 111 |
|
| 112 |
- padtemplates. |
| 113 |
* padtemplates are added to elementfactory to describe the possible |
| 114 |
pads that an element created from said factory can have. |
| 115 |
* padtemplates contain the name, direction and presence of the pads. |
| 116 |
* padtemplates also describe the media types that this element can |
| 117 |
accept/produce using GstCaps. |
| 118 |
* padtemplates can provide fixed or variable GstCaps for the pads. |
| 119 |
* padtemplates can be used by the element to create its pads and is |
| 120 |
highly recommended. |
| 121 |
* the padtemplate GstCaps are saved into the registry so that the |
| 122 |
media types an element can operate on, are known without having to |
| 123 |
bring the element into memory. |
| 124 |
|
| 125 |
- pad caps |
| 126 |
* pad caps are _fixed_ caps attached to a pad to describe the exact media |
| 127 |
type this pad is handling. A pad with caps is said to be a "tainted" pad. |
| 128 |
|
| 129 |
- connection filters |
| 130 |
* a connection filter is created when two pads are connected. It describes |
| 131 |
the media type(s) that _can_ flow through this connection. |
| 132 |
|
| 133 |
- application connection filters |
| 134 |
* When the application connects two pads, it can specify an application |
| 135 |
connection filter that will act as additional constraints on the media types |
| 136 |
that can flow through the connection. |
| 137 |
|
| 138 |
Connection filters and application filters are cleared when two connected pads |
| 139 |
are disconnected. Pad caps are not cleared. Padtemplates are immutable and |
| 140 |
never cleared. |
| 141 |
|
| 142 |
|
| 143 |
The GstPad get_caps function |
| 144 |
---------------------------- |
| 145 |
|
| 146 |
the gst_pad_get_caps function returns the caps of a given pad. The pad caps are |
| 147 |
calculated as: |
| 148 |
|
| 149 |
- if the pad has pad caps, return those |
| 150 |
- else if the pad has a getcaps function, call the function and return the result |
| 151 |
- else if the pad has a padtemplate, return the padtemplate caps |
| 152 |
- else return NULL |
| 153 |
|
| 154 |
|
| 155 |
Purpose of caps negotiation |
| 156 |
--------------------------- |
| 157 |
|
| 158 |
The purpose of the caps negotiation procedure is to set "pad caps" on a pad |
| 159 |
so that it is compatible with the "connection filter". This has to be done |
| 160 |
_before_ any data passing happens through the connection. Data passing between two |
| 161 |
pads is _not_ allowed when the pad caps are not compatible with the connection |
| 162 |
filter or when the pad caps of two pads participating in the connection are not |
| 163 |
equal. |
| 164 |
|
| 165 |
Real caps negotiation starts as soon as an element is in the READY state or higher. |
| 166 |
This means that the connect functions of the pads are only called when the element |
| 167 |
is at least READY. The intersection between two pads is made at connect time, |
| 168 |
regardless of element state. |
| 169 |
|
| 170 |
|
| 171 |
GstPad connection |
| 172 |
----------------- |
| 173 |
|
| 174 |
When two pads are connected get_caps is called on both pads to get their caps. |
| 175 |
Then the intersection between those caps is made, this will give us all the |
| 176 |
possible media types that can flow through this pad connection. Optionally the |
| 177 |
application can provide additional caps, the pad intersection is also made with |
| 178 |
the application caps. |
| 179 |
|
| 180 |
The intersection and the optional application caps are stored in the two pads. |
| 181 |
|
| 182 |
If the intersection is NULL, the two pads have no common types and the connection |
| 183 |
is refused. |
| 184 |
|
| 185 |
If the intersection is a fixed caps, this means there is only one possible media type |
| 186 |
that can be used for this connection. |
| 187 |
|
| 188 |
For all not NULL intersections the pad connect functions are called with the |
| 189 |
intersection. Depending on the result of the connect function the connection is |
| 190 |
allowed or refused. |
| 191 |
|
| 192 |
If the intersection is fixed and the pad connect functions agreed to the caps, |
| 193 |
the caps are set on the pads. |
| 194 |
|
| 195 |
Note that pad caps are never set for non fixed caps. |
| 196 |
|
| 197 |
!Example 1: |
| 198 |
! |
| 199 |
! 1. before connecting the pads, they both have a set of possible media types, |
| 200 |
! on the pad, through the getcaps function or on the padtemplate (here |
| 201 |
! represented with capital letters) |
| 202 |
! |
| 203 |
! srcpad sinkpad |
| 204 |
! A B |
| 205 |
! B F |
| 206 |
! C A |
| 207 |
! G |
| 208 |
! |
| 209 |
! 2. when performing the connection, the intersection between the two sets of caps |
| 210 |
! is made. |
| 211 |
! |
| 212 |
! srcpad sinkpad |
| 213 |
! A ) ( B |
| 214 |
! B )--------> A <------( F |
| 215 |
! C ) B ( A |
| 216 |
! ( G |
| 217 |
! |
| 218 |
! 3. In this case the intersection is not a fixed caps (it's a chained caps). |
| 219 |
! the connect function of the two pads are called (if any), the connect |
| 220 |
! function can accept of refuse the caps. |
| 221 |
! |
| 222 |
! 4. if the caps are accepted, the intersection caps are set on both pads. |
| 223 |
! |
| 224 |
! 5. plugins will typically not configure themselves if they get variable caps. |
| 225 |
! It is possible though for a plugin to select one of the caps, fixate |
| 226 |
! some properties and refine the filter to fixed caps (see later) |
| 227 |
! |
| 228 |
|
| 229 |
|
| 230 |
!Example 2: |
| 231 |
! |
| 232 |
! 1. we take two pads that intersect to fixed caps (B): |
| 233 |
! |
| 234 |
! srcpad sinkpad |
| 235 |
! A ) ( B |
| 236 |
! B )-------> B <-------( F |
| 237 |
! |
| 238 |
! 2. both pad connect functions are called. |
| 239 |
! |
| 240 |
! 3. assuming the connect functions did not refuse the connection, the caps |
| 241 |
! are set on both pads (because they are fixed). |
| 242 |
! |
| 243 |
! srcpad (B)--------------(B) sinkpad |
| 244 |
! A B |
| 245 |
! B F |
| 246 |
! |
| 247 |
|
| 248 |
!Example 3: |
| 249 |
! |
| 250 |
! 1. we take two pads, one with pad caps, another one with a padtemplate. |
| 251 |
! |
| 252 |
! srcpad (B) sinkpad |
| 253 |
! ! ( B |
| 254 |
! ------> B <-------( F |
| 255 |
! |
| 256 |
! 2. the pad get_caps function will return the pad caps of the srcpad and |
| 257 |
! the padtemplate caps of the sinkpad. The intersection is made, yielding |
| 258 |
! a fixed caps (B) which is sent to both connect functions (if any) |
| 259 |
! |
| 260 |
! 3. If the connect function(s) didn't refuse the connection the fixed caps |
| 261 |
! are set on both pads. The intersection is also kept in the pad object. |
| 262 |
! |
| 263 |
! srcpad (B)--------------(B) sinkpad |
| 264 |
! B |
| 265 |
! F |
| 266 |
! |
| 267 |
! 4. when fixed caps are received on the sinkpad the plugin will typically |
| 268 |
! configure itself to deal with the format. |
| 269 |
! |
| 270 |
|
| 271 |
|
| 272 |
Pad connect functions |
| 273 |
--------------------- |
| 274 |
|
| 275 |
A Pad can be notified when another pad is connected or reconnected to it with |
| 276 |
fixed or variable caps. This notification will be done with the optional |
| 277 |
GstPadConnectFunction callback that an element can provide for a pad. |
| 278 |
Remember that this connection is _always_ called, not only with fixed caps |
| 279 |
but also with variable caps (if any). |
| 280 |
|
| 281 |
We will explain some of the common situations with some examples. |
| 282 |
|
| 283 |
!Example 4: |
| 284 |
! |
| 285 |
! 1. We have a simple mpeg audio decoder connected to a simple audio sink. |
| 286 |
! The mpeg decoder doesn't have a connect function on |
| 287 |
! |
| 288 |
! |
| 289 |
! |
| 290 |
! |
| 291 |
! |
| 292 |
! |
| 293 |
! |
| 294 |
! |
| 295 |
! |
| 296 |
! |
| 297 |
! |
| 298 |
! |
| 299 |
! |
| 300 |
! |
| 301 |
! |
| 302 |
! |
| 303 |
|
| 304 |
|
| 305 |
|
| 306 |
|
| 307 |
|
| 308 |
|
| 309 |
|
| 310 |
|
| 311 |
|
| 312 |
|
| 313 |
|
| 314 |
|
| 315 |
|
| 316 |
|
| 317 |
|
| 318 |
|
| 319 |
|
| 320 |
|
| 321 |
|