| 1 |
|
| 2 |
the current capsnegotiation sucks and plainly doesn't work for queues. |
| 3 |
one of the reasons is that the core is trying to do too much, like |
| 4 |
being a mediator in the negotiation process. There is also way too much |
| 5 |
interaction between plugins. Plugins have to keep a lot of state too. |
| 6 |
|
| 7 |
The proposed method works by sending chains of caps instead, requiring |
| 8 |
a worst case of 2 interactions between plugins. The drawback is that |
| 9 |
a negotiating plugin has to do a bit more work, but in pactice this is |
| 10 |
not a real argument against the proposed method, as the actions it has |
| 11 |
to perform cannot be avoided. We will provide sufficient APIs to |
| 12 |
facilitate this. |
| 13 |
|
| 14 |
We are not going to send random chains of caps between plugins but we |
| 15 |
are going to filter them by using a new gst_caps_intersect() function |
| 16 |
that can find the common media types and properties between the plugins. |
| 17 |
The main idea is that the simple case should remain simple. |
| 18 |
|
| 19 |
We also introduce app specific filters that can be set while doing a |
| 20 |
pad connection. |
| 21 |
|
| 22 |
We proceed with a few use cases because they tend to explain the ideas |
| 23 |
better. |
| 24 |
|
| 25 |
|
| 26 |
use case 1 |
| 27 |
---------- |
| 28 |
|
| 29 |
v4lsrc -> xvideosink |
| 30 |
|
| 31 |
no pad restrictions set on pad_connect |
| 32 |
v4lsrc can do many formats, so can xvideosink |
| 33 |
|
| 34 |
- v4lsrc request caps: |
| 35 |
|
| 36 |
width and height properties were set on v4lsrc so it provides them as hints. |
| 37 |
|
| 38 |
GstCaps *peercaps = gst_pad_request_caps (v4lsrc->srcpad, |
| 39 |
GST_CAPS_NEW ( |
| 40 |
"v4lsrc_srccaps", |
| 41 |
"video/raw", |
| 42 |
"width", GST_PROPS_INT (352), |
| 43 |
"height", GST_PROPS_INT (288))); |
| 44 |
|
| 45 |
- core fetches xvideosink caps. if xvideosink has a request_caps function, |
| 46 |
then the intersection of the hint, v4lsrcpad template caps and xvideosink |
| 47 |
sinkpad template are sent as input. |
| 48 |
If xvideosink has no request_caps function, the padtemplate |
| 49 |
caps of xvideosink are taken. |
| 50 |
- after intersect on both caps, peercaps equals: |
| 51 |
|
| 52 |
peercaps == |
| 53 |
gst_caps_chain ( |
| 54 |
GST_CAPS_NEW ( |
| 55 |
"xvideosink_caps", |
| 56 |
"video/raw", |
| 57 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("YV12")), |
| 58 |
"width", GST_PROPS_INT (352), |
| 59 |
"height", GST_PROPS_INT (288) |
| 60 |
), |
| 61 |
GST_CAPS_NEW ( |
| 62 |
"xvideosink_caps", |
| 63 |
"video/raw", |
| 64 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("I420")), |
| 65 |
"width", GST_PROPS_INT (352), |
| 66 |
"height", GST_PROPS_INT (288) |
| 67 |
), |
| 68 |
GST_CAPS_NEW ( |
| 69 |
"xvideosink_caps", |
| 70 |
"video/raw", |
| 71 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("YUY2")), |
| 72 |
"width", GST_PROPS_INT (352), |
| 73 |
"height", GST_PROPS_INT (288) |
| 74 |
), |
| 75 |
GST_CAPS_NEW ( |
| 76 |
"xvideosink_caps", |
| 77 |
"video/raw", |
| 78 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("UYVY")), |
| 79 |
"width", GST_PROPS_INT (352), |
| 80 |
"height", GST_PROPS_INT (288) |
| 81 |
), |
| 82 |
GST_CAPS_NEW ( |
| 83 |
"xvideosink_caps", |
| 84 |
"video/raw", |
| 85 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("RGB ")), |
| 86 |
"bpp", GST_PROPS_INT (16), |
| 87 |
"depth", GST_PROPS_INT (16), |
| 88 |
"endianness", GST_PROPS_INT (G_BYTE_ORDER), |
| 89 |
"red_mask", GST_PROPS_INT (0xf800), |
| 90 |
"green_mask", GST_PROPS_INT (0x07e0), |
| 91 |
"blue_mask", GST_PROPS_INT (0x001f), |
| 92 |
"width", GST_PROPS_INT (352), |
| 93 |
"height", GST_PROPS_INT (288) |
| 94 |
) |
| 95 |
) |
| 96 |
|
| 97 |
- v4lsrc does: |
| 98 |
|
| 99 |
while (peercaps) { |
| 100 |
/* check if the caps can be used */ |
| 101 |
if (caps_are_useful (peercaps)) { |
| 102 |
break; |
| 103 |
} |
| 104 |
... |
| 105 |
|
| 106 |
peercaps = gst_caps_new (peercaps); |
| 107 |
} |
| 108 |
|
| 109 |
gst_pad_set_caps (v4lsrc->srcpad, peercaps); |
| 110 |
|
| 111 |
- core check compatibility with v4lsrc and xvideosink caps. if all goes well |
| 112 |
caps are set on the pads. |
| 113 |
|
| 114 |
- negotiation was more effective if v4lsrc provided srcpad caps since maybe |
| 115 |
YUY2 etc could be removed in the intersect (when v4lsrc doesn't support that |
| 116 |
format). |
| 117 |
|
| 118 |
|
| 119 |
use case 2 |
| 120 |
---------- |
| 121 |
|
| 122 |
mpeg2dec -> colorspace -> xvideosink |
| 123 |
|
| 124 |
|
| 125 |
- mpeg2dec does: |
| 126 |
|
| 127 |
gst_pad_set_caps (mpeg2dec->srcpad, |
| 128 |
GST_CAPS_NEW ( |
| 129 |
"mpeg2dec_srccaps", |
| 130 |
"video/raw", |
| 131 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("I420")), |
| 132 |
"width", GST_PROPS_INT (720), |
| 133 |
"height", GST_PROPS_INT (480) |
| 134 |
) |
| 135 |
); |
| 136 |
|
| 137 |
- core checks compatibility with mpeg2dec srcpad caps and colorspace caps. |
| 138 |
- newcaps function of colorspace is called. |
| 139 |
- colorspace requests peer pad caps with the hint. |
| 140 |
- xvideosink doesn't have a request_caps function, the hint is ignored and |
| 141 |
the padtemplate caps are returned. |
| 142 |
- colorspace runs intersect on the two caps. |
| 143 |
- if the intersection is NULL, colorspace has to set up a conversion function. |
| 144 |
- runs through the list of caps, picking the first one for which a |
| 145 |
converter exists between the received caps and the target caps. set up |
| 146 |
the converter and set the target caps on the srcpad. |
| 147 |
- if the intersection is not NULL, set received caps on colorspace srcpad. |
| 148 |
|
| 149 |
|
| 150 |
use case 3 |
| 151 |
---------- |
| 152 |
|
| 153 |
mpeg2dec -> queue -> colorspace -> queue -> xvideosink |
| 154 |
|
| 155 |
|
| 156 |
Same as use case 2, really. the queue just forwards the caps and the |
| 157 |
request caps. |
| 158 |
|
| 159 |
can we use a default implementation for this proxying? probably. |
| 160 |
for an element with no newcaps function or request_caps function, we can |
| 161 |
simply set the caps on all srcpads when a set_caps is performed, or we |
| 162 |
can request and intersect all srcpad caps when a peer element does |
| 163 |
request_caps. This will actually work for tee too, to some degree. |
| 164 |
|
| 165 |
|
| 166 |
use case 4 |
| 167 |
---------- |
| 168 |
|
| 169 |
gst_pad_connect_caps (...), |
| 170 |
gst_element_connect_caps (v4lsrc, "src", xvideosink, "sink", |
| 171 |
GST_CAPS_NEW ( |
| 172 |
"app_caps", |
| 173 |
"video/raw", |
| 174 |
"format", GST_PROPS_FOURCC (GST_STR_FOURCC ("I420")), |
| 175 |
"width", GST_PROPS_INT (352), |
| 176 |
"height", GST_PROPS_INT (288) |
| 177 |
) |
| 178 |
); |
| 179 |
|
| 180 |
- as long as the connection is not broken, the caps are set as a filter to the |
| 181 |
to pads. |
| 182 |
|
| 183 |
- when v4lsrc requests pads, the intersection with the filter is also made. |
| 184 |
|
| 185 |
- v4lsrc has no other choice but to use I420. |
| 186 |
|
| 187 |
use case 5 |
| 188 |
---------- |
| 189 |
|
| 190 |
mad -> osssink |
| 191 |
|
| 192 |
in this case, mad can both send "int" audio samples and "float" samples. |
| 193 |
|
| 194 |
- mad request_caps from osssink, sending the fixed properties as hints (rate, |
| 195 |
channels, ...) |
| 196 |
- osssink has a request_caps function, the core first makes the intersection |
| 197 |
between the hint and mad srcpad template. the core then does the intersection |
| 198 |
between osssink sinkpad template and sends this list to osssinks request_caps |
| 199 |
function. osssink can try this list (or whatever remains of it) and whatever |
| 200 |
works is sent back as the return value of the request_caps function. |
| 201 |
- mad receives the caps list, takes the top caps and does a pad_set_caps |
| 202 |
- osssink receives the final caps on its new_caps function and configures the |
| 203 |
oss driver. |
| 204 |
|
| 205 |
|
| 206 |
use case 6 |
| 207 |
---------- |
| 208 |
|
| 209 |
gsm -> audioscaler -> osssink |
| 210 |
|
| 211 |
Somebody sets some crazy low rate on the gsm srcpad (1000Hz). |
| 212 |
|
| 213 |
- audioscalers newcaps function is called. |
| 214 |
- audioscaler does a request_caps on its srcpad with the provided |
| 215 |
caps as a hint. |
| 216 |
- osssink tries the caps in the request_caps function (by opening the |
| 217 |
device, seeing that it doesn't support this rate at all...) |
| 218 |
- osssink tries to comply as closely with the rate and sends the |
| 219 |
intersection between its padtemplate caps and the findings (rate 8000Hz |
| 220 |
is possible). |
| 221 |
- audioscaler does the intersection between the caps list and the hint. |
| 222 |
- intersection == NULL, a converter is needed between the hint and |
| 223 |
the first caps of the request_caps list. |
| 224 |
- intersection != NULL, set those caps (== the hint). |
| 225 |
|
| 226 |
|
| 227 |
|