| 1 |
A lot of data streams need a few initial buffers to make sense of the |
| 2 |
data stream. With these initial buffers, it can pick up at any point in |
| 3 |
the rest of the data stream. |
| 4 |
|
| 5 |
Examples: |
| 6 |
- Vorbis and Theora have three initial Ogg packets |
| 7 |
- MPEG4 has codec data |
| 8 |
- GDP protocol serializes the initial new_segment event and the initial caps |
| 9 |
|
| 10 |
It is important that elements that get connected after the stream starts, |
| 11 |
receive these initial buffers. Also, sink elements should know that these |
| 12 |
initial buffers are necessary for every new "client" of the sink element; |
| 13 |
for example, multifdsink needs to send these initial buffers before the |
| 14 |
normal buffers. |
| 15 |
|
| 16 |
Currently, this is done by putting a 'streamheader' value on the caps. |
| 17 |
'streamheader' is a GST_TYPE_ARRAY of GstBuffer, ie. an ordered list of |
| 18 |
buffers. |
| 19 |
|
| 20 |
The buffers themselves still get pushed as normal buffers, but with the |
| 21 |
IN_CAPS flag set. This allows elements that do not know about streamheader |
| 22 |
to still function correctly (ie. filesink and others) |
| 23 |
|
| 24 |
It is of course important that these streamheader buffers are not sent twice |
| 25 |
(once because they're in the caps, and once because they're received as |
| 26 |
buffers). |
| 27 |
|
| 28 |
So, an element that is aware of streamheader in the caps should probably best |
| 29 |
drop all incoming IN_CAPS buffers. |
| 30 |
|
| 31 |
Rules for sending streamheaders: |
| 32 |
- all streamheader buffers should have IN_CAPS set |
| 33 |
- from this list of streamheader buffers, a "streamheader" caps field should |
| 34 |
be created as a GST_TYPE_ARRAY of GstBuffer. This should be done by |
| 35 |
copying the buffers. |
| 36 |
(the only important thing about the buffers in this list is the data; |
| 37 |
other attributes of the buffers get ignored) |
| 38 |
- each of these buffers should then have these caps set on them. |
| 39 |
- when all streamheader buffers are collected in the element, pad caps should |
| 40 |
be set, including this streamheader |
| 41 |
- streamheader buffers should be sent consecutively, and before any of the |
| 42 |
data (non-IN_CAPS) buffers they apply to. If necessary, the element |
| 43 |
should internally queue non-IN_CAPS buffers until the streamheaders |
| 44 |
are completely assembled. |
| 45 |
- when new streamheader buffers need to be pushed out, this process is |
| 46 |
repeated. Receiving a new IN_CAPS buffer after a non-IN_CAPS buffer |
| 47 |
signifies resetting streamheader, as does the new set_caps with different |
| 48 |
streamheader right before. (FIXME: it's probably better to explicitly |
| 49 |
have an event/buffer that clears streamheaders; consider the case of |
| 50 |
an element like GDP that has created streamheader from the first newsegment |
| 51 |
and the first caps, and then receives a new tag event that it also wants |
| 52 |
to put on the streamheader - it should be able to invalidate the previous |
| 53 |
ones) |
| 54 |
|
| 55 |
Elements that can send streamheader caps: |
| 56 |
- vorbisenc |
| 57 |
- theoraenc |
| 58 |
- gdppay |
| 59 |
|
| 60 |
Elements that can receive streamheader caps: |
| 61 |
- multifdsink |
| 62 |
|
| 63 |
Elements that can receive and send streamheader caps: |
| 64 |
- oggmux |
| 65 |
|
| 66 |
Elements that could/should use this: |
| 67 |
- ffenc_mpeg4 (currently uses/sets codec_data) |
| 68 |
- theoradec, vorbisdec (currently only look at incoming buffers) |
| 69 |
- oggdemux |
| 70 |
|
| 71 |
FUTURE PLANS |
| 72 |
------------ |
| 73 |
The concept of streamheader is more generally applicable. We may want to |
| 74 |
find a better way of implementing this than having two ways of telling |
| 75 |
downstream elements about them. Currently an element that is |
| 76 |
streamheader-aware needs to look both at caps and incoming buffers. |
| 77 |
|
| 78 |
One option would be to only rely on the buffer flow, and make all elements |
| 79 |
aware of a HEADER flag, causing them to keep these buffers around and push |
| 80 |
them for each new pad link. This needs to be done in every element; a |
| 81 |
scenario like |
| 82 |
videotestsrc ! theoraenc |
| 83 |
where later on an oggmux gets connected, should still work without dropping |
| 84 |
these header buffers. But similarly, |
| 85 |
videotestsrc ! theoraenc ! identity |
| 86 |
with a later connection of oggmux to identity should work. |
| 87 |
|
| 88 |
This could tie in with an idea to have pads store some initial data (the |
| 89 |
first new_segment event, tag events, ...), which at the moment also get |
| 90 |
dropped on dynamic in-stream links. |