| 1 |
purpose |
| 2 |
------- |
| 3 |
|
| 4 |
A proposal for proper syncing and clocking of a pipeline. |
| 5 |
|
| 6 |
Requirements |
| 7 |
------------ |
| 8 |
|
| 9 |
- elements should be able to get the time and wait for a specific |
| 10 |
time |
| 11 |
- some elements should be able to control and adjust the clock. |
| 12 |
(clock master) |
| 13 |
- the application should be able to provide another clock. |
| 14 |
|
| 15 |
Clocks |
| 16 |
------ |
| 17 |
|
| 18 |
A clock extends the abstract GstClock class. |
| 19 |
|
| 20 |
gst_clock_get_time should always report an equal or increasing |
| 21 |
value with each successive call. |
| 22 |
|
| 23 |
|
| 24 |
|
| 25 |
Clock providers |
| 26 |
--------------- |
| 27 |
|
| 28 |
Clock providers call gst_element_provides_clock (element, clock) |
| 29 |
in their _init function. This will set some element flags. |
| 30 |
This also means that a clock provider cannot stop being a clock |
| 31 |
provider. |
| 32 |
|
| 33 |
Clock providers will update the clock at specific intervals. |
| 34 |
get_resolution/set_resolution can be used to control the resolution. |
| 35 |
|
| 36 |
When a clock provider is not going to update the clock anymore |
| 37 |
it should make sure elements still blocked on the clock get |
| 38 |
unblocked at the right time. This can be done by converting all |
| 39 |
blocking waits to a select call. All further waits on the clock |
| 40 |
should be estimated (using gettimeofday for example) |
| 41 |
|
| 42 |
|
| 43 |
Clock receivers |
| 44 |
--------------- |
| 45 |
|
| 46 |
An element needing a clock must implement the element receive_clock method. |
| 47 |
It has to use the clock received in this function or if the |
| 48 |
clock == NULL it should not do any waiting at all. |
| 49 |
|
| 50 |
|
| 51 |
Clocks and the scheduler |
| 52 |
------------------------ |
| 53 |
|
| 54 |
The scheduler knows about the clocks, else a clock provider and receiver |
| 55 |
in the same scheduler could cause a deadlock. |
| 56 |
|
| 57 |
Only one clock provider is allowed per scheduler. multiple clock |
| 58 |
receivers are allowed per scheduler. |
| 59 |
|
| 60 |
|
| 61 |
|
| 62 |
Specifying clocks |
| 63 |
----------------- |
| 64 |
|
| 65 |
on every bin with a scheduler, gst_bin_use_clock (bin, clock) can be used |
| 66 |
to force the use of this specific clock for all the elements in this bin. |
| 67 |
|
| 68 |
gst_bin_use_clock (bin, NULL) to disable all clocking for this bin. |
| 69 |
|
| 70 |
gst_bin_auto_clock (bin) to use the default algorithm to find a clock. |
| 71 |
|
| 72 |
bins that get a use clock set a flag and store the clock, they also call |
| 73 |
set_clock on all of their children. When other children are added, they |
| 74 |
all get the stored clock. |
| 75 |
|
| 76 |
|
| 77 |
Clock distribution |
| 78 |
------------------ |
| 79 |
|
| 80 |
clock providers and receivers are collected in the bins on element |
| 81 |
add/remove, recursing bins if needed. |
| 82 |
|
| 83 |
the toplevel bin with a scheduler selects the global clock and calls |
| 84 |
set_clock on itself. This happens in the NULL->READY state. |
| 85 |
|
| 86 |
bins dispatch the set_clock to to all of the reveivers. Bins with |
| 87 |
a scheduler and another use_clock do nothing. |
| 88 |
|
| 89 |
Bins with a scheduler also notify the scheduler of the clock. |
| 90 |
|
| 91 |
|
| 92 |
Clock usage |
| 93 |
----------- |
| 94 |
|
| 95 |
when an element wants to wait for a specific time, if calls |
| 96 |
gst_element_clock_wait (elements, clock, time). The call is dispatched |
| 97 |
to the scheduler of the element which can use the owner field |
| 98 |
of the clock to check if the elements are in the same scheduler. |
| 99 |
|
| 100 |
For elements waiting for a clock provided by an element in another |
| 101 |
scheduler there is no problem. |
| 102 |
|
| 103 |
When provider and receiver are in the same scheduler, a deadlock can |
| 104 |
occur since the scheduler will block on the wait without being able to |
| 105 |
schedule the provider again to update the clock. |
| 106 |
|
| 107 |
A solution would be to call the async notify of the clock and schedule |
| 108 |
some other element (the provider?). We probably need an event based |
| 109 |
scheduler for this. When the async event arrives we can reschedule |
| 110 |
the receiver and continue. |
| 111 |
|
| 112 |
The current scheduler will assert on this condition for now. |
| 113 |
|
| 114 |
|
| 115 |
Changing clocks |
| 116 |
--------------- |
| 117 |
|
| 118 |
The clock can only be changed when the bin is in the PAUSED state. |
| 119 |
|
| 120 |
|
| 121 |
State Changes |
| 122 |
------------- |
| 123 |
|
| 124 |
When the pipeline is PAUSED, the clock is stopped with |
| 125 |
gst_clock_enable (clock, FALSE). The clock should unblock all |
| 126 |
waiting elements ASAP and return GST_CLOCK_STOP in the wait. |
| 127 |
|
| 128 |
Elements waiting for a clock and receiving the STOP should |
| 129 |
process the last buffer ASAP and break out of their loop. |
| 130 |
|
| 131 |
When the pipeline is brought to the READY state, the clock is |
| 132 |
set to 0. |
| 133 |
|
| 134 |
When the clock is enabled again, it should start counting from where |
| 135 |
it was last disabled. |
| 136 |
|
| 137 |
NULL->READY : distribute clock, clock_reset, |
| 138 |
READY->PAUSED : clock_activate (FALSE) |
| 139 |
PAUSED->PLAYING : clock_activate (TRUE) |
| 140 |
PLAYING->PAUSED : clock_activate (FALSE); |
| 141 |
PAUSED->READY : clock_reset |
| 142 |
READY->NULL : delete clock; |
| 143 |
|
| 144 |
|
| 145 |
automatic Clock selection |
| 146 |
------------------------- |
| 147 |
|
| 148 |
Select a random clock from the Src elements. |
| 149 |
if no Src elements exist with a clock, select a random clock from the |
| 150 |
Sink elements. |
| 151 |
else use a default System Clock. |
| 152 |
|
| 153 |
Src elements with a clock are preferred because they usually provide |
| 154 |
live audio/video. |
| 155 |
|
| 156 |
Issues |
| 157 |
------ |
| 158 |
|
| 159 |
ossrc ! osssink can cause clock drift if osssink doesn't process the bytes |
| 160 |
at the same rate osssrc provides them (different hardware). Things will |
| 161 |
stutter and cracle if this is the case. QoS and a resampler could solve |
| 162 |
this. |
| 163 |
|
| 164 |
|
| 165 |
Use Cases |
| 166 |
--------- |
| 167 |
-- queue ! mpeg2dec ! videosink |
| 168 |
/ |
| 169 |
filesrc ! mpegdemux |
| 170 |
\ |
| 171 |
-- queue ! mad ! osssink |
| 172 |
|
| 173 |
|
| 174 |
videosink is a receiver |
| 175 |
osssink is a provider |
| 176 |
|
| 177 |
osssink is selected as a clock provider since it is a Sink. The global |
| 178 |
pipeline distributes the clock to videosink and osssink. |
| 179 |
osssink sees that it receives its own clock. |
| 180 |
|
| 181 |
osssink uses the OSS ioctls to determine the number of bytes processed |
| 182 |
by the hardware. using the audio rate it can figure out the exact time |
| 183 |
and updates its clock with a resolution that matches the resolution |
| 184 |
as closely as possible. |
| 185 |
|
| 186 |
videosink blocks on the clock. with each update of the clock videosink |
| 187 |
is unblocked if the current time >= wait time and shows the frame. |
| 188 |
|
| 189 |
when osssink is PAUSED, the clock will not be updated anymore. osssink |
| 190 |
instructs its clock to convert all requests to select() calls. |
| 191 |
When it is set to PLAYING again, it resumes normal operation. |
| 192 |
|
| 193 |
|
| 194 |
|
| 195 |
|
| 196 |
|
| 197 |
|
| 198 |
|
| 199 |
|
| 200 |
|