Debian Codegraph @

docs/usage/channels.rst

raw
1 .. currentmodule:: litestar.channels
2
3 Channels
4 ========
5
6 **Channels** are a group of related functionalities, built to facilitate the routing
7 of event streams, which for example can be used to broadcast messages to WebSocket
8 clients.
9
10 Channels provide:
11
12 1. Independent :term:`broker` backends, optionally handling inter-process communication
13 and data persistence on demand
14 2. "Channel" based :term:`subscription` management
15 3. Subscriber objects as an abstraction over an individualized :term:`event stream`,
16 providing background workers and managed subscriptions
17 4. Synchronous and asynchronous data publishing
18 5. Optional :term:`history` management on a per-:term:`channel` basis
19 6. :doc:`WebSocket </usage/websockets>` integration, generating WebSocket route
20 handlers for an application, to handle the :term:`subscription` and publishing of incoming
21 events to the connected client
22
23
24 Basic concepts
25 --------------
26
27 Utilizing Channels involves a few moving parts. To better familiarize with the concepts,
28 terminology, and the flow of data, the following glossary and flowcharts are provided
29
30 Glossary
31 ++++++++
32
33 .. dropdown:: Click to toggle the glossary
34
35 .. glossary::
36
37 event
38 A single piece of data published to, or received from a :term:`backend` bound
39 to the :term:`channel` it was originally published to
40
41 event stream
42 A stream of :term:`events <event>`, consisting of events from all the channels a
43 :term:`Subscriber` has previously subscribed to
44
45 subscriber
46 A :class:`Subscriber <.subscriber.Subscriber>`: An object wrapping an
47 :term:`event stream` and providing access to it through various methods
48
49 backend
50 A :class:`ChannelsBackend <.backends.base.ChannelsBackend>`. This object
51 manages communication between the plugin and the :term:`broker`, publishing
52 messages to and receiving messages from it. Each plugin instance is associated
53 with exactly one backend.
54
55 broker
56 Responsible for receiving and publishing messages to all connected
57 :term:`backends <backend>`; All backends sharing the same broker will have
58 access to the same messages, allowing for inter-process communication. This is
59 typically handled by a separate entity like `Redis <https://redis.io/>`_
60
61 plugin
62 The :class:`~.plugin.ChannelsPlugin`, a central instance
63 managing :term:`subscribers <subscriber>`, reading messages from the
64 :term:`backend`, putting them in the appropriate :term:`event stream`, and
65 publishing data to the backend
66
67 channel
68 A named group of subscribers, to which data can be published. Subscribers can
69 subscribe to multiple channels, and channels can have multiple subscribers
70
71 subscription
72 A connection between a :term:`subscriber` and a :term:`channel`, allowing the
73 subscriber to receive events from the channel
74
75 backpressure
76 A mechanism to prevent the backlog of a :term:`subscriber` from growing
77 indefinitely, by either dropping new messages or evicting old ones
78
79 history
80 A set of previously published :term:`events <event>`, stored by the :term:`backend`
81 and available to be pushed to a :term:`subscriber`
82
83 fanout
84 The process of sending a message to all subscribers of a channel
85
86 eviction
87 A :term:`backpressure` strategy, dropping the oldest message in the backlog when
88 a new one is added while the backlog is full
89
90 backoff
91 A :term:`backpressure` strategy, dropping newly incoming messages as long as the
92 backlog is full
93
94 Flowcharts
95 ++++++++++
96
97 .. dropdown:: Click to toggle flowcharts
98
99 .. mermaid::
100 :align: center
101 :caption: Publishing flow from the application to the :term:`broker`
102
103 flowchart LR
104 Backend(Backend) --> Broker[(Broker)]
105
106 Plugin{{Plugin}} --> Backend
107
108 Application[[Application]] --> Plugin
109
110 .. mermaid::
111 :align: center
112 :caption: Fanout flow of data from the :term:`broker` to the sockets, showing multiple plugin instances
113
114 flowchart TD
115 Broker[(Broker)]
116
117 Broker --> Backend_1(Backend)
118 Broker --> Backend_2(Backend)
119
120 Backend_1 --> Plugin_1{{Plugin}}
121 Backend_2 --> Plugin_2{{Plugin}}
122
123 Plugin_1 --> Subscriber_1[Subscriber]
124 Plugin_1 --> Subscriber_2[Subscriber]
125 Plugin_1 --> Subscriber_3[Subscriber]
126
127 Plugin_2 --> Subscriber_4[Subscriber]
128 Plugin_2 --> Subscriber_5[Subscriber]
129 Plugin_2 --> Subscriber_6[Subscriber]
130
131 The :class:`ChannelsPlugin`
132 ---------------------------
133
134 .. currentmodule:: litestar.channels.plugin
135
136 The :class:`ChannelsPlugin` acts as the central entity for managing channels and subscribers.
137 It is used to publish messages, control how data is stored, and manage :term:`subscribers <subscriber>`,
138 route handlers, and configuration.
139
140 .. tip:: The plugin makes itself available as a dependency under the :paramref:`~ChannelsPlugin.channels` key, which
141 means it is not necessary to import it and instead, it can be used from within route handlers or other callables
142 within the dependency tree directly
143
144 Configuring the :term:`channels <channel>`
145 ++++++++++++++++++++++++++++++++++++++++++
146
147 The :term:`channels <channel>` managed by the plugin can be either defined upfront, passing them to the
148 :paramref:`~ChannelsPlugin.channels` parameter, or created "on the fly"
149 (i.e., on the first :term:`subscription` to a channel) by setting
150 :paramref:`~ChannelsPlugin.arbitrary_channels_allowed` to ``True``.
151
152 .. code-block:: python
153 :caption: Passing channels explicitly
154
155 from litestar.channels import ChannelsPlugin
156
157 channels_plugin = ChannelsPlugin(..., channels=["foo", "bar"])
158
159 .. code-block:: python
160 :caption: Allowing arbitrary channels
161
162 from litestar.channels import ChannelsPlugin
163
164 channels_plugin = ChannelsPlugin(..., arbitrary_channels_allowed=True)
165
166 If :paramref:`~ChannelsPlugin.arbitrary_channels_allowed` is not ``True``, trying to publish or subscribe to a
167 :term:`channel` not passed to :paramref:`~ChannelsPlugin.channels` will raise a :exc:`ChannelsException`.
168
169 Publishing data
170 +++++++++++++++
171
172 One of the core aspects of the plugin is publishing data, which is done through its
173 :meth:`~ChannelsPlugin.publish` method:
174
175 .. code-block:: python
176 :caption: Publishing data to a channel with :meth:`~ChannelsPlugin.publish`
177
178 channels.publish({"message": "Hello"}, "general")
179
180 The above example will publish the data to the channel ``general``, subsequently putting
181 it into all subscriber's :term:`event stream` to be consumed.
182
183 This method is non-blocking, even though channels and the associated
184 :term:`backends <backend>` are fundamentally asynchronous.
185
186 Calling :meth:`~ChannelsPlugin.publish` effectively enqueues a message to be sent to the backend, from which
187 follows that there is no guarantee that an event will be available in the backend immediately after this call.
188
189 Alternatively, the asynchronous :meth:`~ChannelsPlugin.wait_published` method can be used, which skips the
190 internal message queue, publishing the data to the backend directly.
191
192 .. note::
193 While calling :meth:`~ChannelsPlugin.publish` does not guarantee the
194 message is sent to the backend immediately, it will be sent there *eventually*; On
195 shutdown, the plugin will wait for all queues to empty
196
197 Managing :term:`subscriptions <subscription>`
198 +++++++++++++++++++++++++++++++++++++++++++++
199
200 Another core functionality of the plugin is managing :term:`subscriptions <subscription>`, for which two
201 different approaches exist:
202
203 1. Manually through the :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.unsubscribe` methods
204 2. By using the :meth:`~ChannelsPlugin.start_subscription` context manager
205
206 Both :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.start_subscription` produce a :class:`Subscriber`,
207 which can be used to interact with the streams of events subscribed to.
208
209 The context manager should be preferred, since it ensures that channels are being unsubscribed.
210 Using the :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.unsubscribe` methods directly should only be
211 one when a :term:`context manager <asynchronous context manager>` cannot be used, e.g., when the :term:`subscription`
212 would span different contexts.
213
214 .. code-block:: python
215 :caption: Calling the :term:`subscription` methods manually
216
217 subscriber = await channels.subscribe(["foo", "bar"])
218 try:
219 ... # do some stuff here
220 finally:
221 await channels.unsubscribe(subscriber)
222
223 .. code-block:: python
224 :caption: Using the :term:`async context manager <asynchronous context manager>`
225
226 async with channels.start_subscription(["foo", "bar"]) as subscriber:
227 ... # do some stuff here
228
229 It is also possible to unsubscribe from individual :term:`channels <channel>`, which may be desirable if
230 :term:`subscriptions <subscription>` need to be managed dynamically.
231
232 .. code-block:: python
233 :caption: Unsubscribing from a channel manually
234
235 subscriber = await channels.subscribe(["foo", "bar"])
236 try:
237 ... # do some stuff here
238 finally:
239 await channels.unsubscribe(subscriber, ["foo"])
240
241
242 Or, using the context manager
243
244 .. code-block:: python
245 :caption: Using the :term:`async context manager <asynchronous context manager>` to unsubscribe from a
246 :term:`channel`
247
248 async with channels.start_subscription(["foo", "bar"]) as subscriber:
249 ... # do some stuff here
250 await channels.unsubscribe(subscriber, ["foo"])
251
252 Managing :term:`history`
253 ++++++++++++++++++++++++
254
255 Some backends support per-:term:`channel` :term:`history`, keeping a certain amount of :term:`events <event>` in storage.
256 This :term:`history` can then be pushed to a :term:`subscriber`.
257
258 The plugin's :meth:`put_subscriber_history <ChannelsPlugin.put_subscriber_history>` can
259 be used to fetch this :term:`history` and put it into a subscriber's :term:`event stream`.
260
261 .. literalinclude:: /examples/channels/put_history.py
262 :caption: Retrieving :term:`channel` :term:`history` for a :term:`subscriber` and putting it into the
263 :term:`stream <event stream>`
264
265 .. note::
266 The publication of the :term:`history` happens sequentially, one :term:`channel` and one
267 :term:`event` at a time. This is done to ensure the correct ordering of events and to avoid
268 filling up a :term:`subscriber`'s backlog, which would result in dropped :term:`history` entries. Should the
269 amount of entries exceed the maximum backlog size, the execution will wait until
270 previous events have been processed.
271
272 Read more: `Managing backpressure`_
273
274 The :class:`Subscriber`
275 -----------------------
276
277 .. py:currentmodule:: litestar.channels.subscriber
278
279 The :class:`Subscriber` manages an individual :term:`event stream`, provided to it by the plugin,
280 representing the sum of events from all :term:`channels <channel>` the subscriber has subscribed to.
281
282 It can be considered the endpoint of all :term:`events <event>`, while the backends act as the source,
283 and the plugin as a router, being responsible for supplying events gathered from the
284 backend into the appropriate subscriber's streams.
285
286 In addition to being an abstraction of an :term:`event stream`, the :class:`Subscriber`
287 provides two methods to handle this stream:
288
289 :meth:`iter_events <Subscriber.iter_events>`
290 An :term:`asynchronous generator`, producing one event from the stream at a time, waiting
291 until the next one becomes available
292
293 :meth:`run_in_background <Subscriber.run_in_background>`
294 A :term:`context manager <asynchronous context manager>`, wrapping an :class:`asyncio.Task`,
295 consuming events yielded by :meth:`iter_events <Subscriber.iter_events>`, invoking a provided :term:`callback`
296 for each of them. Upon exit, it will attempt a graceful shutdown of the running task, waiting
297 for all currently enqueued events in the stream to be processed.
298 If the context exits with an error, the task will be cancelled instead.
299
300 It is possible to force the task to stop immediately, by setting :paramref:`~Subscriber.run_in_background.join`
301 to ``False`` in :meth:`run_in_background <Subscriber.run_in_background>`, which will lead to the cancellation of
302 the task. By default this only happens when the context is left with an exception.
303
304 .. important::
305 The :term:`events <event>` in the :term:`event streams <event stream>` are always
306 bytes; When calling :meth:`ChannelsPlugin.publish`, data will be serialized before
307 being sent to the backend.
308
309 Consuming the :term:`event stream`
310 ++++++++++++++++++++++++++++++++++
311
312 There are two general methods of consuming the :term:`event stream`:
313
314 1. By iterating over it directly, using :meth:`iter_events <Subscriber.iter_events>`
315 2. By using the :meth:`run_in_background <Subscriber.run_in_background>` context manager,
316 which starts a background task, iterating over the stream, invoking a provided
317 callback for every :term:`event` received
318
319 Iterating over the :term:`stream <event stream>` directly is mostly useful if processing the events is the only
320 concern, since :meth:`iter_events <Subscriber.iter_events>` is effectively an infinite
321 loop. For all other applications, using the context manager is preferable, since it
322 allows to easily run other code concurrently.
323
324 .. literalinclude:: /examples/channels/iter_stream.py
325 :caption: Iterating over the :term:`event stream` to send data to a WebSocket
326
327 In the above example, the stream is used to send data to a
328 :class:`WebSocket <litestar.connection.WebSocket>`.
329
330 The same can be achieve by passing :meth:`Websocket.send_text() <litestar.connection.WebSocket.send_text>` as the
331 callback to :meth:`~Subscriber.run_in_background`. This will cause the WebSocket's method to be invoked every time
332 a new :term:`event` becomes available in the :term:`stream <event stream>`, but gives control back to the application,
333 providing an opportunity to perform other tasks, such as receiving incoming data from the socket.
334
335 .. literalinclude:: /examples/channels/run_in_background.py
336 :caption: Using :meth:`~Subscriber.run_in_background` to process events concurrently
337
338 .. important:: Iterating over :meth:`~Subscriber.iter_events` should be approached
339 with caution when being used together with WebSockets.
340
341 Since :exc:`WebSocketDisconnect` is only raised after the corresponding ASGI event
342 has been *received*, it can result in an indefinitely suspended coroutine. This can
343 happen if for example the client disconnects, but no further events are received.
344 The generator will then wait for new events, but since it will never receive any,
345 no ``send`` call on the WebSocket will be made, which in turn means no exception
346 will be raised to break the loop.
347
348 Managing :term:`backpressure`
349 -----------------------------
350
351 Each subscriber manages its own backlog: A queue of unprocessed :term:`events <event>`.
352 By default, this backlog is unlimited in size, allowing it to grow indefinitely. For
353 most applications, this should be no issue, but when the recipient consistently can not
354 process messages faster than they come in, an application might opt to handle this case.
355
356 The channels plugin provides two different strategies for managing this :term:`backpressure`:
357
358 1. A :term:`backoff` strategy, dropping newly incoming messages as long as the backlog is full
359 2. An :term:`eviction` strategy, dropping the oldest message in the backlog when a new one is
360 added while the backlog is full
361
362 .. code-block:: python
363 :caption: Backoff strategy
364
365 from litestar.channels import ChannelsPlugin
366 from litestar.channels.memory import MemoryChannelsBackend
367
368 channels = ChannelsPlugin(
369 backend=MemoryChannelsBackend(),
370 max_backlog=1000,
371 backlog_strategy="backoff",
372 )
373
374 .. code-block:: python
375 :caption: :term:`Eviction <eviction>` strategy
376
377 from litestar.channels import ChannelsPlugin
378 from litestar.channels.memory import MemoryChannelsBackend
379
380 channels = ChannelsPlugin(
381 backend=MemoryChannelsBackend(),
382 max_backlog=1000,
383 backlog_strategy="dropleft",
384 )
385
386 Backends
387 --------
388
389 The storing and :term:`fanout` of messages is handled by a
390 :class:`ChannelsBackend <litestar.channels.backends.base.ChannelsBackend>`.
391
392 The following backends are currently implemented:
393
394 :class:`MemoryChannelsBacked <.memory.MemoryChannelsBackend>`
395 A basic in-memory backend, mostly useful for testing and local development, but
396 still fully capable. Since it stores all data in-process, it can achieve the highest
397 performance of all the backends, but at the same time is not suitable for
398 applications running on multiple processes.
399
400 :class:`RedisChannelsPubSubBackend <.redis.RedisChannelsPubSubBackend>`
401 A Redis based backend, using `Pub/Sub <https://redis.io/docs/manual/pubsub/>`_ to
402 delivery messages. This Redis backend has a low latency and overhead and is
403 generally recommended if :term:`history` is not needed
404
405 :class:`RedisChannelsStreamBackend <.redis.RedisChannelsStreamBackend>`
406 A redis based backend, using `streams <https://redis.io/docs/data-types/streams/>`_
407 to deliver messages. It has a slightly higher latency when publishing than the
408 Pub/Sub backend, but achieves the same throughput in message :term:`fanout`. Recommended
409 when :term:`history` is needed
410
411 :class:`AsyncPgChannelsBackend <.asyncpg.AsyncPgChannelsBackend>`
412 A postgres backend using the `asyncpg <https://magicstack.github.io/asyncpg/current/>`_ driver
413
414 :class:`PsycoPgChannelsBackend <.psycopg.PsycoPgChannelsBackend>`
415 A postgres backend using the `psycopg3 <https://www.psycopg.org/psycopg3/docs/>`_ async driver
416
417 Integrating with websocket handlers
418 -----------------------------------
419
420 Generating route handlers
421 +++++++++++++++++++++++++
422
423 A common pattern is to create a route handler per :term:`channel`, sending data to the connected
424 client from that channel. This can be fully automated, using the plugin to create these route handlers.
425
426 .. literalinclude:: /examples/channels/create_route_handlers.py
427 :language: python
428 :caption: Setting ``create_ws_route_handlers=True`` will create route handlers for all ``channels``
429
430 The generated route handlers can optionally be configured to send the :term:`channel`'s :term:`history`
431 after a client has connected:
432
433 .. literalinclude:: /examples/channels/create_route_handlers_send_history.py
434 :caption: Sending the first 10 :term:`history` entries after a client connects
435
436 .. tip:: When using the :paramref:`~litestar.channels.plugin.ChannelsPlugin.arbitrary_channels_allowed` parameter
437 on the :class:`ChannelsPlugin`, a single route handler will be generated instead, using a
438 :ref:`path parameter <usage/routing/parameters:path parameters>` to specify the channel name