| 1 |
.. currentmodule:: litestar.channels |
| 2 |
|
| 3 |
Channels |
| 4 |
======== |
| 5 |
|
| 6 |
**Channels** are a group of related functionalities, built to facilitate the routing |
| 7 |
of event streams, which for example can be used to broadcast messages to WebSocket |
| 8 |
clients. |
| 9 |
|
| 10 |
Channels provide: |
| 11 |
|
| 12 |
1. Independent :term:`broker` backends, optionally handling inter-process communication |
| 13 |
and data persistence on demand |
| 14 |
2. "Channel" based :term:`subscription` management |
| 15 |
3. Subscriber objects as an abstraction over an individualized :term:`event stream`, |
| 16 |
providing background workers and managed subscriptions |
| 17 |
4. Synchronous and asynchronous data publishing |
| 18 |
5. Optional :term:`history` management on a per-:term:`channel` basis |
| 19 |
6. :doc:`WebSocket </usage/websockets>` integration, generating WebSocket route |
| 20 |
handlers for an application, to handle the :term:`subscription` and publishing of incoming |
| 21 |
events to the connected client |
| 22 |
|
| 23 |
|
| 24 |
Basic concepts |
| 25 |
-------------- |
| 26 |
|
| 27 |
Utilizing Channels involves a few moving parts. To better familiarize with the concepts, |
| 28 |
terminology, and the flow of data, the following glossary and flowcharts are provided |
| 29 |
|
| 30 |
Glossary |
| 31 |
++++++++ |
| 32 |
|
| 33 |
.. dropdown:: Click to toggle the glossary |
| 34 |
|
| 35 |
.. glossary:: |
| 36 |
|
| 37 |
event |
| 38 |
A single piece of data published to, or received from a :term:`backend` bound |
| 39 |
to the :term:`channel` it was originally published to |
| 40 |
|
| 41 |
event stream |
| 42 |
A stream of :term:`events <event>`, consisting of events from all the channels a |
| 43 |
:term:`Subscriber` has previously subscribed to |
| 44 |
|
| 45 |
subscriber |
| 46 |
A :class:`Subscriber <.subscriber.Subscriber>`: An object wrapping an |
| 47 |
:term:`event stream` and providing access to it through various methods |
| 48 |
|
| 49 |
backend |
| 50 |
A :class:`ChannelsBackend <.backends.base.ChannelsBackend>`. This object |
| 51 |
manages communication between the plugin and the :term:`broker`, publishing |
| 52 |
messages to and receiving messages from it. Each plugin instance is associated |
| 53 |
with exactly one backend. |
| 54 |
|
| 55 |
broker |
| 56 |
Responsible for receiving and publishing messages to all connected |
| 57 |
:term:`backends <backend>`; All backends sharing the same broker will have |
| 58 |
access to the same messages, allowing for inter-process communication. This is |
| 59 |
typically handled by a separate entity like `Redis <https://redis.io/>`_ |
| 60 |
|
| 61 |
plugin |
| 62 |
The :class:`~.plugin.ChannelsPlugin`, a central instance |
| 63 |
managing :term:`subscribers <subscriber>`, reading messages from the |
| 64 |
:term:`backend`, putting them in the appropriate :term:`event stream`, and |
| 65 |
publishing data to the backend |
| 66 |
|
| 67 |
channel |
| 68 |
A named group of subscribers, to which data can be published. Subscribers can |
| 69 |
subscribe to multiple channels, and channels can have multiple subscribers |
| 70 |
|
| 71 |
subscription |
| 72 |
A connection between a :term:`subscriber` and a :term:`channel`, allowing the |
| 73 |
subscriber to receive events from the channel |
| 74 |
|
| 75 |
backpressure |
| 76 |
A mechanism to prevent the backlog of a :term:`subscriber` from growing |
| 77 |
indefinitely, by either dropping new messages or evicting old ones |
| 78 |
|
| 79 |
history |
| 80 |
A set of previously published :term:`events <event>`, stored by the :term:`backend` |
| 81 |
and available to be pushed to a :term:`subscriber` |
| 82 |
|
| 83 |
fanout |
| 84 |
The process of sending a message to all subscribers of a channel |
| 85 |
|
| 86 |
eviction |
| 87 |
A :term:`backpressure` strategy, dropping the oldest message in the backlog when |
| 88 |
a new one is added while the backlog is full |
| 89 |
|
| 90 |
backoff |
| 91 |
A :term:`backpressure` strategy, dropping newly incoming messages as long as the |
| 92 |
backlog is full |
| 93 |
|
| 94 |
Flowcharts |
| 95 |
++++++++++ |
| 96 |
|
| 97 |
.. dropdown:: Click to toggle flowcharts |
| 98 |
|
| 99 |
.. mermaid:: |
| 100 |
:align: center |
| 101 |
:caption: Publishing flow from the application to the :term:`broker` |
| 102 |
|
| 103 |
flowchart LR |
| 104 |
Backend(Backend) --> Broker[(Broker)] |
| 105 |
|
| 106 |
Plugin{{Plugin}} --> Backend |
| 107 |
|
| 108 |
Application[[Application]] --> Plugin |
| 109 |
|
| 110 |
.. mermaid:: |
| 111 |
:align: center |
| 112 |
:caption: Fanout flow of data from the :term:`broker` to the sockets, showing multiple plugin instances |
| 113 |
|
| 114 |
flowchart TD |
| 115 |
Broker[(Broker)] |
| 116 |
|
| 117 |
Broker --> Backend_1(Backend) |
| 118 |
Broker --> Backend_2(Backend) |
| 119 |
|
| 120 |
Backend_1 --> Plugin_1{{Plugin}} |
| 121 |
Backend_2 --> Plugin_2{{Plugin}} |
| 122 |
|
| 123 |
Plugin_1 --> Subscriber_1[Subscriber] |
| 124 |
Plugin_1 --> Subscriber_2[Subscriber] |
| 125 |
Plugin_1 --> Subscriber_3[Subscriber] |
| 126 |
|
| 127 |
Plugin_2 --> Subscriber_4[Subscriber] |
| 128 |
Plugin_2 --> Subscriber_5[Subscriber] |
| 129 |
Plugin_2 --> Subscriber_6[Subscriber] |
| 130 |
|
| 131 |
The :class:`ChannelsPlugin` |
| 132 |
--------------------------- |
| 133 |
|
| 134 |
.. currentmodule:: litestar.channels.plugin |
| 135 |
|
| 136 |
The :class:`ChannelsPlugin` acts as the central entity for managing channels and subscribers. |
| 137 |
It is used to publish messages, control how data is stored, and manage :term:`subscribers <subscriber>`, |
| 138 |
route handlers, and configuration. |
| 139 |
|
| 140 |
.. tip:: The plugin makes itself available as a dependency under the :paramref:`~ChannelsPlugin.channels` key, which |
| 141 |
means it is not necessary to import it and instead, it can be used from within route handlers or other callables |
| 142 |
within the dependency tree directly |
| 143 |
|
| 144 |
Configuring the :term:`channels <channel>` |
| 145 |
++++++++++++++++++++++++++++++++++++++++++ |
| 146 |
|
| 147 |
The :term:`channels <channel>` managed by the plugin can be either defined upfront, passing them to the |
| 148 |
:paramref:`~ChannelsPlugin.channels` parameter, or created "on the fly" |
| 149 |
(i.e., on the first :term:`subscription` to a channel) by setting |
| 150 |
:paramref:`~ChannelsPlugin.arbitrary_channels_allowed` to ``True``. |
| 151 |
|
| 152 |
.. code-block:: python |
| 153 |
:caption: Passing channels explicitly |
| 154 |
|
| 155 |
from litestar.channels import ChannelsPlugin |
| 156 |
|
| 157 |
channels_plugin = ChannelsPlugin(..., channels=["foo", "bar"]) |
| 158 |
|
| 159 |
.. code-block:: python |
| 160 |
:caption: Allowing arbitrary channels |
| 161 |
|
| 162 |
from litestar.channels import ChannelsPlugin |
| 163 |
|
| 164 |
channels_plugin = ChannelsPlugin(..., arbitrary_channels_allowed=True) |
| 165 |
|
| 166 |
If :paramref:`~ChannelsPlugin.arbitrary_channels_allowed` is not ``True``, trying to publish or subscribe to a |
| 167 |
:term:`channel` not passed to :paramref:`~ChannelsPlugin.channels` will raise a :exc:`ChannelsException`. |
| 168 |
|
| 169 |
Publishing data |
| 170 |
+++++++++++++++ |
| 171 |
|
| 172 |
One of the core aspects of the plugin is publishing data, which is done through its |
| 173 |
:meth:`~ChannelsPlugin.publish` method: |
| 174 |
|
| 175 |
.. code-block:: python |
| 176 |
:caption: Publishing data to a channel with :meth:`~ChannelsPlugin.publish` |
| 177 |
|
| 178 |
channels.publish({"message": "Hello"}, "general") |
| 179 |
|
| 180 |
The above example will publish the data to the channel ``general``, subsequently putting |
| 181 |
it into all subscriber's :term:`event stream` to be consumed. |
| 182 |
|
| 183 |
This method is non-blocking, even though channels and the associated |
| 184 |
:term:`backends <backend>` are fundamentally asynchronous. |
| 185 |
|
| 186 |
Calling :meth:`~ChannelsPlugin.publish` effectively enqueues a message to be sent to the backend, from which |
| 187 |
follows that there is no guarantee that an event will be available in the backend immediately after this call. |
| 188 |
|
| 189 |
Alternatively, the asynchronous :meth:`~ChannelsPlugin.wait_published` method can be used, which skips the |
| 190 |
internal message queue, publishing the data to the backend directly. |
| 191 |
|
| 192 |
.. note:: |
| 193 |
While calling :meth:`~ChannelsPlugin.publish` does not guarantee the |
| 194 |
message is sent to the backend immediately, it will be sent there *eventually*; On |
| 195 |
shutdown, the plugin will wait for all queues to empty |
| 196 |
|
| 197 |
Managing :term:`subscriptions <subscription>` |
| 198 |
+++++++++++++++++++++++++++++++++++++++++++++ |
| 199 |
|
| 200 |
Another core functionality of the plugin is managing :term:`subscriptions <subscription>`, for which two |
| 201 |
different approaches exist: |
| 202 |
|
| 203 |
1. Manually through the :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.unsubscribe` methods |
| 204 |
2. By using the :meth:`~ChannelsPlugin.start_subscription` context manager |
| 205 |
|
| 206 |
Both :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.start_subscription` produce a :class:`Subscriber`, |
| 207 |
which can be used to interact with the streams of events subscribed to. |
| 208 |
|
| 209 |
The context manager should be preferred, since it ensures that channels are being unsubscribed. |
| 210 |
Using the :meth:`~ChannelsPlugin.subscribe` and :meth:`~ChannelsPlugin.unsubscribe` methods directly should only be |
| 211 |
one when a :term:`context manager <asynchronous context manager>` cannot be used, e.g., when the :term:`subscription` |
| 212 |
would span different contexts. |
| 213 |
|
| 214 |
.. code-block:: python |
| 215 |
:caption: Calling the :term:`subscription` methods manually |
| 216 |
|
| 217 |
subscriber = await channels.subscribe(["foo", "bar"]) |
| 218 |
try: |
| 219 |
... # do some stuff here |
| 220 |
finally: |
| 221 |
await channels.unsubscribe(subscriber) |
| 222 |
|
| 223 |
.. code-block:: python |
| 224 |
:caption: Using the :term:`async context manager <asynchronous context manager>` |
| 225 |
|
| 226 |
async with channels.start_subscription(["foo", "bar"]) as subscriber: |
| 227 |
... # do some stuff here |
| 228 |
|
| 229 |
It is also possible to unsubscribe from individual :term:`channels <channel>`, which may be desirable if |
| 230 |
:term:`subscriptions <subscription>` need to be managed dynamically. |
| 231 |
|
| 232 |
.. code-block:: python |
| 233 |
:caption: Unsubscribing from a channel manually |
| 234 |
|
| 235 |
subscriber = await channels.subscribe(["foo", "bar"]) |
| 236 |
try: |
| 237 |
... # do some stuff here |
| 238 |
finally: |
| 239 |
await channels.unsubscribe(subscriber, ["foo"]) |
| 240 |
|
| 241 |
|
| 242 |
Or, using the context manager |
| 243 |
|
| 244 |
.. code-block:: python |
| 245 |
:caption: Using the :term:`async context manager <asynchronous context manager>` to unsubscribe from a |
| 246 |
:term:`channel` |
| 247 |
|
| 248 |
async with channels.start_subscription(["foo", "bar"]) as subscriber: |
| 249 |
... # do some stuff here |
| 250 |
await channels.unsubscribe(subscriber, ["foo"]) |
| 251 |
|
| 252 |
Managing :term:`history` |
| 253 |
++++++++++++++++++++++++ |
| 254 |
|
| 255 |
Some backends support per-:term:`channel` :term:`history`, keeping a certain amount of :term:`events <event>` in storage. |
| 256 |
This :term:`history` can then be pushed to a :term:`subscriber`. |
| 257 |
|
| 258 |
The plugin's :meth:`put_subscriber_history <ChannelsPlugin.put_subscriber_history>` can |
| 259 |
be used to fetch this :term:`history` and put it into a subscriber's :term:`event stream`. |
| 260 |
|
| 261 |
.. literalinclude:: /examples/channels/put_history.py |
| 262 |
:caption: Retrieving :term:`channel` :term:`history` for a :term:`subscriber` and putting it into the |
| 263 |
:term:`stream <event stream>` |
| 264 |
|
| 265 |
.. note:: |
| 266 |
The publication of the :term:`history` happens sequentially, one :term:`channel` and one |
| 267 |
:term:`event` at a time. This is done to ensure the correct ordering of events and to avoid |
| 268 |
filling up a :term:`subscriber`'s backlog, which would result in dropped :term:`history` entries. Should the |
| 269 |
amount of entries exceed the maximum backlog size, the execution will wait until |
| 270 |
previous events have been processed. |
| 271 |
|
| 272 |
Read more: `Managing backpressure`_ |
| 273 |
|
| 274 |
The :class:`Subscriber` |
| 275 |
----------------------- |
| 276 |
|
| 277 |
.. py:currentmodule:: litestar.channels.subscriber |
| 278 |
|
| 279 |
The :class:`Subscriber` manages an individual :term:`event stream`, provided to it by the plugin, |
| 280 |
representing the sum of events from all :term:`channels <channel>` the subscriber has subscribed to. |
| 281 |
|
| 282 |
It can be considered the endpoint of all :term:`events <event>`, while the backends act as the source, |
| 283 |
and the plugin as a router, being responsible for supplying events gathered from the |
| 284 |
backend into the appropriate subscriber's streams. |
| 285 |
|
| 286 |
In addition to being an abstraction of an :term:`event stream`, the :class:`Subscriber` |
| 287 |
provides two methods to handle this stream: |
| 288 |
|
| 289 |
:meth:`iter_events <Subscriber.iter_events>` |
| 290 |
An :term:`asynchronous generator`, producing one event from the stream at a time, waiting |
| 291 |
until the next one becomes available |
| 292 |
|
| 293 |
:meth:`run_in_background <Subscriber.run_in_background>` |
| 294 |
A :term:`context manager <asynchronous context manager>`, wrapping an :class:`asyncio.Task`, |
| 295 |
consuming events yielded by :meth:`iter_events <Subscriber.iter_events>`, invoking a provided :term:`callback` |
| 296 |
for each of them. Upon exit, it will attempt a graceful shutdown of the running task, waiting |
| 297 |
for all currently enqueued events in the stream to be processed. |
| 298 |
If the context exits with an error, the task will be cancelled instead. |
| 299 |
|
| 300 |
It is possible to force the task to stop immediately, by setting :paramref:`~Subscriber.run_in_background.join` |
| 301 |
to ``False`` in :meth:`run_in_background <Subscriber.run_in_background>`, which will lead to the cancellation of |
| 302 |
the task. By default this only happens when the context is left with an exception. |
| 303 |
|
| 304 |
.. important:: |
| 305 |
The :term:`events <event>` in the :term:`event streams <event stream>` are always |
| 306 |
bytes; When calling :meth:`ChannelsPlugin.publish`, data will be serialized before |
| 307 |
being sent to the backend. |
| 308 |
|
| 309 |
Consuming the :term:`event stream` |
| 310 |
++++++++++++++++++++++++++++++++++ |
| 311 |
|
| 312 |
There are two general methods of consuming the :term:`event stream`: |
| 313 |
|
| 314 |
1. By iterating over it directly, using :meth:`iter_events <Subscriber.iter_events>` |
| 315 |
2. By using the :meth:`run_in_background <Subscriber.run_in_background>` context manager, |
| 316 |
which starts a background task, iterating over the stream, invoking a provided |
| 317 |
callback for every :term:`event` received |
| 318 |
|
| 319 |
Iterating over the :term:`stream <event stream>` directly is mostly useful if processing the events is the only |
| 320 |
concern, since :meth:`iter_events <Subscriber.iter_events>` is effectively an infinite |
| 321 |
loop. For all other applications, using the context manager is preferable, since it |
| 322 |
allows to easily run other code concurrently. |
| 323 |
|
| 324 |
.. literalinclude:: /examples/channels/iter_stream.py |
| 325 |
:caption: Iterating over the :term:`event stream` to send data to a WebSocket |
| 326 |
|
| 327 |
In the above example, the stream is used to send data to a |
| 328 |
:class:`WebSocket <litestar.connection.WebSocket>`. |
| 329 |
|
| 330 |
The same can be achieve by passing :meth:`Websocket.send_text() <litestar.connection.WebSocket.send_text>` as the |
| 331 |
callback to :meth:`~Subscriber.run_in_background`. This will cause the WebSocket's method to be invoked every time |
| 332 |
a new :term:`event` becomes available in the :term:`stream <event stream>`, but gives control back to the application, |
| 333 |
providing an opportunity to perform other tasks, such as receiving incoming data from the socket. |
| 334 |
|
| 335 |
.. literalinclude:: /examples/channels/run_in_background.py |
| 336 |
:caption: Using :meth:`~Subscriber.run_in_background` to process events concurrently |
| 337 |
|
| 338 |
.. important:: Iterating over :meth:`~Subscriber.iter_events` should be approached |
| 339 |
with caution when being used together with WebSockets. |
| 340 |
|
| 341 |
Since :exc:`WebSocketDisconnect` is only raised after the corresponding ASGI event |
| 342 |
has been *received*, it can result in an indefinitely suspended coroutine. This can |
| 343 |
happen if for example the client disconnects, but no further events are received. |
| 344 |
The generator will then wait for new events, but since it will never receive any, |
| 345 |
no ``send`` call on the WebSocket will be made, which in turn means no exception |
| 346 |
will be raised to break the loop. |
| 347 |
|
| 348 |
Managing :term:`backpressure` |
| 349 |
----------------------------- |
| 350 |
|
| 351 |
Each subscriber manages its own backlog: A queue of unprocessed :term:`events <event>`. |
| 352 |
By default, this backlog is unlimited in size, allowing it to grow indefinitely. For |
| 353 |
most applications, this should be no issue, but when the recipient consistently can not |
| 354 |
process messages faster than they come in, an application might opt to handle this case. |
| 355 |
|
| 356 |
The channels plugin provides two different strategies for managing this :term:`backpressure`: |
| 357 |
|
| 358 |
1. A :term:`backoff` strategy, dropping newly incoming messages as long as the backlog is full |
| 359 |
2. An :term:`eviction` strategy, dropping the oldest message in the backlog when a new one is |
| 360 |
added while the backlog is full |
| 361 |
|
| 362 |
.. code-block:: python |
| 363 |
:caption: Backoff strategy |
| 364 |
|
| 365 |
from litestar.channels import ChannelsPlugin |
| 366 |
from litestar.channels.memory import MemoryChannelsBackend |
| 367 |
|
| 368 |
channels = ChannelsPlugin( |
| 369 |
backend=MemoryChannelsBackend(), |
| 370 |
max_backlog=1000, |
| 371 |
backlog_strategy="backoff", |
| 372 |
) |
| 373 |
|
| 374 |
.. code-block:: python |
| 375 |
:caption: :term:`Eviction <eviction>` strategy |
| 376 |
|
| 377 |
from litestar.channels import ChannelsPlugin |
| 378 |
from litestar.channels.memory import MemoryChannelsBackend |
| 379 |
|
| 380 |
channels = ChannelsPlugin( |
| 381 |
backend=MemoryChannelsBackend(), |
| 382 |
max_backlog=1000, |
| 383 |
backlog_strategy="dropleft", |
| 384 |
) |
| 385 |
|
| 386 |
Backends |
| 387 |
-------- |
| 388 |
|
| 389 |
The storing and :term:`fanout` of messages is handled by a |
| 390 |
:class:`ChannelsBackend <litestar.channels.backends.base.ChannelsBackend>`. |
| 391 |
|
| 392 |
The following backends are currently implemented: |
| 393 |
|
| 394 |
:class:`MemoryChannelsBacked <.memory.MemoryChannelsBackend>` |
| 395 |
A basic in-memory backend, mostly useful for testing and local development, but |
| 396 |
still fully capable. Since it stores all data in-process, it can achieve the highest |
| 397 |
performance of all the backends, but at the same time is not suitable for |
| 398 |
applications running on multiple processes. |
| 399 |
|
| 400 |
:class:`RedisChannelsPubSubBackend <.redis.RedisChannelsPubSubBackend>` |
| 401 |
A Redis based backend, using `Pub/Sub <https://redis.io/docs/manual/pubsub/>`_ to |
| 402 |
delivery messages. This Redis backend has a low latency and overhead and is |
| 403 |
generally recommended if :term:`history` is not needed |
| 404 |
|
| 405 |
:class:`RedisChannelsStreamBackend <.redis.RedisChannelsStreamBackend>` |
| 406 |
A redis based backend, using `streams <https://redis.io/docs/data-types/streams/>`_ |
| 407 |
to deliver messages. It has a slightly higher latency when publishing than the |
| 408 |
Pub/Sub backend, but achieves the same throughput in message :term:`fanout`. Recommended |
| 409 |
when :term:`history` is needed |
| 410 |
|
| 411 |
:class:`AsyncPgChannelsBackend <.asyncpg.AsyncPgChannelsBackend>` |
| 412 |
A postgres backend using the `asyncpg <https://magicstack.github.io/asyncpg/current/>`_ driver |
| 413 |
|
| 414 |
:class:`PsycoPgChannelsBackend <.psycopg.PsycoPgChannelsBackend>` |
| 415 |
A postgres backend using the `psycopg3 <https://www.psycopg.org/psycopg3/docs/>`_ async driver |
| 416 |
|
| 417 |
Integrating with websocket handlers |
| 418 |
----------------------------------- |
| 419 |
|
| 420 |
Generating route handlers |
| 421 |
+++++++++++++++++++++++++ |
| 422 |
|
| 423 |
A common pattern is to create a route handler per :term:`channel`, sending data to the connected |
| 424 |
client from that channel. This can be fully automated, using the plugin to create these route handlers. |
| 425 |
|
| 426 |
.. literalinclude:: /examples/channels/create_route_handlers.py |
| 427 |
:language: python |
| 428 |
:caption: Setting ``create_ws_route_handlers=True`` will create route handlers for all ``channels`` |
| 429 |
|
| 430 |
The generated route handlers can optionally be configured to send the :term:`channel`'s :term:`history` |
| 431 |
after a client has connected: |
| 432 |
|
| 433 |
.. literalinclude:: /examples/channels/create_route_handlers_send_history.py |
| 434 |
:caption: Sending the first 10 :term:`history` entries after a client connects |
| 435 |
|
| 436 |
.. tip:: When using the :paramref:`~litestar.channels.plugin.ChannelsPlugin.arbitrary_channels_allowed` parameter |
| 437 |
on the :class:`ChannelsPlugin`, a single route handler will be generated instead, using a |
| 438 |
:ref:`path parameter <usage/routing/parameters:path parameters>` to specify the channel name |