| 1 |
.. _migration-guide: |
| 2 |
|
| 3 |
Migrating off of Eventlet |
| 4 |
========================= |
| 5 |
|
| 6 |
There are two main use cases for Eventlet: |
| 7 |
|
| 8 |
1. As a required networking framework, much like one would use ``asyncio``, |
| 9 |
``trio``, or older frameworks like ``Twisted`` and ``tornado``. |
| 10 |
|
| 11 |
2. As an optional, pluggable backend that allows swapping out blocking APIs |
| 12 |
for an event loop, transparently, without changing any code. |
| 13 |
This is how Celery and Gunicorn use eventlet. |
| 14 |
|
| 15 |
Pretending to look like a blocking API while actually using an event loop |
| 16 |
underneath requires exact emulation of an ever-changing and ever-increasing |
| 17 |
API footprint, which is fundamentally unsustainable for a volunteer-driven |
| 18 |
open source project. |
| 19 |
This is why Eventlet is discouraging new users. |
| 20 |
|
| 21 |
**Most of this document will focus on the first use case: Eventlet as the sole |
| 22 |
networking framework.** |
| 23 |
For this use case, we recommend migrating to Python's ``asyncio``, and we are |
| 24 |
providing infrastructure that will make this much easier, and allow for |
| 25 |
*gradual* migration. |
| 26 |
|
| 27 |
For the second use case, we believe this is a fundamentally unsustainable |
| 28 |
approach and encourage the upstream frameworks to come up with different |
| 29 |
solutions. |
| 30 |
|
| 31 |
Step 1. Switch to the ``asyncio`` Hub |
| 32 |
------------------------------------- |
| 33 |
|
| 34 |
Eventlet has different pluggable networking event loops. |
| 35 |
By switching the event loop to use ``asyncio``, you enable running ``asyncio`` |
| 36 |
and Eventlet code in the same thread in the same process. |
| 37 |
|
| 38 |
To do so, set the ``EVENTLET_HUB`` environment variable to ``asyncio`` before |
| 39 |
starting your Eventlet program. |
| 40 |
For example, if you start your program with a shell script, you can do |
| 41 |
``export EVENTLET_HUB=asyncio``. |
| 42 |
|
| 43 |
Alternatively, you can explicitly specify the ``asyncio`` hub at startup, |
| 44 |
before monkey patching or any other setup work:: |
| 45 |
|
| 46 |
import eventlet.hubs |
| 47 |
eventlet.hubs.use_hub("eventlet.hubs.asyncio") |
| 48 |
|
| 49 |
Step 2. Migrate code to ``asyncio`` |
| 50 |
----------------------------------- |
| 51 |
|
| 52 |
Now that you're running Eventlet on top of ``asyncio``, you can use some new |
| 53 |
APIs to call from Eventlet code into ``asyncio``, and vice-versa. |
| 54 |
|
| 55 |
To call ``asyncio`` code from Eventlet code, you can wrap a coroutine (or |
| 56 |
anything you can ``await``) into an Eventlet ``GreenThread``. |
| 57 |
For example, if you want to make a HTTP request from Eventlet, you can use |
| 58 |
the ``asyncio``-based ``aiohttp`` library:: |
| 59 |
|
| 60 |
import aiohttp |
| 61 |
from eventlet.asyncio import spawn_for_awaitable |
| 62 |
|
| 63 |
async def request(): |
| 64 |
async with aiohttp.ClientSession() as session: |
| 65 |
url = "https://example.com" |
| 66 |
async with session.get(url) as response: |
| 67 |
html = await response.text() |
| 68 |
return html |
| 69 |
|
| 70 |
|
| 71 |
# This makes a coroutine; typically you'd ``await`` it: |
| 72 |
coro = request() |
| 73 |
|
| 74 |
# You can wrap this coroutine with an Eventlet GreenThread, similar to |
| 75 |
# ``evenlet.spawn()``: |
| 76 |
gthread = spawn_for_awaitable(request()) |
| 77 |
|
| 78 |
# And then get its result, the body of https://example.com: |
| 79 |
result = gthread.wait() |
| 80 |
|
| 81 |
In the other direction, any ``eventlet.greenthread.GreenThread`` can be |
| 82 |
``await``-ed in ``async`` functions. |
| 83 |
In other words ``async`` functions can call into Eventlet code:: |
| 84 |
|
| 85 |
def blocking_eventlet_api(): |
| 86 |
eventlet.sleep(1) |
| 87 |
# do some other pseudo-blocking work |
| 88 |
# ... |
| 89 |
return 12 |
| 90 |
|
| 91 |
async def my_async_func(): |
| 92 |
gthread = eventlet.spawn(blocking_eventlet_api) |
| 93 |
# In normal Eventlet code we'd call gthread.wait(), but since this is an |
| 94 |
# async function we'll want to await instead: |
| 95 |
result = await gthread |
| 96 |
# result is now 12 |
| 97 |
# ... |
| 98 |
|
| 99 |
Cancellation of ``asyncio.Future`` and killing of ``eventlet.GreenThread`` |
| 100 |
should propagate between the two. |
| 101 |
|
| 102 |
Using these two APIs, with more to come, you can gradually migrate portions of |
| 103 |
your application or library to ``asyncio``. |
| 104 |
Calls to blocking APIs like ``urlopen()`` or ``requests.get()`` can get |
| 105 |
replaced with calls to ``aiohttp``, for example. |
| 106 |
|
| 107 |
Depending on your Eventlet usage, during your migration, you may have to |
| 108 |
deprecate CLI options that are related to Eventlet, we invite the reader |
| 109 |
to take a look to :ref:`manage-your-deprecations`. |
| 110 |
|
| 111 |
For a more comprehensive migration guide, please visit the |
| 112 |
Eventlet migration guide available here: |
| 113 |
`https://removal.eventlet.org/ <https://removal.eventlet.org/>`_. |
| 114 |
|
| 115 |
This guide provides: |
| 116 |
|
| 117 |
* **Detailed migration steps** to transition from Eventlet to modern alternatives. |
| 118 |
* **Multiple alternatives** to Eventlet, including ``asyncio`` for asynchronous |
| 119 |
programming and Python's native ``threading`` module for multithreading. |
| 120 |
* **Advanced migration paradigms** to help you refactor your code incrementally |
| 121 |
and minimize disruptions during the transition. |
| 122 |
|
| 123 |
For example, the section `Preparing for Migration |
| 124 |
<https://removal.eventlet.org/guide/preparing-for-migration/>`_ |
| 125 |
introduces strategies to prepare your codebase for migration. |
| 126 |
It covers topics such as identifying blocking APIs, isolating Eventlet-specific |
| 127 |
code, and gradually replacing it with ``asyncio`` or other alternatives. |
| 128 |
|
| 129 |
The `awesome-asyncio <https://github.com/timofurrer/awesome-asyncio>`_ GitHub |
| 130 |
repository proposes a curated list of awesome Python asyncio frameworks, |
| 131 |
libraries, software, and resources. Do not hesitate to take a look at it. |
| 132 |
You may find candidates compatible with asyncio that can allow you to replace |
| 133 |
some of your actual underlying libraries. |
| 134 |
|
| 135 |
Step 3. Drop Eventlet altogether |
| 136 |
-------------------------------- |
| 137 |
|
| 138 |
Eventually you won't be relying on Eventlet at all: all your code will be |
| 139 |
``asyncio``-based. |
| 140 |
At this point you can drop Eventlet and switch to running the ``asyncio`` |
| 141 |
loop directly. |
| 142 |
|
| 143 |
Known limitations and work in progress |
| 144 |
-------------------------------------- |
| 145 |
|
| 146 |
In general, ``async`` functions and Eventlet green threads are two separate |
| 147 |
universes that just happen to be able to call each other. |
| 148 |
|
| 149 |
In ``async`` functions: |
| 150 |
|
| 151 |
* Eventlet thread locals probably won't work correctly. |
| 152 |
* ``evenlet.greenthread.getcurrent()`` won't give the result you expect. |
| 153 |
* ``eventlet`` locks and queues won't work if used directly. |
| 154 |
* Eventlet multiple readers are not supported, and so using |
| 155 |
``eventtlet.debug.hub_prevent_multiple_readers`` neither. |
| 156 |
|
| 157 |
In Eventlet greenlets: |
| 158 |
|
| 159 |
* ``asyncio`` locks won't work if used directly. |
| 160 |
|
| 161 |
We expect to add more migration and integration APIs over time as we learn |
| 162 |
more about what works, common idioms, and requirements for migration. |
| 163 |
You can track progress in the |
| 164 |
`GitHub issue <https://github.com/eventlet/eventlet/issues/868>`_, and file |
| 165 |
new issues if you have problems. |
| 166 |
|
| 167 |
Alternatives |
| 168 |
------------ |
| 169 |
|
| 170 |
If you really want to continue with Eventlet's pretend-to-be-blocking |
| 171 |
approach, you can use `gevent <https://www.gevent.org/>`_. |
| 172 |
But keep in mind that the same technical issues that make Eventlet maintenance |
| 173 |
unsustainable over the long term also apply to Gevent. |