Debian Codegraph @

doc/source/asyncio/migration.rst

raw
1 .. _migration-guide:
2
3 Migrating off of Eventlet
4 =========================
5
6 There are two main use cases for Eventlet:
7
8 1. As a required networking framework, much like one would use ``asyncio``,
9 ``trio``, or older frameworks like ``Twisted`` and ``tornado``.
10
11 2. As an optional, pluggable backend that allows swapping out blocking APIs
12 for an event loop, transparently, without changing any code.
13 This is how Celery and Gunicorn use eventlet.
14
15 Pretending to look like a blocking API while actually using an event loop
16 underneath requires exact emulation of an ever-changing and ever-increasing
17 API footprint, which is fundamentally unsustainable for a volunteer-driven
18 open source project.
19 This is why Eventlet is discouraging new users.
20
21 **Most of this document will focus on the first use case: Eventlet as the sole
22 networking framework.**
23 For this use case, we recommend migrating to Python's ``asyncio``, and we are
24 providing infrastructure that will make this much easier, and allow for
25 *gradual* migration.
26
27 For the second use case, we believe this is a fundamentally unsustainable
28 approach and encourage the upstream frameworks to come up with different
29 solutions.
30
31 Step 1. Switch to the ``asyncio`` Hub
32 -------------------------------------
33
34 Eventlet has different pluggable networking event loops.
35 By switching the event loop to use ``asyncio``, you enable running ``asyncio``
36 and Eventlet code in the same thread in the same process.
37
38 To do so, set the ``EVENTLET_HUB`` environment variable to ``asyncio`` before
39 starting your Eventlet program.
40 For example, if you start your program with a shell script, you can do
41 ``export EVENTLET_HUB=asyncio``.
42
43 Alternatively, you can explicitly specify the ``asyncio`` hub at startup,
44 before monkey patching or any other setup work::
45
46 import eventlet.hubs
47 eventlet.hubs.use_hub("eventlet.hubs.asyncio")
48
49 Step 2. Migrate code to ``asyncio``
50 -----------------------------------
51
52 Now that you're running Eventlet on top of ``asyncio``, you can use some new
53 APIs to call from Eventlet code into ``asyncio``, and vice-versa.
54
55 To call ``asyncio`` code from Eventlet code, you can wrap a coroutine (or
56 anything you can ``await``) into an Eventlet ``GreenThread``.
57 For example, if you want to make a HTTP request from Eventlet, you can use
58 the ``asyncio``-based ``aiohttp`` library::
59
60 import aiohttp
61 from eventlet.asyncio import spawn_for_awaitable
62
63 async def request():
64 async with aiohttp.ClientSession() as session:
65 url = "https://example.com"
66 async with session.get(url) as response:
67 html = await response.text()
68 return html
69
70
71 # This makes a coroutine; typically you'd ``await`` it:
72 coro = request()
73
74 # You can wrap this coroutine with an Eventlet GreenThread, similar to
75 # ``evenlet.spawn()``:
76 gthread = spawn_for_awaitable(request())
77
78 # And then get its result, the body of https://example.com:
79 result = gthread.wait()
80
81 In the other direction, any ``eventlet.greenthread.GreenThread`` can be
82 ``await``-ed in ``async`` functions.
83 In other words ``async`` functions can call into Eventlet code::
84
85 def blocking_eventlet_api():
86 eventlet.sleep(1)
87 # do some other pseudo-blocking work
88 # ...
89 return 12
90
91 async def my_async_func():
92 gthread = eventlet.spawn(blocking_eventlet_api)
93 # In normal Eventlet code we'd call gthread.wait(), but since this is an
94 # async function we'll want to await instead:
95 result = await gthread
96 # result is now 12
97 # ...
98
99 Cancellation of ``asyncio.Future`` and killing of ``eventlet.GreenThread``
100 should propagate between the two.
101
102 Using these two APIs, with more to come, you can gradually migrate portions of
103 your application or library to ``asyncio``.
104 Calls to blocking APIs like ``urlopen()`` or ``requests.get()`` can get
105 replaced with calls to ``aiohttp``, for example.
106
107 Depending on your Eventlet usage, during your migration, you may have to
108 deprecate CLI options that are related to Eventlet, we invite the reader
109 to take a look to :ref:`manage-your-deprecations`.
110
111 For a more comprehensive migration guide, please visit the
112 Eventlet migration guide available here:
113 `https://removal.eventlet.org/ <https://removal.eventlet.org/>`_.
114
115 This guide provides:
116
117 * **Detailed migration steps** to transition from Eventlet to modern alternatives.
118 * **Multiple alternatives** to Eventlet, including ``asyncio`` for asynchronous
119 programming and Python's native ``threading`` module for multithreading.
120 * **Advanced migration paradigms** to help you refactor your code incrementally
121 and minimize disruptions during the transition.
122
123 For example, the section `Preparing for Migration
124 <https://removal.eventlet.org/guide/preparing-for-migration/>`_
125 introduces strategies to prepare your codebase for migration.
126 It covers topics such as identifying blocking APIs, isolating Eventlet-specific
127 code, and gradually replacing it with ``asyncio`` or other alternatives.
128
129 The `awesome-asyncio <https://github.com/timofurrer/awesome-asyncio>`_ GitHub
130 repository proposes a curated list of awesome Python asyncio frameworks,
131 libraries, software, and resources. Do not hesitate to take a look at it.
132 You may find candidates compatible with asyncio that can allow you to replace
133 some of your actual underlying libraries.
134
135 Step 3. Drop Eventlet altogether
136 --------------------------------
137
138 Eventually you won't be relying on Eventlet at all: all your code will be
139 ``asyncio``-based.
140 At this point you can drop Eventlet and switch to running the ``asyncio``
141 loop directly.
142
143 Known limitations and work in progress
144 --------------------------------------
145
146 In general, ``async`` functions and Eventlet green threads are two separate
147 universes that just happen to be able to call each other.
148
149 In ``async`` functions:
150
151 * Eventlet thread locals probably won't work correctly.
152 * ``evenlet.greenthread.getcurrent()`` won't give the result you expect.
153 * ``eventlet`` locks and queues won't work if used directly.
154 * Eventlet multiple readers are not supported, and so using
155 ``eventtlet.debug.hub_prevent_multiple_readers`` neither.
156
157 In Eventlet greenlets:
158
159 * ``asyncio`` locks won't work if used directly.
160
161 We expect to add more migration and integration APIs over time as we learn
162 more about what works, common idioms, and requirements for migration.
163 You can track progress in the
164 `GitHub issue <https://github.com/eventlet/eventlet/issues/868>`_, and file
165 new issues if you have problems.
166
167 Alternatives
168 ------------
169
170 If you really want to continue with Eventlet's pretend-to-be-blocking
171 approach, you can use `gevent <https://www.gevent.org/>`_.
172 But keep in mind that the same technical issues that make Eventlet maintenance
173 unsustainable over the long term also apply to Gevent.