Debian Codegraph @

Doc/library/http.client.rst

raw
1 :mod:`!http.client` --- HTTP protocol client
2 ============================================
3
4 .. module:: http.client
5 :synopsis: HTTP and HTTPS protocol client (requires sockets).
6
7 **Source code:** :source:`Lib/http/client.py`
8
9 .. index::
10 pair: HTTP; protocol
11 single: HTTP; http.client (standard module)
12
13 .. index:: pair: module; urllib.request
14
15 --------------
16
17 This module defines classes that implement the client side of the HTTP and
18 HTTPS protocols. It is normally not used directly --- the module
19 :mod:`urllib.request` uses it to handle URLs that use HTTP and HTTPS.
20
21 .. seealso::
22
23 The `Requests package <https://requests.readthedocs.io/en/latest/>`_
24 is recommended for a higher-level HTTP client interface.
25
26 .. note::
27
28 HTTPS support is only available if Python was compiled with SSL support
29 (through the :mod:`ssl` module).
30
31 .. include:: ../includes/wasm-notavail.rst
32
33 The module provides the following classes:
34
35
36 .. class:: HTTPConnection(host, port=None[, timeout], source_address=None, \
37 blocksize=8192)
38
39 An :class:`HTTPConnection` instance represents one transaction with an HTTP
40 server. It should be instantiated by passing it a host and optional port
41 number. If no port number is passed, the port is extracted from the host
42 string if it has the form ``host:port``, else the default HTTP port (80) is
43 used. If the optional *timeout* parameter is given, blocking
44 operations (like connection attempts) will timeout after that many seconds
45 (if it is not given, the global default timeout setting is used).
46 The optional *source_address* parameter may be a tuple of a (host, port)
47 to use as the source address the HTTP connection is made from.
48 The optional *blocksize* parameter sets the buffer size in bytes for
49 sending a file-like message body.
50
51 For example, the following calls all create instances that connect to the server
52 at the same host and port::
53
54 >>> h1 = http.client.HTTPConnection('www.python.org')
55 >>> h2 = http.client.HTTPConnection('www.python.org:80')
56 >>> h3 = http.client.HTTPConnection('www.python.org', 80)
57 >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)
58
59 .. versionchanged:: 3.2
60 *source_address* was added.
61
62 .. versionchanged:: 3.4
63 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are
64 no longer supported.
65
66 .. versionchanged:: 3.7
67 *blocksize* parameter was added.
68
69
70 .. class:: HTTPSConnection(host, port=None, *[, timeout], \
71 source_address=None, context=None, \
72 blocksize=8192)
73
74 A subclass of :class:`HTTPConnection` that uses SSL for communication with
75 secure servers. Default port is ``443``. If *context* is specified, it
76 must be a :class:`ssl.SSLContext` instance describing the various SSL
77 options.
78
79 Please read :ref:`ssl-security` for more information on best practices.
80
81 .. versionchanged:: 3.2
82 *source_address*, *context* and *check_hostname* were added.
83
84 .. versionchanged:: 3.2
85 This class now supports HTTPS virtual hosts if possible (that is,
86 if :const:`ssl.HAS_SNI` is true).
87
88 .. versionchanged:: 3.4
89 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are
90 no longer supported.
91
92 .. versionchanged:: 3.4.3
93 This class now performs all the necessary certificate and hostname checks
94 by default. To revert to the previous, unverified, behavior
95 :func:`!ssl._create_unverified_context` can be passed to the *context*
96 parameter.
97
98 .. versionchanged:: 3.8
99 This class now enables TLS 1.3
100 :attr:`ssl.SSLContext.post_handshake_auth` for the default *context* or
101 when *cert_file* is passed with a custom *context*.
102
103 .. versionchanged:: 3.10
104 This class now sends an ALPN extension with protocol indicator
105 ``http/1.1`` when no *context* is given. Custom *context* should set
106 ALPN protocols with :meth:`~ssl.SSLContext.set_alpn_protocols`.
107
108 .. versionchanged:: 3.12
109 The deprecated *key_file*, *cert_file* and *check_hostname* parameters
110 have been removed.
111
112
113 .. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None)
114
115 Class whose instances are returned upon successful connection. Not
116 instantiated directly by user.
117
118 .. versionchanged:: 3.4
119 The *strict* parameter was removed. HTTP 0.9 style "Simple Responses" are
120 no longer supported.
121
122 This module provides the following function:
123
124 .. function:: parse_headers(fp)
125
126 Parse the headers from a file pointer *fp* representing a HTTP
127 request/response. The file has to be a :class:`~io.BufferedIOBase` reader
128 (i.e. not text) and must provide a valid :rfc:`5322` style header.
129
130 This function returns an instance of :class:`http.client.HTTPMessage`
131 that holds the header fields, but no payload
132 (the same as :attr:`HTTPResponse.msg`
133 and :attr:`http.server.BaseHTTPRequestHandler.headers`).
134 After returning, the file pointer *fp* is ready to read the HTTP body.
135
136 .. note::
137 :meth:`parse_headers` does not parse the start-line of a HTTP message;
138 it only parses the ``Name: value`` lines. The file has to be ready to
139 read these field lines, so the first line should already be consumed
140 before calling the function.
141
142 The following exceptions are raised as appropriate:
143
144
145 .. exception:: HTTPException
146
147 The base class of the other exceptions in this module. It is a subclass of
148 :exc:`Exception`.
149
150
151 .. exception:: NotConnected
152
153 A subclass of :exc:`HTTPException`.
154
155
156 .. exception:: InvalidURL
157
158 A subclass of :exc:`HTTPException`, raised if a port is given and is either
159 non-numeric or empty.
160
161
162 .. exception:: UnknownProtocol
163
164 A subclass of :exc:`HTTPException`.
165
166
167 .. exception:: UnknownTransferEncoding
168
169 A subclass of :exc:`HTTPException`.
170
171
172 .. exception:: UnimplementedFileMode
173
174 A subclass of :exc:`HTTPException`.
175
176
177 .. exception:: IncompleteRead
178
179 A subclass of :exc:`HTTPException`.
180
181
182 .. exception:: ImproperConnectionState
183
184 A subclass of :exc:`HTTPException`.
185
186
187 .. exception:: CannotSendRequest
188
189 A subclass of :exc:`ImproperConnectionState`.
190
191
192 .. exception:: CannotSendHeader
193
194 A subclass of :exc:`ImproperConnectionState`.
195
196
197 .. exception:: ResponseNotReady
198
199 A subclass of :exc:`ImproperConnectionState`.
200
201
202 .. exception:: BadStatusLine
203
204 A subclass of :exc:`HTTPException`. Raised if a server responds with a HTTP
205 status code that we don't understand.
206
207
208 .. exception:: LineTooLong
209
210 A subclass of :exc:`HTTPException`. Raised if an excessively long line
211 is received in the HTTP protocol from the server.
212
213
214 .. exception:: RemoteDisconnected
215
216 A subclass of :exc:`ConnectionResetError` and :exc:`BadStatusLine`. Raised
217 by :meth:`HTTPConnection.getresponse` when the attempt to read the response
218 results in no data read from the connection, indicating that the remote end
219 has closed the connection.
220
221 .. versionadded:: 3.5
222 Previously, :exc:`BadStatusLine`\ ``('')`` was raised.
223
224
225 The constants defined in this module are:
226
227 .. data:: HTTP_PORT
228
229 The default port for the HTTP protocol (always ``80``).
230
231 .. data:: HTTPS_PORT
232
233 The default port for the HTTPS protocol (always ``443``).
234
235 .. data:: responses
236
237 This dictionary maps the HTTP 1.1 status codes to the W3C names.
238
239 Example: ``http.client.responses[http.client.NOT_FOUND]`` is ``'Not Found'``.
240
241 See :ref:`http-status-codes` for a list of HTTP status codes that are
242 available in this module as constants.
243
244
245 .. _httpconnection-objects:
246
247 HTTPConnection Objects
248 ----------------------
249
250 :class:`HTTPConnection` instances have the following methods:
251
252
253 .. method:: HTTPConnection.request(method, url, body=None, headers={}, *, \
254 encode_chunked=False)
255
256 This will send a request to the server using the HTTP request
257 method *method* and the request URI *url*. The provided *url* must be
258 an absolute path to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
259 (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or
260 ``CONNECT`` methods).
261
262 If *body* is specified, the specified data is sent after the headers are
263 finished. It may be a :class:`str`, a :term:`bytes-like object`, an
264 open :term:`file object`, or an iterable of :class:`bytes`. If *body*
265 is a string, it is encoded as ISO-8859-1, the default for HTTP. If it
266 is a bytes-like object, the bytes are sent as is. If it is a :term:`file
267 object`, the contents of the file is sent; this file object should
268 support at least the ``read()`` method. If the file object is an
269 instance of :class:`io.TextIOBase`, the data returned by the ``read()``
270 method will be encoded as ISO-8859-1, otherwise the data returned by
271 ``read()`` is sent as is. If *body* is an iterable, the elements of the
272 iterable are sent as is until the iterable is exhausted.
273
274 The *headers* argument should be a mapping of extra HTTP headers to send
275 with the request. A :rfc:`Host header <2616#section-14.23>`
276 must be provided to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
277 (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or
278 ``CONNECT`` methods).
279
280 If *headers* contains neither Content-Length nor Transfer-Encoding,
281 but there is a request body, one of those
282 header fields will be added automatically. If
283 *body* is ``None``, the Content-Length header is set to ``0`` for
284 methods that expect a body (``PUT``, ``POST``, and ``PATCH``). If
285 *body* is a string or a bytes-like object that is not also a
286 :term:`file <file object>`, the Content-Length header is
287 set to its length. Any other type of *body* (files
288 and iterables in general) will be chunk-encoded, and the
289 Transfer-Encoding header will automatically be set instead of
290 Content-Length.
291
292 The *encode_chunked* argument is only relevant if Transfer-Encoding is
293 specified in *headers*. If *encode_chunked* is ``False``, the
294 HTTPConnection object assumes that all encoding is handled by the
295 calling code. If it is ``True``, the body will be chunk-encoded.
296
297 For example, to perform a ``GET`` request to ``https://docs.python.org/3/``::
298
299 >>> import http.client
300 >>> host = "docs.python.org"
301 >>> conn = http.client.HTTPSConnection(host)
302 >>> conn.request("GET", "/3/", headers={"Host": host})
303 >>> response = conn.getresponse()
304 >>> print(response.status, response.reason)
305 200 OK
306
307 .. note::
308 Chunked transfer encoding has been added to the HTTP protocol
309 version 1.1. Unless the HTTP server is known to handle HTTP 1.1,
310 the caller must either specify the Content-Length, or must pass a
311 :class:`str` or bytes-like object that is not also a file as the
312 body representation.
313
314 .. note::
315
316 Note that you must have read the whole response or call :meth:`close`
317 if :meth:`getresponse` raised an non-:exc:`ConnectionError` exception
318 before you can send a new request to the server.
319
320 .. versionchanged:: 3.2
321 *body* can now be an iterable.
322
323 .. versionchanged:: 3.6
324 If neither Content-Length nor Transfer-Encoding are set in
325 *headers*, file and iterable *body* objects are now chunk-encoded.
326 The *encode_chunked* argument was added.
327 No attempt is made to determine the Content-Length for file
328 objects.
329
330 .. method:: HTTPConnection.getresponse()
331
332 Should be called after a request is sent to get the response from the server.
333 Returns an :class:`HTTPResponse` instance.
334
335 .. versionchanged:: 3.5
336 If a :exc:`ConnectionError` or subclass is raised, the
337 :class:`HTTPConnection` object will be ready to reconnect when
338 a new request is sent.
339
340 Note that this does not apply to :exc:`OSError`\s raised by the underlying
341 socket. Instead the caller is responsible to call :meth:`close` on the
342 existing connection.
343
344
345 .. method:: HTTPConnection.set_debuglevel(level)
346
347 Set the debugging level. The default debug level is ``0``, meaning no
348 debugging output is printed. Any value greater than ``0`` will cause all
349 currently defined debug output to be printed to stdout. The ``debuglevel``
350 is passed to any new :class:`HTTPResponse` objects that are created.
351
352 .. versionadded:: 3.1
353
354
355 .. method:: HTTPConnection.set_tunnel(host, port=None, headers=None)
356
357 Set the host and the port for HTTP Connect Tunnelling. This allows running
358 the connection through a proxy server.
359
360 The *host* and *port* arguments specify the endpoint of the tunneled connection
361 (i.e. the address included in the CONNECT request, *not* the address of the
362 proxy server).
363
364 The *headers* argument should be a mapping of extra HTTP headers to send with
365 the CONNECT request.
366
367 As HTTP/1.1 is used for HTTP CONNECT tunnelling request, `as per the RFC
368 <https://datatracker.ietf.org/doc/html/rfc7231#section-4.3.6>`_, a HTTP ``Host:``
369 header must be provided, matching the authority-form of the request target
370 provided as the destination for the CONNECT request. If a HTTP ``Host:``
371 header is not provided via the headers argument, one is generated and
372 transmitted automatically.
373
374 For example, to tunnel through a HTTPS proxy server running locally on port
375 8080, we would pass the address of the proxy to the :class:`HTTPSConnection`
376 constructor, and the address of the host that we eventually want to reach to
377 the :meth:`~HTTPConnection.set_tunnel` method::
378
379 >>> import http.client
380 >>> conn = http.client.HTTPSConnection("localhost", 8080)
381 >>> conn.set_tunnel("www.python.org")
382 >>> conn.request("HEAD","/index.html")
383
384 .. versionadded:: 3.2
385
386 .. versionchanged:: 3.12
387 HTTP CONNECT tunnelling requests use protocol HTTP/1.1, upgraded from
388 protocol HTTP/1.0. ``Host:`` HTTP headers are mandatory for HTTP/1.1, so
389 one will be automatically generated and transmitted if not provided in
390 the headers argument.
391
392
393 .. method:: HTTPConnection.get_proxy_response_headers()
394
395 Returns a dictionary with the headers of the response received from
396 the proxy server to the CONNECT request.
397
398 If the CONNECT request was not sent, the method returns ``None``.
399
400 .. versionadded:: 3.12
401
402
403 .. method:: HTTPConnection.connect()
404
405 Connect to the server specified when the object was created. By default,
406 this is called automatically when making a request if the client does not
407 already have a connection.
408
409 .. audit-event:: http.client.connect self,host,port http.client.HTTPConnection.connect
410
411
412 .. method:: HTTPConnection.close()
413
414 Close the connection to the server.
415
416
417 .. attribute:: HTTPConnection.blocksize
418
419 Buffer size in bytes for sending a file-like message body.
420
421 .. versionadded:: 3.7
422
423
424 As an alternative to using the :meth:`~HTTPConnection.request` method described above, you can
425 also send your request step by step, by using the four functions below.
426
427
428 .. method:: HTTPConnection.putrequest(method, url, skip_host=False, \
429 skip_accept_encoding=False)
430
431 This should be the first call after the connection to the server has been
432 made. It sends a line to the server consisting of the *method* string,
433 the *url* string, and the HTTP version (``HTTP/1.1``). To disable automatic
434 sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept
435 additional content encodings), specify *skip_host* or *skip_accept_encoding*
436 with non-False values.
437
438
439 .. method:: HTTPConnection.putheader(header, argument[, ...])
440
441 Send an :rfc:`822`\ -style header to the server. It sends a line to the server
442 consisting of the header, a colon and a space, and the first argument. If more
443 arguments are given, continuation lines are sent, each consisting of a tab and
444 an argument.
445
446
447 .. method:: HTTPConnection.endheaders(message_body=None, *, encode_chunked=False)
448
449 Send a blank line to the server, signalling the end of the headers. The
450 optional *message_body* argument can be used to pass a message body
451 associated with the request.
452
453 If *encode_chunked* is ``True``, the result of each iteration of
454 *message_body* will be chunk-encoded as specified in :rfc:`7230`,
455 Section 3.3.1. How the data is encoded is dependent on the type of
456 *message_body*. If *message_body* implements the :ref:`buffer interface
457 <bufferobjects>` the encoding will result in a single chunk.
458 If *message_body* is a :class:`collections.abc.Iterable`, each iteration
459 of *message_body* will result in a chunk. If *message_body* is a
460 :term:`file object`, each call to ``.read()`` will result in a chunk.
461 The method automatically signals the end of the chunk-encoded data
462 immediately after *message_body*.
463
464 .. note:: Due to the chunked encoding specification, empty chunks
465 yielded by an iterator body will be ignored by the chunk-encoder.
466 This is to avoid premature termination of the read of the request by
467 the target server due to malformed encoding.
468
469 .. versionchanged:: 3.6
470 Added chunked encoding support and the *encode_chunked* parameter.
471
472
473 .. method:: HTTPConnection.send(data)
474
475 Send data to the server. This should be used directly only after the
476 :meth:`endheaders` method has been called and before :meth:`getresponse` is
477 called.
478
479 .. audit-event:: http.client.send self,data http.client.HTTPConnection.send
480
481
482 .. _httpresponse-objects:
483
484 HTTPResponse Objects
485 --------------------
486
487 An :class:`HTTPResponse` instance wraps the HTTP response from the
488 server. It provides access to the request headers and the entity
489 body. The response is an iterable object and can be used in a with
490 statement.
491
492 .. versionchanged:: 3.5
493 The :class:`io.BufferedIOBase` interface is now implemented and
494 all of its reader operations are supported.
495
496
497 .. method:: HTTPResponse.read([amt])
498
499 Reads and returns the response body, or up to the next *amt* bytes.
500
501 .. method:: HTTPResponse.readinto(b)
502
503 Reads up to the next len(b) bytes of the response body into the buffer *b*.
504 Returns the number of bytes read.
505
506 .. versionadded:: 3.3
507
508 .. method:: HTTPResponse.getheader(name, default=None)
509
510 Return the value of the header *name*, or *default* if there is no header
511 matching *name*. If there is more than one header with the name *name*,
512 return all of the values joined by ', '. If *default* is any iterable other
513 than a single string, its elements are similarly returned joined by commas.
514
515 .. method:: HTTPResponse.getheaders()
516
517 Return a list of (header, value) tuples.
518
519 .. method:: HTTPResponse.fileno()
520
521 Return the ``fileno`` of the underlying socket.
522
523 .. attribute:: HTTPResponse.msg
524
525 A :class:`http.client.HTTPMessage` instance containing the response
526 headers. :class:`http.client.HTTPMessage` is a subclass of
527 :class:`email.message.Message`.
528
529 .. attribute:: HTTPResponse.version
530
531 HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1.
532
533 .. attribute:: HTTPResponse.url
534
535 URL of the resource retrieved, commonly used to determine if a redirect was followed.
536
537 .. attribute:: HTTPResponse.headers
538
539 Headers of the response in the form of an :class:`email.message.EmailMessage` instance.
540
541 .. attribute:: HTTPResponse.status
542
543 Status code returned by server.
544
545 .. attribute:: HTTPResponse.reason
546
547 Reason phrase returned by server.
548
549 .. attribute:: HTTPResponse.debuglevel
550
551 A debugging hook. If :attr:`debuglevel` is greater than zero, messages
552 will be printed to stdout as the response is read and parsed.
553
554 .. attribute:: HTTPResponse.closed
555
556 Is ``True`` if the stream is closed.
557
558 .. method:: HTTPResponse.geturl()
559
560 .. deprecated:: 3.9
561 Deprecated in favor of :attr:`~HTTPResponse.url`.
562
563 .. method:: HTTPResponse.info()
564
565 .. deprecated:: 3.9
566 Deprecated in favor of :attr:`~HTTPResponse.headers`.
567
568 .. method:: HTTPResponse.getcode()
569
570 .. deprecated:: 3.9
571 Deprecated in favor of :attr:`~HTTPResponse.status`.
572
573 Examples
574 --------
575
576 Here is an example session that uses the ``GET`` method::
577
578 >>> import http.client
579 >>> conn = http.client.HTTPSConnection("www.python.org")
580 >>> conn.request("GET", "/")
581 >>> r1 = conn.getresponse()
582 >>> print(r1.status, r1.reason)
583 200 OK
584 >>> data1 = r1.read() # This will return entire content.
585 >>> # The following example demonstrates reading data in chunks.
586 >>> conn.request("GET", "/")
587 >>> r1 = conn.getresponse()
588 >>> while chunk := r1.read(200):
589 ... print(repr(chunk))
590 b'<!doctype html>\n<!--[if"...
591 ...
592 >>> # Example of an invalid request
593 >>> conn = http.client.HTTPSConnection("docs.python.org")
594 >>> conn.request("GET", "/parrot.spam")
595 >>> r2 = conn.getresponse()
596 >>> print(r2.status, r2.reason)
597 404 Not Found
598 >>> data2 = r2.read()
599 >>> conn.close()
600
601 Here is an example session that uses the ``HEAD`` method. Note that the
602 ``HEAD`` method never returns any data. ::
603
604 >>> import http.client
605 >>> conn = http.client.HTTPSConnection("www.python.org")
606 >>> conn.request("HEAD", "/")
607 >>> res = conn.getresponse()
608 >>> print(res.status, res.reason)
609 200 OK
610 >>> data = res.read()
611 >>> print(len(data))
612 0
613 >>> data == b''
614 True
615
616 Here is an example session that uses the ``POST`` method::
617
618 >>> import http.client, urllib.parse
619 >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
620 >>> headers = {"Content-type": "application/x-www-form-urlencoded",
621 ... "Accept": "text/plain"}
622 >>> conn = http.client.HTTPConnection("bugs.python.org")
623 >>> conn.request("POST", "", params, headers)
624 >>> response = conn.getresponse()
625 >>> print(response.status, response.reason)
626 302 Found
627 >>> data = response.read()
628 >>> data
629 b'Redirecting to <a href="https://bugs.python.org/issue12524">https://bugs.python.org/issue12524</a>'
630 >>> conn.close()
631
632 Client side HTTP ``PUT`` requests are very similar to ``POST`` requests. The
633 difference lies only on the server side where HTTP servers will allow resources to
634 be created via ``PUT`` requests. It should be noted that custom HTTP methods
635 are also handled in :class:`urllib.request.Request` by setting the appropriate
636 method attribute. Here is an example session that uses the ``PUT`` method::
637
638 >>> # This creates an HTTP request
639 >>> # with the content of BODY as the enclosed representation
640 >>> # for the resource http://localhost:8080/file
641 ...
642 >>> import http.client
643 >>> BODY = "***filecontents***"
644 >>> conn = http.client.HTTPConnection("localhost", 8080)
645 >>> conn.request("PUT", "/file", BODY)
646 >>> response = conn.getresponse()
647 >>> print(response.status, response.reason)
648 200, OK
649
650 .. _httpmessage-objects:
651
652 HTTPMessage Objects
653 -------------------
654
655 .. class:: HTTPMessage(email.message.Message)
656
657 An :class:`http.client.HTTPMessage` instance holds the headers from an HTTP
658 response. It is implemented using the :class:`email.message.Message` class.
659
660 .. XXX Define the methods that clients can depend upon between versions.