| 1 |
:mod:`!http.cookiejar` --- Cookie handling for HTTP clients |
| 2 |
=========================================================== |
| 3 |
|
| 4 |
.. module:: http.cookiejar |
| 5 |
:synopsis: Classes for automatic handling of HTTP cookies. |
| 6 |
|
| 7 |
.. moduleauthor:: John J. Lee <[email protected]> |
| 8 |
.. sectionauthor:: John J. Lee <[email protected]> |
| 9 |
|
| 10 |
**Source code:** :source:`Lib/http/cookiejar.py` |
| 11 |
|
| 12 |
-------------- |
| 13 |
|
| 14 |
The :mod:`!http.cookiejar` module defines classes for automatic handling of HTTP |
| 15 |
cookies. It is useful for accessing websites that require small pieces of data |
| 16 |
-- :dfn:`cookies` -- to be set on the client machine by an HTTP response from a |
| 17 |
web server, and then returned to the server in later HTTP requests. |
| 18 |
|
| 19 |
Both the regular Netscape cookie protocol and the protocol defined by |
| 20 |
:rfc:`2965` are handled. RFC 2965 handling is switched off by default. |
| 21 |
:rfc:`2109` cookies are parsed as Netscape cookies and subsequently treated |
| 22 |
either as Netscape or RFC 2965 cookies according to the 'policy' in effect. |
| 23 |
Note that the great majority of cookies on the internet are Netscape cookies. |
| 24 |
:mod:`!http.cookiejar` attempts to follow the de-facto Netscape cookie protocol (which |
| 25 |
differs substantially from that set out in the original Netscape specification), |
| 26 |
including taking note of the ``max-age`` and ``port`` cookie-attributes |
| 27 |
introduced with RFC 2965. |
| 28 |
|
| 29 |
.. note:: |
| 30 |
|
| 31 |
The various named parameters found in :mailheader:`Set-Cookie` and |
| 32 |
:mailheader:`Set-Cookie2` headers (eg. ``domain`` and ``expires``) are |
| 33 |
conventionally referred to as :dfn:`attributes`. To distinguish them from |
| 34 |
Python attributes, the documentation for this module uses the term |
| 35 |
:dfn:`cookie-attribute` instead. |
| 36 |
|
| 37 |
|
| 38 |
The module defines the following exception: |
| 39 |
|
| 40 |
|
| 41 |
.. exception:: LoadError |
| 42 |
|
| 43 |
Instances of :class:`FileCookieJar` raise this exception on failure to load |
| 44 |
cookies from a file. :exc:`LoadError` is a subclass of :exc:`OSError`. |
| 45 |
|
| 46 |
.. versionchanged:: 3.3 |
| 47 |
:exc:`LoadError` used to be a subtype of :exc:`IOError`, which is now an |
| 48 |
alias of :exc:`OSError`. |
| 49 |
|
| 50 |
|
| 51 |
The following classes are provided: |
| 52 |
|
| 53 |
|
| 54 |
.. class:: CookieJar(policy=None) |
| 55 |
|
| 56 |
*policy* is an object implementing the :class:`CookiePolicy` interface. |
| 57 |
|
| 58 |
The :class:`CookieJar` class stores HTTP cookies. It extracts cookies from HTTP |
| 59 |
requests, and returns them in HTTP responses. :class:`CookieJar` instances |
| 60 |
automatically expire contained cookies when necessary. Subclasses are also |
| 61 |
responsible for storing and retrieving cookies from a file or database. |
| 62 |
|
| 63 |
|
| 64 |
.. class:: FileCookieJar(filename=None, delayload=None, policy=None) |
| 65 |
|
| 66 |
*policy* is an object implementing the :class:`CookiePolicy` interface. For the |
| 67 |
other arguments, see the documentation for the corresponding attributes. |
| 68 |
|
| 69 |
A :class:`CookieJar` which can load cookies from, and perhaps save cookies to, a |
| 70 |
file on disk. Cookies are **NOT** loaded from the named file until either the |
| 71 |
:meth:`load` or :meth:`revert` method is called. Subclasses of this class are |
| 72 |
documented in section :ref:`file-cookie-jar-classes`. |
| 73 |
|
| 74 |
This should not be initialized directly – use its subclasses below instead. |
| 75 |
|
| 76 |
.. versionchanged:: 3.8 |
| 77 |
|
| 78 |
The filename parameter supports a :term:`path-like object`. |
| 79 |
|
| 80 |
|
| 81 |
.. class:: CookiePolicy() |
| 82 |
|
| 83 |
This class is responsible for deciding whether each cookie should be accepted |
| 84 |
from / returned to the server. |
| 85 |
|
| 86 |
|
| 87 |
.. class:: DefaultCookiePolicy( blocked_domains=None, allowed_domains=None, netscape=True, rfc2965=False, rfc2109_as_netscape=None, hide_cookie2=False, strict_domain=False, strict_rfc2965_unverifiable=True, strict_ns_unverifiable=False, strict_ns_domain=DefaultCookiePolicy.DomainLiberal, strict_ns_set_initial_dollar=False, strict_ns_set_path=False, secure_protocols=("https", "wss") ) |
| 88 |
|
| 89 |
Constructor arguments should be passed as keyword arguments only. |
| 90 |
*blocked_domains* is a sequence of domain names that we never accept cookies |
| 91 |
from, nor return cookies to. *allowed_domains* if not :const:`None`, this is a |
| 92 |
sequence of the only domains for which we accept and return cookies. |
| 93 |
*secure_protocols* is a sequence of protocols for which secure cookies can be |
| 94 |
added to. By default *https* and *wss* (secure websocket) are considered |
| 95 |
secure protocols. For all other arguments, see the documentation for |
| 96 |
:class:`CookiePolicy` and :class:`DefaultCookiePolicy` objects. |
| 97 |
|
| 98 |
:class:`DefaultCookiePolicy` implements the standard accept / reject rules for |
| 99 |
Netscape and :rfc:`2965` cookies. By default, :rfc:`2109` cookies (ie. cookies |
| 100 |
received in a :mailheader:`Set-Cookie` header with a version cookie-attribute of |
| 101 |
1) are treated according to the RFC 2965 rules. However, if RFC 2965 handling |
| 102 |
is turned off or :attr:`rfc2109_as_netscape` is ``True``, RFC 2109 cookies are |
| 103 |
'downgraded' by the :class:`CookieJar` instance to Netscape cookies, by |
| 104 |
setting the :attr:`version` attribute of the :class:`Cookie` instance to 0. |
| 105 |
:class:`DefaultCookiePolicy` also provides some parameters to allow some |
| 106 |
fine-tuning of policy. |
| 107 |
|
| 108 |
|
| 109 |
.. class:: Cookie() |
| 110 |
|
| 111 |
This class represents Netscape, :rfc:`2109` and :rfc:`2965` cookies. It is not |
| 112 |
expected that users of :mod:`!http.cookiejar` construct their own :class:`Cookie` |
| 113 |
instances. Instead, if necessary, call :meth:`make_cookies` on a |
| 114 |
:class:`CookieJar` instance. |
| 115 |
|
| 116 |
|
| 117 |
.. seealso:: |
| 118 |
|
| 119 |
Module :mod:`urllib.request` |
| 120 |
URL opening with automatic cookie handling. |
| 121 |
|
| 122 |
Module :mod:`http.cookies` |
| 123 |
HTTP cookie classes, principally useful for server-side code. The |
| 124 |
:mod:`!http.cookiejar` and :mod:`http.cookies` modules do not depend on each |
| 125 |
other. |
| 126 |
|
| 127 |
https://curl.se/rfc/cookie_spec.html |
| 128 |
The specification of the original Netscape cookie protocol. Though this is |
| 129 |
still the dominant protocol, the 'Netscape cookie protocol' implemented by all |
| 130 |
the major browsers (and :mod:`!http.cookiejar`) only bears a passing resemblance to |
| 131 |
the one sketched out in ``cookie_spec.html``. |
| 132 |
|
| 133 |
:rfc:`2109` - HTTP State Management Mechanism |
| 134 |
Obsoleted by :rfc:`2965`. Uses :mailheader:`Set-Cookie` with version=1. |
| 135 |
|
| 136 |
:rfc:`2965` - HTTP State Management Mechanism |
| 137 |
The Netscape protocol with the bugs fixed. Uses :mailheader:`Set-Cookie2` in |
| 138 |
place of :mailheader:`Set-Cookie`. Not widely used. |
| 139 |
|
| 140 |
https://kristol.org/cookie/errata.html |
| 141 |
Unfinished errata to :rfc:`2965`. |
| 142 |
|
| 143 |
:rfc:`2964` - Use of HTTP State Management |
| 144 |
|
| 145 |
.. _cookie-jar-objects: |
| 146 |
|
| 147 |
CookieJar and FileCookieJar Objects |
| 148 |
----------------------------------- |
| 149 |
|
| 150 |
:class:`CookieJar` objects support the :term:`iterator` protocol for iterating over |
| 151 |
contained :class:`Cookie` objects. |
| 152 |
|
| 153 |
:class:`CookieJar` has the following methods: |
| 154 |
|
| 155 |
|
| 156 |
.. method:: CookieJar.add_cookie_header(request) |
| 157 |
|
| 158 |
Add correct :mailheader:`Cookie` header to *request*. |
| 159 |
|
| 160 |
If policy allows (ie. the :attr:`rfc2965` and :attr:`hide_cookie2` attributes of |
| 161 |
the :class:`CookieJar`'s :class:`CookiePolicy` instance are true and false |
| 162 |
respectively), the :mailheader:`Cookie2` header is also added when appropriate. |
| 163 |
|
| 164 |
The *request* object (usually a :class:`urllib.request.Request` instance) |
| 165 |
must support the methods :meth:`get_full_url`, :meth:`has_header`, |
| 166 |
:meth:`get_header`, :meth:`header_items`, :meth:`add_unredirected_header` |
| 167 |
and the attributes :attr:`host`, :attr:`!type`, :attr:`unverifiable` |
| 168 |
and :attr:`origin_req_host` as documented by :mod:`urllib.request`. |
| 169 |
|
| 170 |
.. versionchanged:: 3.3 |
| 171 |
|
| 172 |
*request* object needs :attr:`origin_req_host` attribute. Dependency on a |
| 173 |
deprecated method :meth:`get_origin_req_host` has been removed. |
| 174 |
|
| 175 |
|
| 176 |
.. method:: CookieJar.extract_cookies(response, request) |
| 177 |
|
| 178 |
Extract cookies from HTTP *response* and store them in the :class:`CookieJar`, |
| 179 |
where allowed by policy. |
| 180 |
|
| 181 |
The :class:`CookieJar` will look for allowable :mailheader:`Set-Cookie` and |
| 182 |
:mailheader:`Set-Cookie2` headers in the *response* argument, and store cookies |
| 183 |
as appropriate (subject to the :meth:`CookiePolicy.set_ok` method's approval). |
| 184 |
|
| 185 |
The *response* object (usually the result of a call to |
| 186 |
:meth:`urllib.request.urlopen`, or similar) should support an :meth:`info` |
| 187 |
method, which returns an :class:`email.message.Message` instance. |
| 188 |
|
| 189 |
The *request* object (usually a :class:`urllib.request.Request` instance) |
| 190 |
must support the method :meth:`get_full_url` and the attributes |
| 191 |
:attr:`host`, :attr:`unverifiable` and :attr:`origin_req_host`, |
| 192 |
as documented by :mod:`urllib.request`. The request is used to set |
| 193 |
default values for cookie-attributes as well as for checking that the |
| 194 |
cookie is allowed to be set. |
| 195 |
|
| 196 |
.. versionchanged:: 3.3 |
| 197 |
|
| 198 |
*request* object needs :attr:`origin_req_host` attribute. Dependency on a |
| 199 |
deprecated method :meth:`get_origin_req_host` has been removed. |
| 200 |
|
| 201 |
.. method:: CookieJar.set_policy(policy) |
| 202 |
|
| 203 |
Set the :class:`CookiePolicy` instance to be used. |
| 204 |
|
| 205 |
|
| 206 |
.. method:: CookieJar.make_cookies(response, request) |
| 207 |
|
| 208 |
Return sequence of :class:`Cookie` objects extracted from *response* object. |
| 209 |
|
| 210 |
See the documentation for :meth:`extract_cookies` for the interfaces required of |
| 211 |
the *response* and *request* arguments. |
| 212 |
|
| 213 |
|
| 214 |
.. method:: CookieJar.set_cookie_if_ok(cookie, request) |
| 215 |
|
| 216 |
Set a :class:`Cookie` if policy says it's OK to do so. |
| 217 |
|
| 218 |
|
| 219 |
.. method:: CookieJar.set_cookie(cookie) |
| 220 |
|
| 221 |
Set a :class:`Cookie`, without checking with policy to see whether or not it |
| 222 |
should be set. |
| 223 |
|
| 224 |
|
| 225 |
.. method:: CookieJar.clear([domain[, path[, name]]]) |
| 226 |
|
| 227 |
Clear some cookies. |
| 228 |
|
| 229 |
If invoked without arguments, clear all cookies. If given a single argument, |
| 230 |
only cookies belonging to that *domain* will be removed. If given two arguments, |
| 231 |
cookies belonging to the specified *domain* and URL *path* are removed. If |
| 232 |
given three arguments, then the cookie with the specified *domain*, *path* and |
| 233 |
*name* is removed. |
| 234 |
|
| 235 |
Raises :exc:`KeyError` if no matching cookie exists. |
| 236 |
|
| 237 |
|
| 238 |
.. method:: CookieJar.clear_session_cookies() |
| 239 |
|
| 240 |
Discard all session cookies. |
| 241 |
|
| 242 |
Discards all contained cookies that have a true :attr:`discard` attribute |
| 243 |
(usually because they had either no ``max-age`` or ``expires`` cookie-attribute, |
| 244 |
or an explicit ``discard`` cookie-attribute). For interactive browsers, the end |
| 245 |
of a session usually corresponds to closing the browser window. |
| 246 |
|
| 247 |
Note that the :meth:`save` method won't save session cookies anyway, unless you |
| 248 |
ask otherwise by passing a true *ignore_discard* argument. |
| 249 |
|
| 250 |
:class:`FileCookieJar` implements the following additional methods: |
| 251 |
|
| 252 |
|
| 253 |
.. method:: FileCookieJar.save(filename=None, ignore_discard=False, ignore_expires=False) |
| 254 |
|
| 255 |
Save cookies to a file. |
| 256 |
|
| 257 |
This base class raises :exc:`NotImplementedError`. Subclasses may leave this |
| 258 |
method unimplemented. |
| 259 |
|
| 260 |
*filename* is the name of file in which to save cookies. If *filename* is not |
| 261 |
specified, :attr:`self.filename` is used (whose default is the value passed to |
| 262 |
the constructor, if any); if :attr:`self.filename` is :const:`None`, |
| 263 |
:exc:`ValueError` is raised. |
| 264 |
|
| 265 |
*ignore_discard*: save even cookies set to be discarded. *ignore_expires*: save |
| 266 |
even cookies that have expired |
| 267 |
|
| 268 |
The file is overwritten if it already exists, thus wiping all the cookies it |
| 269 |
contains. Saved cookies can be restored later using the :meth:`load` or |
| 270 |
:meth:`revert` methods. |
| 271 |
|
| 272 |
|
| 273 |
.. method:: FileCookieJar.load(filename=None, ignore_discard=False, ignore_expires=False) |
| 274 |
|
| 275 |
Load cookies from a file. |
| 276 |
|
| 277 |
Old cookies are kept unless overwritten by newly loaded ones. |
| 278 |
|
| 279 |
Arguments are as for :meth:`save`. |
| 280 |
|
| 281 |
The named file must be in the format understood by the class, or |
| 282 |
:exc:`LoadError` will be raised. Also, :exc:`OSError` may be raised, for |
| 283 |
example if the file does not exist. |
| 284 |
|
| 285 |
.. versionchanged:: 3.3 |
| 286 |
:exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`. |
| 287 |
|
| 288 |
|
| 289 |
.. method:: FileCookieJar.revert(filename=None, ignore_discard=False, ignore_expires=False) |
| 290 |
|
| 291 |
Clear all cookies and reload cookies from a saved file. |
| 292 |
|
| 293 |
:meth:`revert` can raise the same exceptions as :meth:`load`. If there is a |
| 294 |
failure, the object's state will not be altered. |
| 295 |
|
| 296 |
:class:`FileCookieJar` instances have the following public attributes: |
| 297 |
|
| 298 |
|
| 299 |
.. attribute:: FileCookieJar.filename |
| 300 |
|
| 301 |
Filename of default file in which to keep cookies. This attribute may be |
| 302 |
assigned to. |
| 303 |
|
| 304 |
|
| 305 |
.. attribute:: FileCookieJar.delayload |
| 306 |
|
| 307 |
If true, load cookies lazily from disk. This attribute should not be assigned |
| 308 |
to. This is only a hint, since this only affects performance, not behaviour |
| 309 |
(unless the cookies on disk are changing). A :class:`CookieJar` object may |
| 310 |
ignore it. None of the :class:`FileCookieJar` classes included in the standard |
| 311 |
library lazily loads cookies. |
| 312 |
|
| 313 |
|
| 314 |
.. _file-cookie-jar-classes: |
| 315 |
|
| 316 |
FileCookieJar subclasses and co-operation with web browsers |
| 317 |
----------------------------------------------------------- |
| 318 |
|
| 319 |
The following :class:`CookieJar` subclasses are provided for reading and |
| 320 |
writing. |
| 321 |
|
| 322 |
.. class:: MozillaCookieJar(filename=None, delayload=None, policy=None) |
| 323 |
|
| 324 |
A :class:`FileCookieJar` that can load from and save cookies to disk in the |
| 325 |
Mozilla ``cookies.txt`` file format (which is also used by curl and the Lynx |
| 326 |
and Netscape browsers). |
| 327 |
|
| 328 |
.. note:: |
| 329 |
|
| 330 |
This loses information about :rfc:`2965` cookies, and also about newer or |
| 331 |
non-standard cookie-attributes such as ``port``. |
| 332 |
|
| 333 |
.. warning:: |
| 334 |
|
| 335 |
Back up your cookies before saving if you have cookies whose loss / corruption |
| 336 |
would be inconvenient (there are some subtleties which may lead to slight |
| 337 |
changes in the file over a load / save round-trip). |
| 338 |
|
| 339 |
Also note that cookies saved while Mozilla is running will get clobbered by |
| 340 |
Mozilla. |
| 341 |
|
| 342 |
|
| 343 |
.. class:: LWPCookieJar(filename=None, delayload=None, policy=None) |
| 344 |
|
| 345 |
A :class:`FileCookieJar` that can load from and save cookies to disk in format |
| 346 |
compatible with the libwww-perl library's ``Set-Cookie3`` file format. This is |
| 347 |
convenient if you want to store cookies in a human-readable file. |
| 348 |
|
| 349 |
.. versionchanged:: 3.8 |
| 350 |
|
| 351 |
The filename parameter supports a :term:`path-like object`. |
| 352 |
|
| 353 |
.. _cookie-policy-objects: |
| 354 |
|
| 355 |
CookiePolicy Objects |
| 356 |
-------------------- |
| 357 |
|
| 358 |
Objects implementing the :class:`CookiePolicy` interface have the following |
| 359 |
methods: |
| 360 |
|
| 361 |
|
| 362 |
.. method:: CookiePolicy.set_ok(cookie, request) |
| 363 |
|
| 364 |
Return boolean value indicating whether cookie should be accepted from server. |
| 365 |
|
| 366 |
*cookie* is a :class:`Cookie` instance. *request* is an object |
| 367 |
implementing the interface defined by the documentation for |
| 368 |
:meth:`CookieJar.extract_cookies`. |
| 369 |
|
| 370 |
|
| 371 |
.. method:: CookiePolicy.return_ok(cookie, request) |
| 372 |
|
| 373 |
Return boolean value indicating whether cookie should be returned to server. |
| 374 |
|
| 375 |
*cookie* is a :class:`Cookie` instance. *request* is an object |
| 376 |
implementing the interface defined by the documentation for |
| 377 |
:meth:`CookieJar.add_cookie_header`. |
| 378 |
|
| 379 |
|
| 380 |
.. method:: CookiePolicy.domain_return_ok(domain, request) |
| 381 |
|
| 382 |
Return ``False`` if cookies should not be returned, given cookie domain. |
| 383 |
|
| 384 |
This method is an optimization. It removes the need for checking every cookie |
| 385 |
with a particular domain (which might involve reading many files). Returning |
| 386 |
true from :meth:`domain_return_ok` and :meth:`path_return_ok` leaves all the |
| 387 |
work to :meth:`return_ok`. |
| 388 |
|
| 389 |
If :meth:`domain_return_ok` returns true for the cookie domain, |
| 390 |
:meth:`path_return_ok` is called for the cookie path. Otherwise, |
| 391 |
:meth:`path_return_ok` and :meth:`return_ok` are never called for that cookie |
| 392 |
domain. If :meth:`path_return_ok` returns true, :meth:`return_ok` is called |
| 393 |
with the :class:`Cookie` object itself for a full check. Otherwise, |
| 394 |
:meth:`return_ok` is never called for that cookie path. |
| 395 |
|
| 396 |
Note that :meth:`domain_return_ok` is called for every *cookie* domain, not just |
| 397 |
for the *request* domain. For example, the function might be called with both |
| 398 |
``".example.com"`` and ``"www.example.com"`` if the request domain is |
| 399 |
``"www.example.com"``. The same goes for :meth:`path_return_ok`. |
| 400 |
|
| 401 |
The *request* argument is as documented for :meth:`return_ok`. |
| 402 |
|
| 403 |
|
| 404 |
.. method:: CookiePolicy.path_return_ok(path, request) |
| 405 |
|
| 406 |
Return ``False`` if cookies should not be returned, given cookie path. |
| 407 |
|
| 408 |
See the documentation for :meth:`domain_return_ok`. |
| 409 |
|
| 410 |
In addition to implementing the methods above, implementations of the |
| 411 |
:class:`CookiePolicy` interface must also supply the following attributes, |
| 412 |
indicating which protocols should be used, and how. All of these attributes may |
| 413 |
be assigned to. |
| 414 |
|
| 415 |
|
| 416 |
.. attribute:: CookiePolicy.netscape |
| 417 |
|
| 418 |
Implement Netscape protocol. |
| 419 |
|
| 420 |
|
| 421 |
.. attribute:: CookiePolicy.rfc2965 |
| 422 |
|
| 423 |
Implement :rfc:`2965` protocol. |
| 424 |
|
| 425 |
|
| 426 |
.. attribute:: CookiePolicy.hide_cookie2 |
| 427 |
|
| 428 |
Don't add :mailheader:`Cookie2` header to requests (the presence of this header |
| 429 |
indicates to the server that we understand :rfc:`2965` cookies). |
| 430 |
|
| 431 |
The most useful way to define a :class:`CookiePolicy` class is by subclassing |
| 432 |
from :class:`DefaultCookiePolicy` and overriding some or all of the methods |
| 433 |
above. :class:`CookiePolicy` itself may be used as a 'null policy' to allow |
| 434 |
setting and receiving any and all cookies (this is unlikely to be useful). |
| 435 |
|
| 436 |
|
| 437 |
.. _default-cookie-policy-objects: |
| 438 |
|
| 439 |
DefaultCookiePolicy Objects |
| 440 |
--------------------------- |
| 441 |
|
| 442 |
Implements the standard rules for accepting and returning cookies. |
| 443 |
|
| 444 |
Both :rfc:`2965` and Netscape cookies are covered. RFC 2965 handling is switched |
| 445 |
off by default. |
| 446 |
|
| 447 |
The easiest way to provide your own policy is to override this class and call |
| 448 |
its methods in your overridden implementations before adding your own additional |
| 449 |
checks:: |
| 450 |
|
| 451 |
import http.cookiejar |
| 452 |
class MyCookiePolicy(http.cookiejar.DefaultCookiePolicy): |
| 453 |
def set_ok(self, cookie, request): |
| 454 |
if not http.cookiejar.DefaultCookiePolicy.set_ok(self, cookie, request): |
| 455 |
return False |
| 456 |
if i_dont_want_to_store_this_cookie(cookie): |
| 457 |
return False |
| 458 |
return True |
| 459 |
|
| 460 |
In addition to the features required to implement the :class:`CookiePolicy` |
| 461 |
interface, this class allows you to block and allow domains from setting and |
| 462 |
receiving cookies. There are also some strictness switches that allow you to |
| 463 |
tighten up the rather loose Netscape protocol rules a little bit (at the cost of |
| 464 |
blocking some benign cookies). |
| 465 |
|
| 466 |
A domain blocklist and allowlist is provided (both off by default). Only domains |
| 467 |
not in the blocklist and present in the allowlist (if the allowlist is active) |
| 468 |
participate in cookie setting and returning. Use the *blocked_domains* |
| 469 |
constructor argument, and :meth:`blocked_domains` and |
| 470 |
:meth:`set_blocked_domains` methods (and the corresponding argument and methods |
| 471 |
for *allowed_domains*). If you set an allowlist, you can turn it off again by |
| 472 |
setting it to :const:`None`. |
| 473 |
|
| 474 |
Domains in block or allow lists that do not start with a dot must equal the |
| 475 |
cookie domain to be matched. For example, ``"example.com"`` matches a blocklist |
| 476 |
entry of ``"example.com"``, but ``"www.example.com"`` does not. Domains that do |
| 477 |
start with a dot are matched by more specific domains too. For example, both |
| 478 |
``"www.example.com"`` and ``"www.coyote.example.com"`` match ``".example.com"`` |
| 479 |
(but ``"example.com"`` itself does not). IP addresses are an exception, and |
| 480 |
must match exactly. For example, if blocked_domains contains ``"192.168.1.2"`` |
| 481 |
and ``".168.1.2"``, 192.168.1.2 is blocked, but 193.168.1.2 is not. |
| 482 |
|
| 483 |
:class:`DefaultCookiePolicy` implements the following additional methods: |
| 484 |
|
| 485 |
|
| 486 |
.. method:: DefaultCookiePolicy.blocked_domains() |
| 487 |
|
| 488 |
Return the sequence of blocked domains (as a tuple). |
| 489 |
|
| 490 |
|
| 491 |
.. method:: DefaultCookiePolicy.set_blocked_domains(blocked_domains) |
| 492 |
|
| 493 |
Set the sequence of blocked domains. |
| 494 |
|
| 495 |
|
| 496 |
.. method:: DefaultCookiePolicy.is_blocked(domain) |
| 497 |
|
| 498 |
Return ``True`` if *domain* is on the blocklist for setting or receiving |
| 499 |
cookies. |
| 500 |
|
| 501 |
|
| 502 |
.. method:: DefaultCookiePolicy.allowed_domains() |
| 503 |
|
| 504 |
Return :const:`None`, or the sequence of allowed domains (as a tuple). |
| 505 |
|
| 506 |
|
| 507 |
.. method:: DefaultCookiePolicy.set_allowed_domains(allowed_domains) |
| 508 |
|
| 509 |
Set the sequence of allowed domains, or :const:`None`. |
| 510 |
|
| 511 |
|
| 512 |
.. method:: DefaultCookiePolicy.is_not_allowed(domain) |
| 513 |
|
| 514 |
Return ``True`` if *domain* is not on the allowlist for setting or receiving |
| 515 |
cookies. |
| 516 |
|
| 517 |
:class:`DefaultCookiePolicy` instances have the following attributes, which are |
| 518 |
all initialised from the constructor arguments of the same name, and which may |
| 519 |
all be assigned to. |
| 520 |
|
| 521 |
|
| 522 |
.. attribute:: DefaultCookiePolicy.rfc2109_as_netscape |
| 523 |
|
| 524 |
If true, request that the :class:`CookieJar` instance downgrade :rfc:`2109` cookies |
| 525 |
(ie. cookies received in a :mailheader:`Set-Cookie` header with a version |
| 526 |
cookie-attribute of 1) to Netscape cookies by setting the version attribute of |
| 527 |
the :class:`Cookie` instance to 0. The default value is :const:`None`, in which |
| 528 |
case RFC 2109 cookies are downgraded if and only if :rfc:`2965` handling is turned |
| 529 |
off. Therefore, RFC 2109 cookies are downgraded by default. |
| 530 |
|
| 531 |
|
| 532 |
General strictness switches: |
| 533 |
|
| 534 |
.. attribute:: DefaultCookiePolicy.strict_domain |
| 535 |
|
| 536 |
Don't allow sites to set two-component domains with country-code top-level |
| 537 |
domains like ``.co.uk``, ``.gov.uk``, ``.co.nz``.etc. This is far from perfect |
| 538 |
and isn't guaranteed to work! |
| 539 |
|
| 540 |
|
| 541 |
:rfc:`2965` protocol strictness switches: |
| 542 |
|
| 543 |
.. attribute:: DefaultCookiePolicy.strict_rfc2965_unverifiable |
| 544 |
|
| 545 |
Follow :rfc:`2965` rules on unverifiable transactions (usually, an unverifiable |
| 546 |
transaction is one resulting from a redirect or a request for an image hosted on |
| 547 |
another site). If this is false, cookies are *never* blocked on the basis of |
| 548 |
verifiability |
| 549 |
|
| 550 |
|
| 551 |
Netscape protocol strictness switches: |
| 552 |
|
| 553 |
.. attribute:: DefaultCookiePolicy.strict_ns_unverifiable |
| 554 |
|
| 555 |
Apply :rfc:`2965` rules on unverifiable transactions even to Netscape cookies. |
| 556 |
|
| 557 |
|
| 558 |
.. attribute:: DefaultCookiePolicy.strict_ns_domain |
| 559 |
|
| 560 |
Flags indicating how strict to be with domain-matching rules for Netscape |
| 561 |
cookies. See below for acceptable values. |
| 562 |
|
| 563 |
|
| 564 |
.. attribute:: DefaultCookiePolicy.strict_ns_set_initial_dollar |
| 565 |
|
| 566 |
Ignore cookies in Set-Cookie: headers that have names starting with ``'$'``. |
| 567 |
|
| 568 |
|
| 569 |
.. attribute:: DefaultCookiePolicy.strict_ns_set_path |
| 570 |
|
| 571 |
Don't allow setting cookies whose path doesn't path-match request URI. |
| 572 |
|
| 573 |
:attr:`~DefaultCookiePolicy.strict_ns_domain` is a collection of flags. Its value is constructed by |
| 574 |
or-ing together (for example, ``DomainStrictNoDots|DomainStrictNonDomain`` means |
| 575 |
both flags are set). |
| 576 |
|
| 577 |
|
| 578 |
.. attribute:: DefaultCookiePolicy.DomainStrictNoDots |
| 579 |
|
| 580 |
When setting cookies, the 'host prefix' must not contain a dot (eg. |
| 581 |
``www.foo.bar.com`` can't set a cookie for ``.bar.com``, because ``www.foo`` |
| 582 |
contains a dot). |
| 583 |
|
| 584 |
|
| 585 |
.. attribute:: DefaultCookiePolicy.DomainStrictNonDomain |
| 586 |
|
| 587 |
Cookies that did not explicitly specify a ``domain`` cookie-attribute can only |
| 588 |
be returned to a domain equal to the domain that set the cookie (eg. |
| 589 |
``spam.example.com`` won't be returned cookies from ``example.com`` that had no |
| 590 |
``domain`` cookie-attribute). |
| 591 |
|
| 592 |
|
| 593 |
.. attribute:: DefaultCookiePolicy.DomainRFC2965Match |
| 594 |
|
| 595 |
When setting cookies, require a full :rfc:`2965` domain-match. |
| 596 |
|
| 597 |
The following attributes are provided for convenience, and are the most useful |
| 598 |
combinations of the above flags: |
| 599 |
|
| 600 |
|
| 601 |
.. attribute:: DefaultCookiePolicy.DomainLiberal |
| 602 |
|
| 603 |
Equivalent to 0 (ie. all of the above Netscape domain strictness flags switched |
| 604 |
off). |
| 605 |
|
| 606 |
|
| 607 |
.. attribute:: DefaultCookiePolicy.DomainStrict |
| 608 |
|
| 609 |
Equivalent to ``DomainStrictNoDots|DomainStrictNonDomain``. |
| 610 |
|
| 611 |
|
| 612 |
Cookie Objects |
| 613 |
-------------- |
| 614 |
|
| 615 |
:class:`Cookie` instances have Python attributes roughly corresponding to the |
| 616 |
standard cookie-attributes specified in the various cookie standards. The |
| 617 |
correspondence is not one-to-one, because there are complicated rules for |
| 618 |
assigning default values, because the ``max-age`` and ``expires`` |
| 619 |
cookie-attributes contain equivalent information, and because :rfc:`2109` cookies |
| 620 |
may be 'downgraded' by :mod:`!http.cookiejar` from version 1 to version 0 (Netscape) |
| 621 |
cookies. |
| 622 |
|
| 623 |
Assignment to these attributes should not be necessary other than in rare |
| 624 |
circumstances in a :class:`CookiePolicy` method. The class does not enforce |
| 625 |
internal consistency, so you should know what you're doing if you do that. |
| 626 |
|
| 627 |
|
| 628 |
.. attribute:: Cookie.version |
| 629 |
|
| 630 |
Integer or :const:`None`. Netscape cookies have :attr:`version` 0. :rfc:`2965` and |
| 631 |
:rfc:`2109` cookies have a ``version`` cookie-attribute of 1. However, note that |
| 632 |
:mod:`!http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in which |
| 633 |
case :attr:`version` is 0. |
| 634 |
|
| 635 |
|
| 636 |
.. attribute:: Cookie.name |
| 637 |
|
| 638 |
Cookie name (a string). |
| 639 |
|
| 640 |
|
| 641 |
.. attribute:: Cookie.value |
| 642 |
|
| 643 |
Cookie value (a string), or :const:`None`. |
| 644 |
|
| 645 |
|
| 646 |
.. attribute:: Cookie.port |
| 647 |
|
| 648 |
String representing a port or a set of ports (eg. '80', or '80,8080'), or |
| 649 |
:const:`None`. |
| 650 |
|
| 651 |
|
| 652 |
.. attribute:: Cookie.domain |
| 653 |
|
| 654 |
Cookie domain (a string). |
| 655 |
|
| 656 |
|
| 657 |
.. attribute:: Cookie.path |
| 658 |
|
| 659 |
Cookie path (a string, eg. ``'/acme/rocket_launchers'``). |
| 660 |
|
| 661 |
|
| 662 |
.. attribute:: Cookie.secure |
| 663 |
|
| 664 |
``True`` if cookie should only be returned over a secure connection. |
| 665 |
|
| 666 |
|
| 667 |
.. attribute:: Cookie.expires |
| 668 |
|
| 669 |
Integer expiry date in seconds since epoch, or :const:`None`. See also the |
| 670 |
:meth:`is_expired` method. |
| 671 |
|
| 672 |
|
| 673 |
.. attribute:: Cookie.discard |
| 674 |
|
| 675 |
``True`` if this is a session cookie. |
| 676 |
|
| 677 |
|
| 678 |
.. attribute:: Cookie.comment |
| 679 |
|
| 680 |
String comment from the server explaining the function of this cookie, or |
| 681 |
:const:`None`. |
| 682 |
|
| 683 |
|
| 684 |
.. attribute:: Cookie.comment_url |
| 685 |
|
| 686 |
URL linking to a comment from the server explaining the function of this cookie, |
| 687 |
or :const:`None`. |
| 688 |
|
| 689 |
|
| 690 |
.. attribute:: Cookie.rfc2109 |
| 691 |
|
| 692 |
``True`` if this cookie was received as an :rfc:`2109` cookie (ie. the cookie |
| 693 |
arrived in a :mailheader:`Set-Cookie` header, and the value of the Version |
| 694 |
cookie-attribute in that header was 1). This attribute is provided because |
| 695 |
:mod:`!http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in |
| 696 |
which case :attr:`version` is 0. |
| 697 |
|
| 698 |
|
| 699 |
.. attribute:: Cookie.port_specified |
| 700 |
|
| 701 |
``True`` if a port or set of ports was explicitly specified by the server (in the |
| 702 |
:mailheader:`Set-Cookie` / :mailheader:`Set-Cookie2` header). |
| 703 |
|
| 704 |
|
| 705 |
.. attribute:: Cookie.domain_specified |
| 706 |
|
| 707 |
``True`` if a domain was explicitly specified by the server. |
| 708 |
|
| 709 |
|
| 710 |
.. attribute:: Cookie.domain_initial_dot |
| 711 |
|
| 712 |
``True`` if the domain explicitly specified by the server began with a dot |
| 713 |
(``'.'``). |
| 714 |
|
| 715 |
Cookies may have additional non-standard cookie-attributes. These may be |
| 716 |
accessed using the following methods: |
| 717 |
|
| 718 |
|
| 719 |
.. method:: Cookie.has_nonstandard_attr(name) |
| 720 |
|
| 721 |
Return ``True`` if cookie has the named cookie-attribute. |
| 722 |
|
| 723 |
|
| 724 |
.. method:: Cookie.get_nonstandard_attr(name, default=None) |
| 725 |
|
| 726 |
If cookie has the named cookie-attribute, return its value. Otherwise, return |
| 727 |
*default*. |
| 728 |
|
| 729 |
|
| 730 |
.. method:: Cookie.set_nonstandard_attr(name, value) |
| 731 |
|
| 732 |
Set the value of the named cookie-attribute. |
| 733 |
|
| 734 |
The :class:`Cookie` class also defines the following method: |
| 735 |
|
| 736 |
|
| 737 |
.. method:: Cookie.is_expired(now=None) |
| 738 |
|
| 739 |
``True`` if cookie has passed the time at which the server requested it should |
| 740 |
expire. If *now* is given (in seconds since the epoch), return whether the |
| 741 |
cookie has expired at the specified time. |
| 742 |
|
| 743 |
|
| 744 |
Examples |
| 745 |
-------- |
| 746 |
|
| 747 |
The first example shows the most common usage of :mod:`!http.cookiejar`:: |
| 748 |
|
| 749 |
import http.cookiejar, urllib.request |
| 750 |
cj = http.cookiejar.CookieJar() |
| 751 |
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) |
| 752 |
r = opener.open("http://example.com/") |
| 753 |
|
| 754 |
This example illustrates how to open a URL using your Netscape, Mozilla, or Lynx |
| 755 |
cookies (assumes Unix/Netscape convention for location of the cookies file):: |
| 756 |
|
| 757 |
import os, http.cookiejar, urllib.request |
| 758 |
cj = http.cookiejar.MozillaCookieJar() |
| 759 |
cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt")) |
| 760 |
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) |
| 761 |
r = opener.open("http://example.com/") |
| 762 |
|
| 763 |
The next example illustrates the use of :class:`DefaultCookiePolicy`. Turn on |
| 764 |
:rfc:`2965` cookies, be more strict about domains when setting and returning |
| 765 |
Netscape cookies, and block some domains from setting cookies or having them |
| 766 |
returned:: |
| 767 |
|
| 768 |
import urllib.request |
| 769 |
from http.cookiejar import CookieJar, DefaultCookiePolicy |
| 770 |
policy = DefaultCookiePolicy( |
| 771 |
rfc2965=True, strict_ns_domain=Policy.DomainStrict, |
| 772 |
blocked_domains=["ads.net", ".ads.net"]) |
| 773 |
cj = CookieJar(policy) |
| 774 |
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) |
| 775 |
r = opener.open("http://example.com/") |