| 1 |
.. _tut-brieftour: |
| 2 |
|
| 3 |
********************************** |
| 4 |
Brief tour of the standard library |
| 5 |
********************************** |
| 6 |
|
| 7 |
|
| 8 |
.. _tut-os-interface: |
| 9 |
|
| 10 |
Operating system interface |
| 11 |
========================== |
| 12 |
|
| 13 |
The :mod:`os` module provides dozens of functions for interacting with the |
| 14 |
operating system:: |
| 15 |
|
| 16 |
>>> import os |
| 17 |
>>> os.getcwd() # Return the current working directory |
| 18 |
'C:\\Python314' |
| 19 |
>>> os.chdir('/server/accesslogs') # Change current working directory |
| 20 |
>>> os.system('mkdir today') # Run the command mkdir in the system shell |
| 21 |
0 |
| 22 |
|
| 23 |
Be sure to use the ``import os`` style instead of ``from os import *``. This |
| 24 |
will keep :func:`os.open` from shadowing the built-in :func:`open` function which |
| 25 |
operates much differently. |
| 26 |
|
| 27 |
.. index:: pair: built-in function; help |
| 28 |
|
| 29 |
The built-in :func:`dir` and :func:`help` functions are useful as interactive |
| 30 |
aids for working with large modules like :mod:`os`:: |
| 31 |
|
| 32 |
>>> import os |
| 33 |
>>> dir(os) |
| 34 |
<returns a list of all module functions> |
| 35 |
>>> help(os) |
| 36 |
<returns an extensive manual page created from the module's docstrings> |
| 37 |
|
| 38 |
For daily file and directory management tasks, the :mod:`shutil` module provides |
| 39 |
a higher level interface that is easier to use:: |
| 40 |
|
| 41 |
>>> import shutil |
| 42 |
>>> shutil.copyfile('data.db', 'archive.db') |
| 43 |
'archive.db' |
| 44 |
>>> shutil.move('/build/executables', 'installdir') |
| 45 |
'installdir' |
| 46 |
|
| 47 |
|
| 48 |
.. _tut-file-wildcards: |
| 49 |
|
| 50 |
File wildcards |
| 51 |
============== |
| 52 |
|
| 53 |
The :mod:`glob` module provides a function for making file lists from directory |
| 54 |
wildcard searches:: |
| 55 |
|
| 56 |
>>> import glob |
| 57 |
>>> glob.glob('*.py') |
| 58 |
['primes.py', 'random.py', 'quote.py'] |
| 59 |
|
| 60 |
|
| 61 |
.. _tut-command-line-arguments: |
| 62 |
|
| 63 |
Command-line arguments |
| 64 |
====================== |
| 65 |
|
| 66 |
Common utility scripts often need to process command line arguments. These |
| 67 |
arguments are stored in the :mod:`sys` module's *argv* attribute as a list. For |
| 68 |
instance, let's take the following :file:`demo.py` file:: |
| 69 |
|
| 70 |
# File demo.py |
| 71 |
import sys |
| 72 |
print(sys.argv) |
| 73 |
|
| 74 |
Here is the output from running ``python demo.py one two three`` at the command |
| 75 |
line:: |
| 76 |
|
| 77 |
['demo.py', 'one', 'two', 'three'] |
| 78 |
|
| 79 |
The :mod:`argparse` module provides a more sophisticated mechanism to process |
| 80 |
command line arguments. The following script extracts one or more filenames |
| 81 |
and an optional number of lines to be displayed:: |
| 82 |
|
| 83 |
import argparse |
| 84 |
|
| 85 |
parser = argparse.ArgumentParser( |
| 86 |
prog='top', |
| 87 |
description='Show top lines from each file') |
| 88 |
parser.add_argument('filenames', nargs='+') |
| 89 |
parser.add_argument('-l', '--lines', type=int, default=10) |
| 90 |
args = parser.parse_args() |
| 91 |
print(args) |
| 92 |
|
| 93 |
When run at the command line with ``python top.py --lines=5 alpha.txt |
| 94 |
beta.txt``, the script sets ``args.lines`` to ``5`` and ``args.filenames`` |
| 95 |
to ``['alpha.txt', 'beta.txt']``. |
| 96 |
|
| 97 |
|
| 98 |
.. _tut-stderr: |
| 99 |
|
| 100 |
Error output redirection and program termination |
| 101 |
================================================ |
| 102 |
|
| 103 |
The :mod:`sys` module also has attributes for *stdin*, *stdout*, and *stderr*. |
| 104 |
The latter is useful for emitting warnings and error messages to make them |
| 105 |
visible even when *stdout* has been redirected:: |
| 106 |
|
| 107 |
>>> sys.stderr.write('Warning, log file not found starting a new one\n') |
| 108 |
Warning, log file not found starting a new one |
| 109 |
|
| 110 |
The most direct way to terminate a script is to use ``sys.exit()``. |
| 111 |
|
| 112 |
|
| 113 |
.. _tut-string-pattern-matching: |
| 114 |
|
| 115 |
String pattern matching |
| 116 |
======================= |
| 117 |
|
| 118 |
The :mod:`re` module provides regular expression tools for advanced string |
| 119 |
processing. For complex matching and manipulation, regular expressions offer |
| 120 |
succinct, optimized solutions:: |
| 121 |
|
| 122 |
>>> import re |
| 123 |
>>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest') |
| 124 |
['foot', 'fell', 'fastest'] |
| 125 |
>>> re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat') |
| 126 |
'cat in the hat' |
| 127 |
|
| 128 |
When only simple capabilities are needed, string methods are preferred because |
| 129 |
they are easier to read and debug:: |
| 130 |
|
| 131 |
>>> 'tea for too'.replace('too', 'two') |
| 132 |
'tea for two' |
| 133 |
|
| 134 |
|
| 135 |
.. _tut-mathematics: |
| 136 |
|
| 137 |
Mathematics |
| 138 |
=========== |
| 139 |
|
| 140 |
The :mod:`math` module gives access to the underlying C library functions for |
| 141 |
floating-point math:: |
| 142 |
|
| 143 |
>>> import math |
| 144 |
>>> math.cos(math.pi / 4) |
| 145 |
0.70710678118654757 |
| 146 |
>>> math.log(1024, 2) |
| 147 |
10.0 |
| 148 |
|
| 149 |
The :mod:`random` module provides tools for making random selections:: |
| 150 |
|
| 151 |
>>> import random |
| 152 |
>>> random.choice(['apple', 'pear', 'banana']) |
| 153 |
'apple' |
| 154 |
>>> random.sample(range(100), 10) # sampling without replacement |
| 155 |
[30, 83, 16, 4, 8, 81, 41, 50, 18, 33] |
| 156 |
>>> random.random() # random float from the interval [0.0, 1.0) |
| 157 |
0.17970987693706186 |
| 158 |
>>> random.randrange(6) # random integer chosen from range(6) |
| 159 |
4 |
| 160 |
|
| 161 |
The :mod:`statistics` module calculates basic statistical properties |
| 162 |
(the mean, median, variance, etc.) of numeric data:: |
| 163 |
|
| 164 |
>>> import statistics |
| 165 |
>>> data = [2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5] |
| 166 |
>>> statistics.mean(data) |
| 167 |
1.6071428571428572 |
| 168 |
>>> statistics.median(data) |
| 169 |
1.25 |
| 170 |
>>> statistics.variance(data) |
| 171 |
1.3720238095238095 |
| 172 |
|
| 173 |
The SciPy project <https://scipy.org> has many other modules for numerical |
| 174 |
computations. |
| 175 |
|
| 176 |
.. _tut-internet-access: |
| 177 |
|
| 178 |
Internet access |
| 179 |
=============== |
| 180 |
|
| 181 |
There are a number of modules for accessing the internet and processing internet |
| 182 |
protocols. Two of the simplest are :mod:`urllib.request` for retrieving data |
| 183 |
from URLs and :mod:`smtplib` for sending mail:: |
| 184 |
|
| 185 |
>>> from urllib.request import urlopen |
| 186 |
>>> with urlopen('https://docs.python.org/3/') as response: |
| 187 |
... for line in response: |
| 188 |
... line = line.decode() # Convert bytes to a str |
| 189 |
... if 'updated' in line: |
| 190 |
... print(line.rstrip()) # Remove trailing newline |
| 191 |
... |
| 192 |
Last updated on Nov 11, 2025 (20:11 UTC). |
| 193 |
|
| 194 |
>>> import smtplib |
| 195 |
>>> server = smtplib.SMTP('localhost') |
| 196 |
>>> server.sendmail('[email protected]', '[email protected]', |
| 197 |
... """To: [email protected] |
| 198 |
... From: [email protected] |
| 199 |
... |
| 200 |
... Beware the Ides of March. |
| 201 |
... """) |
| 202 |
>>> server.quit() |
| 203 |
|
| 204 |
(Note that the second example needs a mailserver running on localhost.) |
| 205 |
|
| 206 |
|
| 207 |
.. _tut-dates-and-times: |
| 208 |
|
| 209 |
Dates and times |
| 210 |
=============== |
| 211 |
|
| 212 |
The :mod:`datetime` module supplies classes for manipulating dates and times in |
| 213 |
both simple and complex ways. While date and time arithmetic is supported, the |
| 214 |
focus of the implementation is on efficient member extraction for output |
| 215 |
formatting and manipulation. The module also supports objects that are timezone |
| 216 |
aware. :: |
| 217 |
|
| 218 |
>>> # dates are easily constructed and formatted |
| 219 |
>>> import datetime as dt |
| 220 |
>>> now = dt.date.today() |
| 221 |
>>> now |
| 222 |
datetime.date(2003, 12, 2) |
| 223 |
>>> now.strftime("%m-%d-%y. %d %b %Y is a %A on the %d day of %B.") |
| 224 |
'12-02-03. 02 Dec 2003 is a Tuesday on the 02 day of December.' |
| 225 |
|
| 226 |
>>> # dates support calendar arithmetic |
| 227 |
>>> birthday = dt.date(1964, 7, 31) |
| 228 |
>>> age = now - birthday |
| 229 |
>>> age.days |
| 230 |
14368 |
| 231 |
|
| 232 |
|
| 233 |
.. _tut-data-compression: |
| 234 |
|
| 235 |
Data compression |
| 236 |
================ |
| 237 |
|
| 238 |
Common data archiving and compression formats are directly supported by modules |
| 239 |
including: :mod:`zlib`, :mod:`gzip`, :mod:`bz2`, :mod:`lzma`, :mod:`zipfile` and |
| 240 |
:mod:`tarfile`. :: |
| 241 |
|
| 242 |
>>> import zlib |
| 243 |
>>> s = b'witch which has which witches wrist watch' |
| 244 |
>>> len(s) |
| 245 |
41 |
| 246 |
>>> t = zlib.compress(s) |
| 247 |
>>> len(t) |
| 248 |
37 |
| 249 |
>>> zlib.decompress(t) |
| 250 |
b'witch which has which witches wrist watch' |
| 251 |
>>> zlib.crc32(s) |
| 252 |
226805979 |
| 253 |
|
| 254 |
|
| 255 |
.. _tut-performance-measurement: |
| 256 |
|
| 257 |
Performance measurement |
| 258 |
======================= |
| 259 |
|
| 260 |
Some Python users develop a deep interest in knowing the relative performance of |
| 261 |
different approaches to the same problem. Python provides a measurement tool |
| 262 |
that answers those questions immediately. |
| 263 |
|
| 264 |
For example, it may be tempting to use the tuple packing and unpacking feature |
| 265 |
instead of the traditional approach to swapping arguments. The :mod:`timeit` |
| 266 |
module quickly demonstrates a modest performance advantage:: |
| 267 |
|
| 268 |
>>> from timeit import Timer |
| 269 |
>>> Timer('t=a; a=b; b=t', 'a=1; b=2').timeit() |
| 270 |
0.57535828626024577 |
| 271 |
>>> Timer('a,b = b,a', 'a=1; b=2').timeit() |
| 272 |
0.54962537085770791 |
| 273 |
|
| 274 |
In contrast to :mod:`timeit`'s fine level of granularity, the :mod:`profile` and |
| 275 |
:mod:`pstats` modules provide tools for identifying time critical sections in |
| 276 |
larger blocks of code. |
| 277 |
|
| 278 |
|
| 279 |
.. _tut-quality-control: |
| 280 |
|
| 281 |
Quality control |
| 282 |
=============== |
| 283 |
|
| 284 |
One approach for developing high quality software is to write tests for each |
| 285 |
function as it is developed and to run those tests frequently during the |
| 286 |
development process. |
| 287 |
|
| 288 |
The :mod:`doctest` module provides a tool for scanning a module and validating |
| 289 |
tests embedded in a program's docstrings. Test construction is as simple as |
| 290 |
cutting-and-pasting a typical call along with its results into the docstring. |
| 291 |
This improves the documentation by providing the user with an example and it |
| 292 |
allows the doctest module to make sure the code remains true to the |
| 293 |
documentation:: |
| 294 |
|
| 295 |
def average(values): |
| 296 |
"""Computes the arithmetic mean of a list of numbers. |
| 297 |
|
| 298 |
>>> print(average([20, 30, 70])) |
| 299 |
40.0 |
| 300 |
""" |
| 301 |
return sum(values) / len(values) |
| 302 |
|
| 303 |
import doctest |
| 304 |
doctest.testmod() # automatically validate the embedded tests |
| 305 |
|
| 306 |
The :mod:`unittest` module is not as effortless as the :mod:`doctest` module, |
| 307 |
but it allows a more comprehensive set of tests to be maintained in a separate |
| 308 |
file:: |
| 309 |
|
| 310 |
import unittest |
| 311 |
|
| 312 |
class TestStatisticalFunctions(unittest.TestCase): |
| 313 |
|
| 314 |
def test_average(self): |
| 315 |
self.assertEqual(average([20, 30, 70]), 40.0) |
| 316 |
self.assertEqual(round(average([1, 5, 7]), 1), 4.3) |
| 317 |
with self.assertRaises(ZeroDivisionError): |
| 318 |
average([]) |
| 319 |
with self.assertRaises(TypeError): |
| 320 |
average(20, 30, 70) |
| 321 |
|
| 322 |
unittest.main() # Calling from the command line invokes all tests |
| 323 |
|
| 324 |
|
| 325 |
.. _tut-batteries-included: |
| 326 |
|
| 327 |
Batteries included |
| 328 |
================== |
| 329 |
|
| 330 |
Python has a "batteries included" philosophy. This is best seen through the |
| 331 |
sophisticated and robust capabilities of its larger packages. For example: |
| 332 |
|
| 333 |
* The :mod:`xmlrpc.client` and :mod:`xmlrpc.server` modules make implementing |
| 334 |
remote procedure calls into an almost trivial task. Despite the modules' |
| 335 |
names, no direct knowledge or handling of XML is needed. |
| 336 |
|
| 337 |
* The :mod:`email` package is a library for managing email messages, including |
| 338 |
MIME and other :rfc:`5322`-based message documents. Unlike :mod:`smtplib` and |
| 339 |
:mod:`poplib` which actually send and receive messages, the email package has |
| 340 |
a complete toolset for building or decoding complex message structures |
| 341 |
(including attachments) and for implementing internet encoding and header |
| 342 |
protocols. |
| 343 |
|
| 344 |
* The :mod:`json` package provides robust support for parsing this |
| 345 |
popular data interchange format. The :mod:`csv` module supports |
| 346 |
direct reading and writing of files in Comma-Separated Value format, |
| 347 |
commonly supported by databases and spreadsheets. XML processing is |
| 348 |
supported by the :mod:`xml.etree.ElementTree`, :mod:`xml.dom` and |
| 349 |
:mod:`xml.sax` packages. Together, these modules and packages |
| 350 |
greatly simplify data interchange between Python applications and |
| 351 |
other tools. |
| 352 |
|
| 353 |
* The :mod:`sqlite3` module is a wrapper for the SQLite database |
| 354 |
library, providing a persistent database that can be updated and |
| 355 |
accessed using slightly nonstandard SQL syntax. |
| 356 |
|
| 357 |
* Internationalization is supported by a number of modules including |
| 358 |
:mod:`gettext`, :mod:`locale`, and the :mod:`codecs` package. |