Debian Codegraph @

Doc/faq/library.rst

raw
1 :tocdepth: 2
2
3 =========================
4 Library and Extension FAQ
5 =========================
6
7 .. only:: html
8
9 .. contents::
10
11 General Library Questions
12 =========================
13
14 How do I find a module or application to perform task X?
15 --------------------------------------------------------
16
17 Check :ref:`the Library Reference <library-index>` to see if there's a relevant
18 standard library module. (Eventually you'll learn what's in the standard
19 library and will be able to skip this step.)
20
21 For third-party packages, search the `Python Package Index
22 <https://pypi.org>`_ or try `Google <https://www.google.com>`_ or
23 another web search engine. Searching for "Python" plus a keyword or two for
24 your topic of interest will usually find something helpful.
25
26
27 Where is the math.py (socket.py, regex.py, etc.) source file?
28 -------------------------------------------------------------
29
30 If you can't find a source file for a module it may be a built-in or
31 dynamically loaded module implemented in C, C++ or other compiled language.
32 In this case you may not have the source file or it may be something like
33 :file:`mathmodule.c`, somewhere in a C source directory (not on the Python Path).
34
35 There are (at least) three kinds of modules in Python:
36
37 1) modules written in Python (.py);
38 2) modules written in C and dynamically loaded (.dll, .pyd, .so, .sl, etc);
39 3) modules written in C and linked with the interpreter; to get a list of these,
40 type::
41
42 import sys
43 print(sys.builtin_module_names)
44
45
46 How do I make a Python script executable on Unix?
47 -------------------------------------------------
48
49 You need to do two things: the script file's mode must be executable and the
50 first line must begin with ``#!`` followed by the path of the Python
51 interpreter.
52
53 The first is done by executing ``chmod +x scriptfile`` or perhaps ``chmod 755
54 scriptfile``.
55
56 The second can be done in a number of ways. The most straightforward way is to
57 write ::
58
59 #!/usr/local/bin/python
60
61 as the very first line of your file, using the pathname for where the Python
62 interpreter is installed on your platform.
63
64 If you would like the script to be independent of where the Python interpreter
65 lives, you can use the :program:`env` program. Almost all Unix variants support
66 the following, assuming the Python interpreter is in a directory on the user's
67 :envvar:`PATH`::
68
69 #!/usr/bin/env python
70
71 *Don't* do this for CGI scripts. The :envvar:`PATH` variable for CGI scripts is
72 often very minimal, so you need to use the actual absolute pathname of the
73 interpreter.
74
75 Occasionally, a user's environment is so full that the :program:`/usr/bin/env`
76 program fails; or there's no env program at all. In that case, you can try the
77 following hack (due to Alex Rezinsky):
78
79 .. code-block:: sh
80
81 #! /bin/sh
82 """:"
83 exec python $0 ${1+"$@"}
84 """
85
86 The minor disadvantage is that this defines the script's __doc__ string.
87 However, you can fix that by adding ::
88
89 __doc__ = """...Whatever..."""
90
91
92
93 Is there a curses/termcap package for Python?
94 ---------------------------------------------
95
96 .. XXX curses *is* built by default, isn't it?
97
98 For Unix variants: The standard Python source distribution comes with a curses
99 module in the :source:`Modules` subdirectory, though it's not compiled by default.
100 (Note that this is not available in the Windows distribution -- there is no
101 curses module for Windows.)
102
103 The :mod:`curses` module supports basic curses features as well as many additional
104 functions from ncurses and SYSV curses such as colour, alternative character set
105 support, pads, and mouse support. This means the module isn't compatible with
106 operating systems that only have BSD curses, but there don't seem to be any
107 currently maintained OSes that fall into this category.
108
109
110 Is there an equivalent to C's onexit() in Python?
111 -------------------------------------------------
112
113 The :mod:`atexit` module provides a register function that is similar to C's
114 :c:func:`!onexit`.
115
116
117 Why don't my signal handlers work?
118 ----------------------------------
119
120 The most common problem is that the signal handler is declared with the wrong
121 argument list. It is called as ::
122
123 handler(signum, frame)
124
125 so it should be declared with two parameters::
126
127 def handler(signum, frame):
128 ...
129
130
131 Common tasks
132 ============
133
134 How do I test a Python program or component?
135 --------------------------------------------
136
137 Python comes with two testing frameworks. The :mod:`doctest` module finds
138 examples in the docstrings for a module and runs them, comparing the output with
139 the expected output given in the docstring.
140
141 The :mod:`unittest` module is a fancier testing framework modelled on Java and
142 Smalltalk testing frameworks.
143
144 To make testing easier, you should use good modular design in your program.
145 Your program should have almost all functionality
146 encapsulated in either functions or class methods -- and this sometimes has the
147 surprising and delightful effect of making the program run faster (because local
148 variable accesses are faster than global accesses). Furthermore the program
149 should avoid depending on mutating global variables, since this makes testing
150 much more difficult to do.
151
152 The "global main logic" of your program may be as simple as ::
153
154 if __name__ == "__main__":
155 main_logic()
156
157 at the bottom of the main module of your program.
158
159 Once your program is organized as a tractable collection of function and class
160 behaviours, you should write test functions that exercise the behaviours. A
161 test suite that automates a sequence of tests can be associated with each module.
162 This sounds like a lot of work, but since Python is so terse and flexible it's
163 surprisingly easy. You can make coding much more pleasant and fun by writing
164 your test functions in parallel with the "production code", since this makes it
165 easy to find bugs and even design flaws earlier.
166
167 "Support modules" that are not intended to be the main module of a program may
168 include a self-test of the module. ::
169
170 if __name__ == "__main__":
171 self_test()
172
173 Even programs that interact with complex external interfaces may be tested when
174 the external interfaces are unavailable by using "fake" interfaces implemented
175 in Python.
176
177
178 How do I create documentation from doc strings?
179 -----------------------------------------------
180
181 The :mod:`pydoc` module can create HTML from the doc strings in your Python
182 source code. An alternative for creating API documentation purely from
183 docstrings is `epydoc <https://epydoc.sourceforge.net/>`_. `Sphinx
184 <https://www.sphinx-doc.org>`_ can also include docstring content.
185
186
187 How do I get a single keypress at a time?
188 -----------------------------------------
189
190 For Unix variants there are several solutions. It's straightforward to do this
191 using curses, but curses is a fairly large module to learn.
192
193 .. XXX this doesn't work out of the box, some IO expert needs to check why
194
195 Here's a solution without curses::
196
197 import termios, fcntl, sys, os
198 fd = sys.stdin.fileno()
199
200 oldterm = termios.tcgetattr(fd)
201 newattr = termios.tcgetattr(fd)
202 newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO
203 termios.tcsetattr(fd, termios.TCSANOW, newattr)
204
205 oldflags = fcntl.fcntl(fd, fcntl.F_GETFL)
206 fcntl.fcntl(fd, fcntl.F_SETFL, oldflags | os.O_NONBLOCK)
207
208 try:
209 while True:
210 try:
211 c = sys.stdin.read(1)
212 print("Got character", repr(c))
213 except OSError:
214 pass
215 finally:
216 termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm)
217 fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)
218
219 You need the :mod:`termios` and the :mod:`fcntl` module for any of this to
220 work, and I've only tried it on Linux, though it should work elsewhere. In
221 this code, characters are read and printed one at a time.
222
223 :func:`termios.tcsetattr` turns off stdin's echoing and disables canonical
224 mode. :func:`fcntl.fnctl` is used to obtain stdin's file descriptor flags
225 and modify them for non-blocking mode. Since reading stdin when it is empty
226 results in an :exc:`OSError`, this error is caught and ignored.
227
228 .. versionchanged:: 3.3
229 *sys.stdin.read* used to raise :exc:`IOError`. Starting from Python 3.3
230 :exc:`IOError` is alias for :exc:`OSError`.
231
232
233 Threads
234 =======
235
236 How do I program using threads?
237 -------------------------------
238
239 Be sure to use the :mod:`threading` module and not the :mod:`_thread` module.
240 The :mod:`threading` module builds convenient abstractions on top of the
241 low-level primitives provided by the :mod:`_thread` module.
242
243
244 None of my threads seem to run: why?
245 ------------------------------------
246
247 As soon as the main thread exits, all threads are killed. Your main thread is
248 running too quickly, giving the threads no time to do any work.
249
250 A simple fix is to add a sleep to the end of the program that's long enough for
251 all the threads to finish::
252
253 import threading, time
254
255 def thread_task(name, n):
256 for i in range(n):
257 print(name, i)
258
259 for i in range(10):
260 T = threading.Thread(target=thread_task, args=(str(i), i))
261 T.start()
262
263 time.sleep(10) # <---------------------------!
264
265 But now (on many platforms) the threads don't run in parallel, but appear to run
266 sequentially, one at a time! The reason is that the OS thread scheduler doesn't
267 start a new thread until the previous thread is blocked.
268
269 A simple fix is to add a tiny sleep to the start of the run function::
270
271 def thread_task(name, n):
272 time.sleep(0.001) # <--------------------!
273 for i in range(n):
274 print(name, i)
275
276 for i in range(10):
277 T = threading.Thread(target=thread_task, args=(str(i), i))
278 T.start()
279
280 time.sleep(10)
281
282 Instead of trying to guess a good delay value for :func:`time.sleep`,
283 it's better to use some kind of semaphore mechanism. One idea is to use the
284 :mod:`queue` module to create a queue object, let each thread append a token to
285 the queue when it finishes, and let the main thread read as many tokens from the
286 queue as there are threads.
287
288
289 How do I parcel out work among a bunch of worker threads?
290 ---------------------------------------------------------
291
292 The easiest way is to use the :mod:`concurrent.futures` module,
293 especially the :mod:`~concurrent.futures.ThreadPoolExecutor` class.
294
295 Or, if you want fine control over the dispatching algorithm, you can write
296 your own logic manually. Use the :mod:`queue` module to create a queue
297 containing a list of jobs. The :class:`~queue.Queue` class maintains a
298 list of objects and has a ``.put(obj)`` method that adds items to the queue and
299 a ``.get()`` method to return them. The class will take care of the locking
300 necessary to ensure that each job is handed out exactly once.
301
302 Here's a trivial example::
303
304 import threading, queue, time
305
306 # The worker thread gets jobs off the queue. When the queue is empty, it
307 # assumes there will be no more work and exits.
308 # (Realistically workers will run until terminated.)
309 def worker():
310 print('Running worker')
311 time.sleep(0.1)
312 while True:
313 try:
314 arg = q.get(block=False)
315 except queue.Empty:
316 print('Worker', threading.current_thread(), end=' ')
317 print('queue empty')
318 break
319 else:
320 print('Worker', threading.current_thread(), end=' ')
321 print('running with argument', arg)
322 time.sleep(0.5)
323
324 # Create queue
325 q = queue.Queue()
326
327 # Start a pool of 5 workers
328 for i in range(5):
329 t = threading.Thread(target=worker, name='worker %i' % (i+1))
330 t.start()
331
332 # Begin adding work to the queue
333 for i in range(50):
334 q.put(i)
335
336 # Give threads time to run
337 print('Main thread sleeping')
338 time.sleep(5)
339
340 When run, this will produce the following output:
341
342 .. code-block:: none
343
344 Running worker
345 Running worker
346 Running worker
347 Running worker
348 Running worker
349 Main thread sleeping
350 Worker <Thread(worker 1, started 130283832797456)> running with argument 0
351 Worker <Thread(worker 2, started 130283824404752)> running with argument 1
352 Worker <Thread(worker 3, started 130283816012048)> running with argument 2
353 Worker <Thread(worker 4, started 130283807619344)> running with argument 3
354 Worker <Thread(worker 5, started 130283799226640)> running with argument 4
355 Worker <Thread(worker 1, started 130283832797456)> running with argument 5
356 ...
357
358 Consult the module's documentation for more details; the :class:`~queue.Queue`
359 class provides a featureful interface.
360
361
362 What kinds of global value mutation are thread-safe?
363 ----------------------------------------------------
364
365 A :term:`global interpreter lock` (GIL) is used internally to ensure that only one
366 thread runs in the Python VM at a time. In general, Python offers to switch
367 among threads only between bytecode instructions; how frequently it switches can
368 be set via :func:`sys.setswitchinterval`. Each bytecode instruction and
369 therefore all the C implementation code reached from each instruction is
370 therefore atomic from the point of view of a Python program.
371
372 In theory, this means an exact accounting requires an exact understanding of the
373 PVM bytecode implementation. In practice, it means that operations on shared
374 variables of built-in data types (ints, lists, dicts, etc) that "look atomic"
375 really are.
376
377 For example, the following operations are all atomic (L, L1, L2 are lists, D,
378 D1, D2 are dicts, x, y are objects, i, j are ints)::
379
380 L.append(x)
381 L1.extend(L2)
382 x = L[i]
383 x = L.pop()
384 L1[i:j] = L2
385 L.sort()
386 x = y
387 x.field = y
388 D[x] = y
389 D1.update(D2)
390 D.keys()
391
392 These aren't::
393
394 i = i+1
395 L.append(L[-1])
396 L[i] = L[j]
397 D[x] = D[x] + 1
398
399 Operations that replace other objects may invoke those other objects'
400 :meth:`~object.__del__` method when their reference count reaches zero, and that can
401 affect things. This is especially true for the mass updates to dictionaries and
402 lists. When in doubt, use a mutex!
403
404
405 Can't we get rid of the Global Interpreter Lock?
406 ------------------------------------------------
407
408 The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's
409 deployment on high-end multiprocessor server machines, because a multi-threaded
410 Python program effectively only uses one CPU, due to the insistence that
411 (almost) all Python code can only run while the GIL is held.
412
413 With the approval of :pep:`703` work is now underway to remove the GIL from the
414 CPython implementation of Python. Initially it will be implemented as an
415 optional compiler flag when building the interpreter, and so separate
416 builds will be available with and without the GIL. Long-term, the hope is
417 to settle on a single build, once the performance implications of removing the
418 GIL are fully understood. Python 3.13 is likely to be the first release
419 containing this work, although it may not be completely functional in this
420 release.
421
422 The current work to remove the GIL is based on a
423 `fork of Python 3.9 with the GIL removed <https://github.com/colesbury/nogil>`_
424 by Sam Gross.
425 Prior to that,
426 in the days of Python 1.5, Greg Stein actually implemented a comprehensive
427 patch set (the "free threading" patches) that removed the GIL and replaced it
428 with fine-grained locking. Adam Olsen did a similar experiment
429 in his `python-safethread <https://code.google.com/archive/p/python-safethread>`_
430 project. Unfortunately, both of these earlier experiments exhibited a sharp
431 drop in single-thread
432 performance (at least 30% slower), due to the amount of fine-grained locking
433 necessary to compensate for the removal of the GIL. The Python 3.9 fork
434 is the first attempt at removing the GIL with an acceptable performance
435 impact.
436
437 The presence of the GIL in current Python releases
438 doesn't mean that you can't make good use of Python on multi-CPU machines!
439 You just have to be creative with dividing the work up between multiple
440 *processes* rather than multiple *threads*. The
441 :class:`~concurrent.futures.ProcessPoolExecutor` class in the new
442 :mod:`concurrent.futures` module provides an easy way of doing so; the
443 :mod:`multiprocessing` module provides a lower-level API in case you want
444 more control over dispatching of tasks.
445
446 Judicious use of C extensions will also help; if you use a C extension to
447 perform a time-consuming task, the extension can release the GIL while the
448 thread of execution is in the C code and allow other threads to get some work
449 done. Some standard library modules such as :mod:`zlib` and :mod:`hashlib`
450 already do this.
451
452 An alternative approach to reducing the impact of the GIL is
453 to make the GIL a per-interpreter-state lock rather than truly global.
454 This was :ref:`first implemented in Python 3.12 <whatsnew312-pep684>` and is
455 available in the C API. A Python interface to it is expected in Python 3.13.
456 The main limitation to it at the moment is likely to be 3rd party extension
457 modules, since these must be written with multiple interpreters in mind in
458 order to be usable, so many older extension modules will not be usable.
459
460
461 Input and Output
462 ================
463
464 How do I delete a file? (And other file questions...)
465 -----------------------------------------------------
466
467 Use ``os.remove(filename)`` or ``os.unlink(filename)``; for documentation, see
468 the :mod:`os` module. The two functions are identical; :func:`~os.unlink` is simply
469 the name of the Unix system call for this function.
470
471 To remove a directory, use :func:`os.rmdir`; use :func:`os.mkdir` to create one.
472 ``os.makedirs(path)`` will create any intermediate directories in ``path`` that
473 don't exist. ``os.removedirs(path)`` will remove intermediate directories as
474 long as they're empty; if you want to delete an entire directory tree and its
475 contents, use :func:`shutil.rmtree`.
476
477 To rename a file, use ``os.rename(old_path, new_path)``.
478
479 To truncate a file, open it using ``f = open(filename, "rb+")``, and use
480 ``f.truncate(offset)``; offset defaults to the current seek position. There's
481 also ``os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where
482 *fd* is the file descriptor (a small integer).
483
484 The :mod:`shutil` module also contains a number of functions to work on files
485 including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and
486 :func:`~shutil.rmtree`.
487
488
489 How do I copy a file?
490 ---------------------
491
492 The :mod:`shutil` module contains a :func:`~shutil.copyfile` function.
493 Note that on Windows NTFS volumes, it does not copy
494 `alternate data streams
495 <https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(ADS)>`_
496 nor `resource forks <https://en.wikipedia.org/wiki/Resource_fork>`__
497 on macOS HFS+ volumes, though both are now rarely used.
498 It also doesn't copy file permissions and metadata, though using
499 :func:`shutil.copy2` instead will preserve most (though not all) of it.
500
501
502 How do I read (or write) binary data?
503 -------------------------------------
504
505 To read or write complex binary data formats, it's best to use the :mod:`struct`
506 module. It allows you to take a string containing binary data (usually numbers)
507 and convert it to Python objects; and vice versa.
508
509 For example, the following code reads two 2-byte integers and one 4-byte integer
510 in big-endian format from a file::
511
512 import struct
513
514 with open(filename, "rb") as f:
515 s = f.read(8)
516 x, y, z = struct.unpack(">hhl", s)
517
518 The '>' in the format string forces big-endian data; the letter 'h' reads one
519 "short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the
520 string.
521
522 For data that is more regular (e.g. a homogeneous list of ints or floats),
523 you can also use the :mod:`array` module.
524
525 .. note::
526
527 To read and write binary data, it is mandatory to open the file in
528 binary mode (here, passing ``"rb"`` to :func:`open`). If you use
529 ``"r"`` instead (the default), the file will be open in text mode
530 and ``f.read()`` will return :class:`str` objects rather than
531 :class:`bytes` objects.
532
533
534 I can't seem to use os.read() on a pipe created with os.popen(); why?
535 ---------------------------------------------------------------------
536
537 :func:`os.read` is a low-level function which takes a file descriptor, a small
538 integer representing the opened file. :func:`os.popen` creates a high-level
539 file object, the same type returned by the built-in :func:`open` function.
540 Thus, to read *n* bytes from a pipe *p* created with :func:`os.popen`, you need to
541 use ``p.read(n)``.
542
543
544 .. XXX update to use subprocess. See the :ref:`subprocess-replacements` section.
545
546 How do I run a subprocess with pipes connected to both input and output?
547 ------------------------------------------------------------------------
548
549 Use the :mod:`popen2` module. For example::
550
551 import popen2
552 fromchild, tochild = popen2.popen2("command")
553 tochild.write("input\n")
554 tochild.flush()
555 output = fromchild.readline()
556
557 Warning: in general it is unwise to do this because you can easily cause a
558 deadlock where your process is blocked waiting for output from the child
559 while the child is blocked waiting for input from you. This can be caused
560 by the parent expecting the child to output more text than it does or
561 by data being stuck in stdio buffers due to lack of flushing.
562 The Python parent can of course explicitly flush the data it sends to the
563 child before it reads any output, but if the child is a naive C program it
564 may have been written to never explicitly flush its output, even if it is
565 interactive, since flushing is normally automatic.
566
567 Note that a deadlock is also possible if you use :func:`popen3` to read
568 stdout and stderr. If one of the two is too large for the internal buffer
569 (increasing the buffer size does not help) and you ``read()`` the other one
570 first, there is a deadlock, too.
571
572 Note on a bug in popen2: unless your program calls ``wait()`` or
573 ``waitpid()``, finished child processes are never removed, and eventually
574 calls to popen2 will fail because of a limit on the number of child
575 processes. Calling :func:`os.waitpid` with the :const:`os.WNOHANG` option can
576 prevent this; a good place to insert such a call would be before calling
577 ``popen2`` again.
578
579 In many cases, all you really need is to run some data through a command and
580 get the result back. Unless the amount of data is very large, the easiest
581 way to do this is to write it to a temporary file and run the command with
582 that temporary file as input. The standard module :mod:`tempfile` exports a
583 :func:`~tempfile.mktemp` function to generate unique temporary file names. ::
584
585 import tempfile
586 import os
587
588 class Popen3:
589 """
590 This is a deadlock-safe version of popen that returns
591 an object with errorlevel, out (a string) and err (a string).
592 (capturestderr may not work under windows.)
593 Example: print(Popen3('grep spam','\n\nhere spam\n\n').out)
594 """
595 def __init__(self,command,input=None,capturestderr=None):
596 outfile=tempfile.mktemp()
597 command="( %s ) > %s" % (command,outfile)
598 if input:
599 infile=tempfile.mktemp()
600 open(infile,"w").write(input)
601 command=command+" <"+infile
602 if capturestderr:
603 errfile=tempfile.mktemp()
604 command=command+" 2>"+errfile
605 self.errorlevel=os.system(command) >> 8
606 self.out=open(outfile,"r").read()
607 os.remove(outfile)
608 if input:
609 os.remove(infile)
610 if capturestderr:
611 self.err=open(errfile,"r").read()
612 os.remove(errfile)
613
614 Note that many interactive programs (e.g. vi) don't work well with pipes
615 substituted for standard input and output. You will have to use pseudo ttys
616 ("ptys") instead of pipes. Or you can use a Python interface to Don Libes'
617 "expect" library. A Python extension that interfaces to expect is called
618 "expy" and available from https://expectpy.sourceforge.net. A pure Python
619 solution that works like expect is :pypi:`pexpect`.
620
621
622 How do I access the serial (RS232) port?
623 ----------------------------------------
624
625 For Win32, OSX, Linux, BSD, Jython, IronPython:
626
627 :pypi:`pyserial`
628
629 For Unix, see a Usenet post by Mitch Chapman:
630
631 https://groups.google.com/[email protected]
632
633
634 Why doesn't closing sys.stdout (stdin, stderr) really close it?
635 ---------------------------------------------------------------
636
637 Python :term:`file objects <file object>` are a high-level layer of
638 abstraction on low-level C file descriptors.
639
640 For most file objects you create in Python via the built-in :func:`open`
641 function, ``f.close()`` marks the Python file object as being closed from
642 Python's point of view, and also arranges to close the underlying C file
643 descriptor. This also happens automatically in ``f``'s destructor, when
644 ``f`` becomes garbage.
645
646 But stdin, stdout and stderr are treated specially by Python, because of the
647 special status also given to them by C. Running ``sys.stdout.close()`` marks
648 the Python-level file object as being closed, but does *not* close the
649 associated C file descriptor.
650
651 To close the underlying C file descriptor for one of these three, you should
652 first be sure that's what you really want to do (e.g., you may confuse
653 extension modules trying to do I/O). If it is, use :func:`os.close`::
654
655 os.close(stdin.fileno())
656 os.close(stdout.fileno())
657 os.close(stderr.fileno())
658
659 Or you can use the numeric constants 0, 1 and 2, respectively.
660
661
662 Network/Internet Programming
663 ============================
664
665 What WWW tools are there for Python?
666 ------------------------------------
667
668 See the chapters titled :ref:`internet` and :ref:`netdata` in the Library
669 Reference Manual. Python has many modules that will help you build server-side
670 and client-side web systems.
671
672 .. XXX check if wiki page is still up to date
673
674 A summary of available frameworks is maintained by Paul Boddie at
675 https://wiki.python.org/moin/WebProgramming\ .
676
677
678 What module should I use to help with generating HTML?
679 ------------------------------------------------------
680
681 .. XXX add modern template languages
682
683 You can find a collection of useful links on the `Web Programming wiki page
684 <https://wiki.python.org/moin/WebProgramming>`_.
685
686
687 How do I send mail from a Python script?
688 ----------------------------------------
689
690 Use the standard library module :mod:`smtplib`.
691
692 Here's a very simple interactive mail sender that uses it. This method will
693 work on any host that supports an SMTP listener. ::
694
695 import sys, smtplib
696
697 fromaddr = input("From: ")
698 toaddrs = input("To: ").split(',')
699 print("Enter message, end with ^D:")
700 msg = ''
701 while True:
702 line = sys.stdin.readline()
703 if not line:
704 break
705 msg += line
706
707 # The actual mail send
708 server = smtplib.SMTP('localhost')
709 server.sendmail(fromaddr, toaddrs, msg)
710 server.quit()
711
712 A Unix-only alternative uses sendmail. The location of the sendmail program
713 varies between systems; sometimes it is ``/usr/lib/sendmail``, sometimes
714 ``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's
715 some sample code::
716
717 import os
718
719 SENDMAIL = "/usr/sbin/sendmail" # sendmail location
720 p = os.popen("%s -t -i" % SENDMAIL, "w")
721 p.write("To: [email protected]\n")
722 p.write("Subject: test\n")
723 p.write("\n") # blank line separating headers from body
724 p.write("Some text\n")
725 p.write("some more text\n")
726 sts = p.close()
727 if sts != 0:
728 print("Sendmail exit status", sts)
729
730
731 How do I avoid blocking in the connect() method of a socket?
732 ------------------------------------------------------------
733
734 The :mod:`select` module is commonly used to help with asynchronous I/O on
735 sockets.
736
737 To prevent the TCP connect from blocking, you can set the socket to non-blocking
738 mode. Then when you do the :meth:`~socket.socket.connect`,
739 you will either connect immediately
740 (unlikely) or get an exception that contains the error number as ``.errno``.
741 ``errno.EINPROGRESS`` indicates that the connection is in progress, but hasn't
742 finished yet. Different OSes will return different values, so you're going to
743 have to check what's returned on your system.
744
745 You can use the :meth:`~socket.socket.connect_ex` method
746 to avoid creating an exception.
747 It will just return the errno value.
748 To poll, you can call :meth:`~socket.socket.connect_ex` again later
749 -- ``0`` or ``errno.EISCONN`` indicate that you're connected -- or you can pass this
750 socket to :meth:`select.select` to check if it's writable.
751
752 .. note::
753 The :mod:`asyncio` module provides a general purpose single-threaded and
754 concurrent asynchronous library, which can be used for writing non-blocking
755 network code.
756 The third-party `Twisted <https://twisted.org/>`_ library is
757 a popular and feature-rich alternative.
758
759
760 Databases
761 =========
762
763 Are there any interfaces to database packages in Python?
764 --------------------------------------------------------
765
766 Yes.
767
768 Interfaces to disk-based hashes such as :mod:`DBM <dbm.ndbm>` and :mod:`GDBM
769 <dbm.gnu>` are also included with standard Python. There is also the
770 :mod:`sqlite3` module, which provides a lightweight disk-based relational
771 database.
772
773 Support for most relational databases is available. See the
774 `DatabaseProgramming wiki page
775 <https://wiki.python.org/moin/DatabaseProgramming>`_ for details.
776
777
778 How do you implement persistent objects in Python?
779 --------------------------------------------------
780
781 The :mod:`pickle` library module solves this in a very general way (though you
782 still can't store things like open files, sockets or windows), and the
783 :mod:`shelve` library module uses pickle and (g)dbm to create persistent
784 mappings containing arbitrary Python objects.
785
786
787 Mathematics and Numerics
788 ========================
789
790 How do I generate random numbers in Python?
791 -------------------------------------------
792
793 The standard module :mod:`random` implements a random number generator. Usage
794 is simple::
795
796 import random
797 random.random()
798
799 This returns a random floating-point number in the range [0, 1).
800
801 There are also many other specialized generators in this module, such as:
802
803 * ``randrange(a, b)`` chooses an integer in the range [a, b).
804 * ``uniform(a, b)`` chooses a floating-point number in the range [a, b).
805 * ``normalvariate(mean, sdev)`` samples the normal (Gaussian) distribution.
806
807 Some higher-level functions operate on sequences directly, such as:
808
809 * ``choice(S)`` chooses a random element from a given sequence.
810 * ``shuffle(L)`` shuffles a list in-place, i.e. permutes it randomly.
811
812 There's also a ``Random`` class you can instantiate to create independent
813 multiple random number generators.