Debian Codegraph @

Doc/tutorial/floatingpoint.rst

raw
1 .. testsetup::
2
3 import math
4 from fractions import Fraction
5
6 .. _tut-fp-issues:
7
8 **************************************************
9 Floating-Point Arithmetic: Issues and Limitations
10 **************************************************
11
12 .. sectionauthor:: Tim Peters <[email protected]>
13 .. sectionauthor:: Raymond Hettinger <python at rcn dot com>
14
15
16 Floating-point numbers are represented in computer hardware as base 2 (binary)
17 fractions. For example, the **decimal** fraction ``0.625``
18 has value 6/10 + 2/100 + 5/1000, and in the same way the **binary** fraction ``0.101``
19 has value 1/2 + 0/4 + 1/8. These two fractions have identical values, the only
20 real difference being that the first is written in base 10 fractional notation,
21 and the second in base 2.
22
23 Unfortunately, most decimal fractions cannot be represented exactly as binary
24 fractions. A consequence is that, in general, the decimal floating-point
25 numbers you enter are only approximated by the binary floating-point numbers
26 actually stored in the machine.
27
28 The problem is easier to understand at first in base 10. Consider the fraction
29 1/3. You can approximate that as a base 10 fraction::
30
31 0.3
32
33 or, better, ::
34
35 0.33
36
37 or, better, ::
38
39 0.333
40
41 and so on. No matter how many digits you're willing to write down, the result
42 will never be exactly 1/3, but will be an increasingly better approximation of
43 1/3.
44
45 In the same way, no matter how many base 2 digits you're willing to use, the
46 decimal value 0.1 cannot be represented exactly as a base 2 fraction. In base
47 2, 1/10 is the infinitely repeating fraction ::
48
49 0.0001100110011001100110011001100110011001100110011...
50
51 Stop at any finite number of bits, and you get an approximation. On most
52 machines today, floats are approximated using a binary fraction with
53 the numerator using the first 53 bits starting with the most significant bit and
54 with the denominator as a power of two. In the case of 1/10, the binary fraction
55 is ``3602879701896397 / 2 ** 55`` which is close to but not exactly
56 equal to the true value of 1/10.
57
58 Many users are not aware of the approximation because of the way values are
59 displayed. Python only prints a decimal approximation to the true decimal
60 value of the binary approximation stored by the machine. On most machines, if
61 Python were to print the true decimal value of the binary approximation stored
62 for 0.1, it would have to display::
63
64 >>> 0.1
65 0.1000000000000000055511151231257827021181583404541015625
66
67 That is more digits than most people find useful, so Python keeps the number
68 of digits manageable by displaying a rounded value instead:
69
70 .. doctest::
71
72 >>> 1 / 10
73 0.1
74
75 Just remember, even though the printed result looks like the exact value
76 of 1/10, the actual stored value is the nearest representable binary fraction.
77
78 Interestingly, there are many different decimal numbers that share the same
79 nearest approximate binary fraction. For example, the numbers ``0.1`` and
80 ``0.10000000000000001`` and
81 ``0.1000000000000000055511151231257827021181583404541015625`` are all
82 approximated by ``3602879701896397 / 2 ** 55``. Since all of these decimal
83 values share the same approximation, any one of them could be displayed
84 while still preserving the invariant ``eval(repr(x)) == x``.
85
86 Historically, the Python prompt and built-in :func:`repr` function would choose
87 the one with 17 significant digits, ``0.10000000000000001``. Starting with
88 Python 3.1, Python (on most systems) is now able to choose the shortest of
89 these and simply display ``0.1``.
90
91 Note that this is in the very nature of binary floating point: this is not a bug
92 in Python, and it is not a bug in your code either. You'll see the same kind of
93 thing in all languages that support your hardware's floating-point arithmetic
94 (although some languages may not *display* the difference by default, or in all
95 output modes).
96
97 For more pleasant output, you may wish to use string formatting to produce a
98 limited number of significant digits:
99
100 .. doctest::
101
102 >>> format(math.pi, '.12g') # give 12 significant digits
103 '3.14159265359'
104
105 >>> format(math.pi, '.2f') # give 2 digits after the point
106 '3.14'
107
108 >>> repr(math.pi)
109 '3.141592653589793'
110
111 It's important to realize that this is, in a real sense, an illusion: you're
112 simply rounding the *display* of the true machine value.
113
114 One illusion may beget another. For example, since 0.1 is not exactly 1/10,
115 summing three values of 0.1 may not yield exactly 0.3, either:
116
117 .. doctest::
118
119 >>> 0.1 + 0.1 + 0.1 == 0.3
120 False
121
122 Also, since the 0.1 cannot get any closer to the exact value of 1/10 and
123 0.3 cannot get any closer to the exact value of 3/10, then pre-rounding with
124 :func:`round` function cannot help:
125
126 .. doctest::
127
128 >>> round(0.1, 1) + round(0.1, 1) + round(0.1, 1) == round(0.3, 1)
129 False
130
131 Though the numbers cannot be made closer to their intended exact values,
132 the :func:`math.isclose` function can be useful for comparing inexact values:
133
134 .. doctest::
135
136 >>> math.isclose(0.1 + 0.1 + 0.1, 0.3)
137 True
138
139 Alternatively, the :func:`round` function can be used to compare rough
140 approximations:
141
142 .. doctest::
143
144 >>> round(math.pi, ndigits=2) == round(22 / 7, ndigits=2)
145 True
146
147 Binary floating-point arithmetic holds many surprises like this. The problem
148 with "0.1" is explained in precise detail below, in the "Representation Error"
149 section. See `Examples of Floating Point Problems
150 <https://jvns.ca/blog/2023/01/13/examples-of-floating-point-problems/>`_ for
151 a pleasant summary of how binary floating point works and the kinds of
152 problems commonly encountered in practice. Also see
153 `The Perils of Floating Point <http://www.indowsway.com/floatingpoint.htm>`_
154 for a more complete account of other common surprises.
155
156 As that says near the end, "there are no easy answers." Still, don't be unduly
157 wary of floating point! The errors in Python float operations are inherited
158 from the floating-point hardware, and on most machines are on the order of no
159 more than 1 part in 2\*\*53 per operation. That's more than adequate for most
160 tasks, but you do need to keep in mind that it's not decimal arithmetic and
161 that every float operation can suffer a new rounding error.
162
163 While pathological cases do exist, for most casual use of floating-point
164 arithmetic you'll see the result you expect in the end if you simply round the
165 display of your final results to the number of decimal digits you expect.
166 :func:`str` usually suffices, and for finer control see the :meth:`str.format`
167 method's format specifiers in :ref:`formatstrings`.
168
169 For use cases which require exact decimal representation, try using the
170 :mod:`decimal` module which implements decimal arithmetic suitable for
171 accounting applications and high-precision applications.
172
173 Another form of exact arithmetic is supported by the :mod:`fractions` module
174 which implements arithmetic based on rational numbers (so the numbers like
175 1/3 can be represented exactly).
176
177 If you are a heavy user of floating-point operations you should take a look
178 at the NumPy package and many other packages for mathematical and
179 statistical operations supplied by the SciPy project. See <https://scipy.org>.
180
181 Python provides tools that may help on those rare occasions when you really
182 *do* want to know the exact value of a float. The
183 :meth:`float.as_integer_ratio` method expresses the value of a float as a
184 fraction:
185
186 .. doctest::
187
188 >>> x = 3.14159
189 >>> x.as_integer_ratio()
190 (3537115888337719, 1125899906842624)
191
192 Since the ratio is exact, it can be used to losslessly recreate the
193 original value:
194
195 .. doctest::
196
197 >>> x == 3537115888337719 / 1125899906842624
198 True
199
200 The :meth:`float.hex` method expresses a float in hexadecimal (base
201 16), again giving the exact value stored by your computer:
202
203 .. doctest::
204
205 >>> x.hex()
206 '0x1.921f9f01b866ep+1'
207
208 This precise hexadecimal representation can be used to reconstruct
209 the float value exactly:
210
211 .. doctest::
212
213 >>> x == float.fromhex('0x1.921f9f01b866ep+1')
214 True
215
216 Since the representation is exact, it is useful for reliably porting values
217 across different versions of Python (platform independence) and exchanging
218 data with other languages that support the same format (such as Java and C99).
219
220 Another helpful tool is the :func:`sum` function which helps mitigate
221 loss-of-precision during summation. It uses extended precision for
222 intermediate rounding steps as values are added onto a running total.
223 That can make a difference in overall accuracy so that the errors do not
224 accumulate to the point where they affect the final total:
225
226 .. doctest::
227
228 >>> 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 == 1.0
229 False
230 >>> sum([0.1] * 10) == 1.0
231 True
232
233 The :func:`math.fsum` goes further and tracks all of the "lost digits"
234 as values are added onto a running total so that the result has only a
235 single rounding. This is slower than :func:`sum` but will be more
236 accurate in uncommon cases where large magnitude inputs mostly cancel
237 each other out leaving a final sum near zero:
238
239 .. doctest::
240
241 >>> arr = [-0.10430216751806065, -266310978.67179024, 143401161448607.16,
242 ... -143401161400469.7, 266262841.31058735, -0.003244936839808227]
243 >>> float(sum(map(Fraction, arr))) # Exact summation with single rounding
244 8.042173697819788e-13
245 >>> math.fsum(arr) # Single rounding
246 8.042173697819788e-13
247 >>> sum(arr) # Multiple roundings in extended precision
248 8.042178034628478e-13
249 >>> total = 0.0
250 >>> for x in arr:
251 ... total += x # Multiple roundings in standard precision
252 ...
253 >>> total # Straight addition has no correct digits!
254 -0.0051575902860057365
255
256
257 .. _tut-fp-error:
258
259 Representation Error
260 ====================
261
262 This section explains the "0.1" example in detail, and shows how you can perform
263 an exact analysis of cases like this yourself. Basic familiarity with binary
264 floating-point representation is assumed.
265
266 :dfn:`Representation error` refers to the fact that some (most, actually)
267 decimal fractions cannot be represented exactly as binary (base 2) fractions.
268 This is the chief reason why Python (or Perl, C, C++, Java, Fortran, and many
269 others) often won't display the exact decimal number you expect.
270
271 Why is that? 1/10 is not exactly representable as a binary fraction. Since at
272 least 2000, almost all machines use IEEE 754 binary floating-point arithmetic,
273 and almost all platforms map Python floats to IEEE 754 binary64 "double
274 precision" values. IEEE 754 binary64 values contain 53 bits of precision, so
275 on input the computer strives to convert 0.1 to the closest fraction it can of
276 the form *J*/2**\ *N* where *J* is an integer containing exactly 53 bits.
277 Rewriting
278 ::
279
280 1 / 10 ~= J / (2**N)
281
282 as ::
283
284 J ~= 2**N / 10
285
286 and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
287 the best value for *N* is 56:
288
289 .. doctest::
290
291 >>> 2**52 <= 2**56 // 10 < 2**53
292 True
293
294 That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The
295 best possible value for *J* is then that quotient rounded:
296
297 .. doctest::
298
299 >>> q, r = divmod(2**56, 10)
300 >>> r
301 6
302
303 Since the remainder is more than half of 10, the best approximation is obtained
304 by rounding up:
305
306 .. doctest::
307
308
309
310 >>> q+1
311 7205759403792794
312
313 Therefore the best possible approximation to 1/10 in IEEE 754 double precision
314 is::
315
316 7205759403792794 / 2 ** 56
317
318 Dividing both the numerator and denominator by two reduces the fraction to::
319
320 3602879701896397 / 2 ** 55
321
322 Note that since we rounded up, this is actually a little bit larger than 1/10;
323 if we had not rounded up, the quotient would have been a little bit smaller than
324 1/10. But in no case can it be *exactly* 1/10!
325
326 So the computer never "sees" 1/10: what it sees is the exact fraction given
327 above, the best IEEE 754 double approximation it can get:
328
329 .. doctest::
330
331 >>> 0.1 * 2 ** 55
332 3602879701896397.0
333
334 If we multiply that fraction by 10\*\*55, we can see the value out to
335 55 decimal digits:
336
337 .. doctest::
338
339 >>> 3602879701896397 * 10 ** 55 // 2 ** 55
340 1000000000000000055511151231257827021181583404541015625
341
342 meaning that the exact number stored in the computer is equal to
343 the decimal value 0.1000000000000000055511151231257827021181583404541015625.
344 Instead of displaying the full decimal value, many languages (including
345 older versions of Python), round the result to 17 significant digits:
346
347 .. doctest::
348
349 >>> format(0.1, '.17f')
350 '0.10000000000000001'
351
352 The :mod:`fractions` and :mod:`decimal` modules make these calculations
353 easy:
354
355 .. doctest::
356
357 >>> from decimal import Decimal
358 >>> from fractions import Fraction
359
360 >>> Fraction.from_float(0.1)
361 Fraction(3602879701896397, 36028797018963968)
362
363 >>> (0.1).as_integer_ratio()
364 (3602879701896397, 36028797018963968)
365
366 >>> Decimal.from_float(0.1)
367 Decimal('0.1000000000000000055511151231257827021181583404541015625')
368
369 >>> format(Decimal.from_float(0.1), '.17')
370 '0.10000000000000001'