/Doc/library/timeit.rst

  1
  2:mod:`timeit` --- Measure execution time of small code snippets
  3===============================================================
  4
  5.. module:: timeit
  6   :synopsis: Measure the execution time of small code snippets.
  7
  8
  9.. versionadded:: 2.3
 10
 11.. index::
 12   single: Benchmarking
 13   single: Performance
 14
 15This module provides a simple way to time small bits of Python code. It has both
 16command line as well as callable interfaces.  It avoids a number of common traps
 17for measuring execution times.  See also Tim Peters' introduction to the
 18"Algorithms" chapter in the Python Cookbook, published by O'Reilly.
 19
 20The module defines the following public class:
 21
 22
 23.. class:: Timer([stmt='pass' [, setup='pass' [, timer=<timer function>]]])
 24
 25   Class for timing execution speed of small code snippets.
 26
 27   The constructor takes a statement to be timed, an additional statement used for
 28   setup, and a timer function.  Both statements default to ``'pass'``; the timer
 29   function is platform-dependent (see the module doc string).  The statements may
 30   contain newlines, as long as they don't contain multi-line string literals.
 31
 32   To measure the execution time of the first statement, use the :meth:`timeit`
 33   method.  The :meth:`repeat` method is a convenience to call :meth:`timeit`
 34   multiple times and return a list of results.
 35
 36   .. versionchanged:: 2.6
 37      The *stmt* and *setup* parameters can now also take objects that are callable
 38      without arguments. This will embed calls to them in a timer function that will
 39      then be executed by :meth:`timeit`.  Note that the timing overhead is a little
 40      larger in this case because of the extra function calls.
 41
 42
 43.. method:: Timer.print_exc([file=None])
 44
 45   Helper to print a traceback from the timed code.
 46
 47   Typical use::
 48
 49      t = Timer(...)       # outside the try/except
 50      try:
 51          t.timeit(...)    # or t.repeat(...)
 52      except:
 53          t.print_exc()
 54
 55   The advantage over the standard traceback is that source lines in the compiled
 56   template will be displayed. The optional *file* argument directs where the
 57   traceback is sent; it defaults to ``sys.stderr``.
 58
 59
 60.. method:: Timer.repeat([repeat=3 [, number=1000000]])
 61
 62   Call :meth:`timeit` a few times.
 63
 64   This is a convenience function that calls the :meth:`timeit` repeatedly,
 65   returning a list of results.  The first argument specifies how many times to
 66   call :meth:`timeit`.  The second argument specifies the *number* argument for
 67   :func:`timeit`.
 68
 69   .. note::
 70
 71      It's tempting to calculate mean and standard deviation from the result vector
 72      and report these.  However, this is not very useful.  In a typical case, the
 73      lowest value gives a lower bound for how fast your machine can run the given
 74      code snippet; higher values in the result vector are typically not caused by
 75      variability in Python's speed, but by other processes interfering with your
 76      timing accuracy.  So the :func:`min` of the result is probably the only number
 77      you should be interested in.  After that, you should look at the entire vector
 78      and apply common sense rather than statistics.
 79
 80
 81.. method:: Timer.timeit([number=1000000])
 82
 83   Time *number* executions of the main statement. This executes the setup
 84   statement once, and then returns the time it takes to execute the main statement
 85   a number of times, measured in seconds as a float.  The argument is the number
 86   of times through the loop, defaulting to one million.  The main statement, the
 87   setup statement and the timer function to be used are passed to the constructor.
 88
 89   .. note::
 90
 91      By default, :meth:`timeit` temporarily turns off :term:`garbage collection`
 92      during the timing.  The advantage of this approach is that it makes
 93      independent timings more comparable.  This disadvantage is that GC may be
 94      an important component of the performance of the function being measured.
 95      If so, GC can be re-enabled as the first statement in the *setup* string.
 96      For example::
 97
 98         timeit.Timer('for i in xrange(10): oct(i)', 'gc.enable()').timeit()
 99
100Starting with version 2.6, the module also defines two convenience functions:
101
102
103.. function:: repeat(stmt[, setup[, timer[, repeat=3 [, number=1000000]]]])
104
105   Create a :class:`Timer` instance with the given statement, setup code and timer
106   function and run its :meth:`repeat` method with the given repeat count and
107   *number* executions.
108
109   .. versionadded:: 2.6
110
111
112.. function:: timeit(stmt[, setup[, timer[, number=1000000]]])
113
114   Create a :class:`Timer` instance with the given statement, setup code and timer
115   function and run its :meth:`timeit` method with *number* executions.
116
117   .. versionadded:: 2.6
118
119
120Command Line Interface
121----------------------
122
123When called as a program from the command line, the following form is used::
124
125   python -m timeit [-n N] [-r N] [-s S] [-t] [-c] [-h] [statement ...]
126
127where the following options are understood:
128
129-n N/:option:`--number=N`
130   how many times to execute 'statement'
131
132-r N/:option:`--repeat=N`
133   how many times to repeat the timer (default 3)
134
135-s S/:option:`--setup=S`
136   statement to be executed once initially (default ``'pass'``)
137
138-t/:option:`--time`
139   use :func:`time.time` (default on all platforms but Windows)
140
141-c/:option:`--clock`
142   use :func:`time.clock` (default on Windows)
143
144-v/:option:`--verbose`
145   print raw timing results; repeat for more digits precision
146
147-h/:option:`--help`
148   print a short usage message and exit
149
150A multi-line statement may be given by specifying each line as a separate
151statement argument; indented lines are possible by enclosing an argument in
152quotes and using leading spaces.  Multiple :option:`-s` options are treated
153similarly.
154
155If :option:`-n` is not given, a suitable number of loops is calculated by trying
156successive powers of 10 until the total time is at least 0.2 seconds.
157
158The default timer function is platform dependent.  On Windows,
159:func:`time.clock` has microsecond granularity but :func:`time.time`'s
160granularity is 1/60th of a second; on Unix, :func:`time.clock` has 1/100th of a
161second granularity and :func:`time.time` is much more precise.  On either
162platform, the default timer functions measure wall clock time, not the CPU time.
163This means that other processes running on the same computer may interfere with
164the timing.  The best thing to do when accurate timing is necessary is to repeat
165the timing a few times and use the best time.  The :option:`-r` option is good
166for this; the default of 3 repetitions is probably enough in most cases.  On
167Unix, you can use :func:`time.clock` to measure CPU time.
168
169.. note::
170
171   There is a certain baseline overhead associated with executing a pass statement.
172   The code here doesn't try to hide it, but you should be aware of it.  The
173   baseline overhead can be measured by invoking the program without arguments.
174
175The baseline overhead differs between Python versions!  Also, to fairly compare
176older Python versions to Python 2.3, you may want to use Python's :option:`-O`
177option for the older versions to avoid timing ``SET_LINENO`` instructions.
178
179
180Examples
181--------
182
183Here are two example sessions (one using the command line, one using the module
184interface) that compare the cost of using :func:`hasattr` vs.
185:keyword:`try`/:keyword:`except` to test for missing and present object
186attributes. ::
187
188   % timeit.py 'try:' '  str.__nonzero__' 'except AttributeError:' '  pass'
189   100000 loops, best of 3: 15.7 usec per loop
190   % timeit.py 'if hasattr(str, "__nonzero__"): pass'
191   100000 loops, best of 3: 4.26 usec per loop
192   % timeit.py 'try:' '  int.__nonzero__' 'except AttributeError:' '  pass'
193   1000000 loops, best of 3: 1.43 usec per loop
194   % timeit.py 'if hasattr(int, "__nonzero__"): pass'
195   100000 loops, best of 3: 2.23 usec per loop
196
197::
198
199   >>> import timeit
200   >>> s = """\
201   ... try:
202   ...     str.__nonzero__
203   ... except AttributeError:
204   ...     pass
205   ... """
206   >>> t = timeit.Timer(stmt=s)
207   >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)
208   17.09 usec/pass
209   >>> s = """\
210   ... if hasattr(str, '__nonzero__'): pass
211   ... """
212   >>> t = timeit.Timer(stmt=s)
213   >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)
214   4.85 usec/pass
215   >>> s = """\
216   ... try:
217   ...     int.__nonzero__
218   ... except AttributeError:
219   ...     pass
220   ... """
221   >>> t = timeit.Timer(stmt=s)
222   >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)
223   1.97 usec/pass
224   >>> s = """\
225   ... if hasattr(int, '__nonzero__'): pass
226   ... """
227   >>> t = timeit.Timer(stmt=s)
228   >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)
229   3.15 usec/pass
230
231To give the :mod:`timeit` module access to functions you define, you can pass a
232``setup`` parameter which contains an import statement::
233
234   def test():
235       "Stupid test function"
236       L = []
237       for i in range(100):
238           L.append(i)
239
240   if __name__=='__main__':
241       from timeit import Timer
242       t = Timer("test()", "from __main__ import test")
243       print t.timeit()
244