As the main topic of this blog is going to be timing experiments in Python, let’s start by having a look at the main tool I use to perform these experiments: Python’s built-in
In this post I will give a short and minimal introduction of the
timeit module, and go over the three main ways to use it to time small snippets of Python code.
- Timey Wimey Python: timeit
- In a script: import timeit
- Commandline: python -m timeit
- IPython/Jupyter magics: %timeit
Timey Wimey Python: timeit
What is Python’s
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.
– timeit documentation
In contrast to profiling the runtime of your entire program, the
timeit module is better suited to time small snippets. We can divide these comparisons in three classes, as illustrated by the examples I will use throughout this blogpost.
- Standalone one-liners
", ".join(str(n) for n in range(100))
", ".join([str(n) for n in range(100)])
", ".join(map(str, range(100)))
- Standalone multi-liners
x =  for i in range(1000): x.append(i)
x =  for i in range(1000): x += [i]
(ignoring for the sake of example that the list-comprehension
[i for i in range(1000)]is twice as fast)
- Either of the above with some setup required
text = "sample string"; char = "g"
char in text
In a script: import timeit
timeit is a Python module, you can import it and write Python scripts for your tests. This method is the most self-documenting and repeatable way of writing your timing experiments, and makes it easy to store the results for further processing.
Before we can run any experiments, we need to import the relevant function:
The function has the following specification:
timeit() without any further arguments, we execute the default statement
stmt='pass' a million times, showing the minimal overhead of Python’s
Unless specified otherwise,
timeit() will always run the default of one million iterations. This also means you have to pick a useful number yourself to make it finish in a reasonable amount of time that’s not too long or too short. The returned value is the number of seconds (as a
float) it has taken to run the statement
Let’s start with comparing the simple one-liners. We give the statement we want to time as a string, and specify a custom number of iterations. Note that when your statement already deals with double-quoted strings, the whole string should be given single-quoted, or vice-versa.
From the given
number and returned time in seconds, we can calculate that each of these lines took 33.4, 20.9 and 18.3 microseconds to execute respectively.
Sometimes the snippet you want to test will consist of multiple statements. Python allows you to put multiple statements on a single line with a semicolon
x = 1; y = 2. Then you can simply run your test as explained above for the standalone one-liners. This does not work for snippets with indented code like
To time these snippets that have to span multiple lines, you can simply give
timeit() a multi-line string as argument. The beautiful and Pythonic way to do this is using Python’s multi-line strings. (The non-Pythonic way is to add
\n characters in your regular strings.)
If there is some setup that only has to be run once, including it in a multi-line snippet means it’s executed at every iteration. Then you’d be measuring something you don’t want to measure! Instead, you can pass this setup statement as a separate argument to
Setup snippets most commonly consist of non-indented code, so you can usually just use semicolons to join those statements in a regular string. Remember that the
stmt argument comes before the
setup argument if you don’t pass them as keyword-arguments, even though the setup would normally be first.
Commandline: python -m timeit
timeit module can also be run as a commandline tool.
The usage is similar to the imported function as shown in the previous section, although the
-n option is more optional than in the imported function. What I mean with that? If you don’t specify a number of times to run your snippet, it will try successive powers of 10 (10, 100, 1000, …) until the total time spent is at least 0.2 seconds. It also reports the actual time per iteration, instead of having to calculate that yourself. This is reported in
sec, for nano-, micro-, mili- and whole seconds respectively.
Testing one-liners with the commandline is as easy as replacing
pass from the introduction with the code you want to test, and the
timeit tool will automatically report the time in a nice human-readable format.
As before, the simplest way to test a snippet of multiple lines is to join the statements with a semicolon if no indentation is required. When indentation is required, the other option is to pass multiple strings as arguments to the command. Note that you still have to add the indentation properly yourself! This can get tricky to count if your indentation is more than a single level deep, but is usually not too hard.
To add some initial setup such as imports or variable declarations, we can use the
The setup does come before the statement to test in this case, so that makes it a bit more intuitive to read.
IPython/Jupyter magics: %timeit
When working in the IPython interactive shell, or in a Jupyter notebook with the IPython kernel, you have access to the so-called ‘magic commands’. For timeit, there is the
%timeit magic. It simply takes the code you type after it and runs the
timeit command on it! Like magic!
One small caveat is that the outcome is reported as a mean +/- standard deviation of multiple repetitions, while the original Python documentation suggests always using the minimum.
For simple one-liners, just type the code as you would normally, and type
%timeit before it. That’s all!
%timeit ", ".join(str(n) for n in range(100))
24.1 µs ± 314 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
%timeit ", ".join([str(n) for n in range(100)])
20.4 µs ± 266 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
%timeit ", ".join(map(str, range(100)))
18.2 µs ± 777 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
%timeit magic doesn’t quite work for multi-line snippets though. Why? Because magics with a single
% are line-magics. For cell-magics, you just have to add another
% to make it
%%timeit. Then it will time all the code in your cell. No further difficulties whatsoever!
%%timeit x =  for i in range(1000): x.append(i)
75.1 µs ± 39.8 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
%%timeit x =  for i in range(1000): x += [i]
72.4 µs ± 2.33 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Now you might be thinking “but if this
%timeit just takes your code and times it, where do I put my setup command?” The answer: just run it in some previous cell! As IPython already takes care of passing your code on to the
timeit module properly, it also automatically passes along all current global variables. So we can simply first run a cell with our setup:
text = "sample string" char = "g"
And use the simple
%timeit magic to time the code we are actually interested in, without specifying which setup is associated with it.
%timeit char in text
49.9 ns ± 1.09 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
178 ns ± 24.6 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
I hope to have shown in this article how you can easily use Python’s
timeit module to measure execution times of your snippets, whether for fun or profit. Although each has it’s pros and cons (see below), I will personally recommend to use IPython’s
(%)%timeit magics as they are the most intuitive to use: just write the code as you would normally with the
%timeit magic in front.
Pros and Cons
from timeit import timeit
- Time taken returned within Python process, so can easily be used in further processing
- Must specify number of iterations manually
- Must manually calculate amount of time spent per iteration
python -m timeit
- Gives nice and clear output
- Multi-line snippets are less intuitive
- Commandline can be ‘scary’ to some
- Easiest to use
- Setup is dealt with automatically
- Needs IPython and/or Jupyter installed
- Gives mean +/- standard deviation as result, while Python’s documentation suggests using the minimum