This example is prepared by Qingkai Kong (qingkai.kong@gmail.com) from Berkeley Seismological Lab for the lightning talk at The Hacker Within at BIDS on April 6th 2016. You can find the code on Qingkai's Github.

The purpose of this script is to show how I usually do to speedup my python script. And hope this is useful to you.

## Line_profiler¶

Many times, you find your python script is slow, but you just don't know which part drags the whole performance down. To get an idea, I usually use the line_profiler from Robert Kern, this is really good if you want to identify which line uses more time than you expceted, and how often each line executed. Here's a blog talking about profile python Marco BonzaniniYou can use line_profiler either in command line or in ipython notebook.

### 1 Command line¶

In a typical workflow, one only cares about line timings of a few functions because wading through the results of timing every single line of code would be overwhelming. However, LineProfiler does need to be explicitly told what functions to profile. The easiest way to get started is to use the kernprof script.Steps:

- in your script, you decorate the functions you want to profile with @profile. For example:
@profile def function_to_profile(a, b, c): ...- Run the following command in the terminal:
kernprof -v -l profile_test.py

### 2 Run in Ipython notebook¶

To run line_profiler in the notebook, you need load the extension first, and then use the magic commands %lprun to profile the script.
In [1]:

```
%load_ext line_profiler
```

In [2]:

```
def example_function(myRange):
# directly convert range to string list
str_list = []
for i in myRange:
str_list.append(str(i))
def example_function2(myRange):
# use list comprehension to convert range to string list
str_list = [str(i) for i in myRange]
```

In [3]:

```
%lprun -f example_function example_function(range(1000000))
```

In [4]:

```
%lprun -f example_function2 example_function2(range(1000000))
```

## Using f2py¶

f2py - Fortran to Python interface generator, is used to call Fortran 77/90/95 external subroutines and Fortran 90/95 module subroutines as well as C functions. It is part of the Numpy now. You can find more details here.Let's grab the example from the above link, and compare a python version and a fortran version for speed.

### Python version¶

In [5]:

```
import numpy as np
def fib(A):
'''
CALCULATE FIRST N FIBONACCI NUMBERS
'''
n = len(A)
for i in range(n):
if i == 0:
A[i] = 0.
elif i == 1:
A[i] = 1.
else:
A[i] = A[i-1] + A[i-2]
return A
```

In [6]:

```
dat_in = np.zeros(10)
dat_out = fib(dat_in)
dat_out
```

Out[6]:

In [7]:

```
dat_in = np.zeros(1000)
```

In [8]:

```
%lprun -f fib fib(dat_in)
```

### Fortran version¶

let's first write a simple fortran subroutine
In [9]:

```
!ls
```

In [10]:

```
%%writefile fib1.f
C FILE: FIB1.F
SUBROUTINE FIB(A,N)
C
C CALCULATE FIRST N FIBONACCI NUMBERS
C
INTEGER N
REAL*8 A(N)
DO I=1,N
IF (I.EQ.1) THEN
A(I) = 0.0D0
ELSEIF (I.EQ.2) THEN
A(I) = 1.0D0
ELSE
A(I) = A(I-1) + A(I-2)
ENDIF
ENDDO
END
C END FILE FIB1.F
```

In [11]:

```
!f2py -c fib1.f -m fib1
```

In [12]:

```
!ls
```

You can see it created an python interface fib1.so. Now you can import the function into python.

In [13]:

```
import fib1
import numpy as np
```

In [14]:

```
print fib1.fib.__doc__
```

In [15]:

```
a = np.zeros(9)
```

In [16]:

```
fib1.fib(a)
```

In [17]:

```
a
```

Out[17]:

Let's compare the time of both function, we can see for this exmaple, the fortran version is about 100 times faster than the python version.

In [18]:

```
a = np.zeros(1000)
```

In [19]:

```
%timeit fib1.fib(a)
```

In [20]:

```
%timeit fib(a)
```

## My workflow¶

When I start some script, I usually use the following workflow to speedup:- write python script
- profile it line by line
- finding some stupid thing that can be easily fix (I found this many times)
- parallel the part that used a lot of time
- use a f2py to take advantage of the fotran speed

## No comments:

## Post a Comment