Testing your package

Developing Python Packages

James Fulton

Climate informatics researcher

The art and discipline of testing

Imagine you are working on this function

def get_ends(x):
    """Get the first and last element in a list"""
    return x[0], x[-1]

You might test it to make sure it works

# Check the function
get_ends([1,1,5,39,0])

(1, 0)

The art and discipline of testing

Good packages brag about how many tests they have

91% of the pandas package code has test

Writing tests

def get_ends(x):
    """Get the first and last element in a list"""
    return x[0], x[-1]

def test_get_ends():
    assert get_ends([1,5,39,0]) == (1,0)

test_get_ends()

Writing tests

def get_ends(x):
    """Get the first and last element in a list"""
    return x[0], x[1]

def test_get_ends():
    assert get_ends([1,5,39,0]) == (1,0)

test_get_ends()

AssertionError: 
...

Writing tests

def get_ends(x):
    """Get the first and last element in a list"""
    return x[0], x[-1]

def test_get_ends():
    assert get_ends([1,5,39,0]) == (1,0)
    assert get_ends(['n','e','r','d']) == ('n','d')

Organizing tests inside your package

mysklearn/
|-- mysklearn   <-- package
|-- tests       <-- tests directory
|-- setup.py
|-- LICENSE
|-- MANIFEST.in

Organizing tests inside your package

Test directory layout

mysklearn/tests/
|-- __init__.py
|-- preprocessing

|   |-- __init__.py
|   |-- test_normalize.py

|   |-- test_standardize.py
|-- regression
|   |-- __init__.py
|   |-- test_regression.py
|-- test_utils.py

Code directory layout

mysklearn/mysklearn/
|-- __init__.py
|-- preprocessing

|   |-- __init__.py
|   |-- normalize.py

|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Organizing a test module

Inside test_normalize.py

from mysklearn.preprocessing.normalize import (
    find_max, find_min, normalize_data
)

def test_find_max(x):
    assert find_max([1,4,7,1])==7

def test_find_min(x):
    assert ...

def test_normalize_data(x):
    assert ...

DataCamp: Unit testing for data science

Inside normalize.py

def find_max(x):
    ...
    return x_max

def find_min(x):
    ...
    return x_min

def normalize_data(x):
    ...
    return x_norm

Running tests with pytest

pytest

pytest looks inside the test directory
It looks for modules like test_modulename.py
It looks for functions like test_functionname()
It runs these functions and shows output

mysklearn/ <-- navigate to here
|-- mysklearn
|-- tests
|-- setup.py
|-- LICENSE
|-- MANIFEST.in

Running tests with pytest

pytest

======================== test session starts ========================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/workspace/mypackages/mysklearn
collected 6 items

tests/preprocessing/test_normalize.py ...                     [ 50%]
tests/preprocessing/test_standardize.py ...                   [100%]

========================= 6 passed in 0.23s =========================

Running tests with pytest

pytest

======================== test session starts ========================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/workspace/mypackages/mysklearn     <--    ran in this directory
collected 6 items                                 <--    found 6 test functions

tests/preprocessing/test_normalize.py ...                     [ 50%]
tests/preprocessing/test_standardize.py ...                   [100%]

========================= 6 passed in 0.23s =========================

Running tests with pytest

pytest

======================== test session starts ========================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/workspace/mypackages/mysklearn
collected 6 items

tests/preprocessing/test_normalize.py ...                     [ 50%]  <-- 
tests/preprocessing/test_standardize.py ...                   [100%]  <-- 

========================= 6 passed in 0.23s =========================

Running tests with pytest

pytest

======================== test session starts ========================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/workspace/mypackages/mysklearn
collected 6 items

tests/preprocessing/test_normalize.py ...                     [ 50%]
tests/preprocessing/test_standardize.py ...                   [100%]

========================= 6 passed in 0.23s =========================

Running tests with pytest

pytest

==================================== test session starts ====================================
...
tests/preprocessing/test_normalize.py .F.                                              [ 50%]
tests/preprocessing/test_standardize.py ...                                            [100%]

========================================= FAILURES ==========================================
________________________________________ test_mymax _________________________________________
...

tests/preprocessing/test_normalize.py:10: AssertionError
================================== short test summary info ==================================
FAILED tests/preprocessing/test_normalize.py::test_mymax - assert -100 == 100       <-- test_mymax
================================ 1 failed, 5 passed in 0.17s ================================

Let's practice!

Developing Python Packages