Structuring imports

Developing Python Packages

James Fulton

Climate informatics researcher

Without package imports

import mysklearn

help(mysklearn.preprocessing)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'mysklearn' has no 
  attribute 'preprocessing'

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Without package imports

import mysklearn.preprocessing

help(mysklearn.preprocessing)

Help on package mysklearn.preprocessing in 
mysklearn:

NAME
    mysklearn.preprocessing - A subpackage 
      for standard preprocessing operations.

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Without package imports

import mysklearn.preprocessing

help(mysklearn.preprocessing.normalize)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 
  'mysklearn.preprocessing' has no attribute
  'normalize'

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Without package imports

import mysklearn.preprocessing.normalize

help(mysklearn.preprocessing.normalize)

Help on module mysklearn.preprocessing.normalize
in mysklearn.preprocessing:

NAME
    mysklearn.preprocessing.normalize - A module
      for normalizing data.

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Importing subpackages into packages

mysklearn/__init__.py

Absolute import

from mysklearn import preprocessing

Used most - more explicit

Relative import

from . import preprocessing

Used sometimes - shorter and sometimes simpler

Directory tree for package with subpackages

mysklearn/
|-- __init__.py        <--
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Importing modules

We imported preprocessing into mysklearn

import mysklearn
help(mysklearn.preprocessing)

Help on package mysklearn.preprocessing in 
mysklearn:

NAME
    mysklearn.preprocessing - A subpackage 
      for standard preprocessing operations.

But preprocessing has no link to normalize

import mysklearn
help(mysklearn.preprocessing.normalize)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 
  'mysklearn.preprocessing' has no attribute
  'normalize'

Importing modules

mysklearn/preprocessing/__init__.py

Absolute import

from mysklearn.preprocessing import normalize

Relative import

from . import normalize

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py    <--
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Restructuring imports

import mysklearn

help(mysklearn.preprocessing.normalize.normalize_data)

Help on function normalize_data in module
mysklearn.preprocessing.normalize:

normalize_data(x)
    Normalize the data array.

Import function into subpackage

mysklearn/preprocessing/__init__.py

Absolute import

from mysklearn.preprocessing.normalize import \
    normalize_data

Relative import

from .normalize import normalize_data

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py    <--
|   |-- normalize.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Import function into subpackage

import mysklearn

help(mysklearn.preprocessing.normalize_data)

Help on function normalize_data in module
mysklearn_imp.preprocessing.normalize:

normalize_data(x)
    Normalize the data array.

Importing between sibling modules

In normalize.py

Absolute import

from mysklearn.preprocessing.funcs import (
    mymax, mymin
)

Relative import

from .funcs import mymax, mymin

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py   <--
|   |-- funcs.py
|   |-- standardize.py
|-- regression
|   |-- __init__.py
|   |-- regression.py
|-- utils.py

Importing between modules far apart

A custom exception MyException is in utils.py

In normalize.py, standardize.py and regression.py

Absolute import

from mysklearn.utils import MyException

Relative import

from ..utils import MyException

Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
|   |-- __init__.py
|   |-- normalize.py   <--
|   `-- standardize.py <--
|-- regression
|   |-- __init__.py
|   |-- regression.py  <--
`-- utils.py

Relative import cheat sheet

from . import module
- From current directory, import module
from .. import module
- From one directory up, import module
from .module import function
- From module in current directory, import function
from ..subpackage.module import function
- From subpackage one directory up, from module in that subpackage, import function

Let's practice!

Developing Python Packages