Importing flat files using NumPy

Introduzione all'importazione dei dati in Python

Hugo Bowne-Anderson

Data Scientist at DataCamp

Why NumPy?

NumPy arrays: standard for storing numerical data

ch_1_3.003.png

Why NumPy?

NumPy arrays: standard for storing numerical data
Essential for other packages: e.g. scikit-learn
loadtxt()
genfromtxt()

Importing flat files using NumPy

import numpy as np
filename = 'MNIST.txt'
data = np.loadtxt(filename, delimiter=',')
data

[[   0.    0.    0.    0.    0.]
 [  86.  250.  254.  254.  254.]
 [   0.    0.    0.    9.  254.]
 ..., 
 [   0.    0.    0.    0.    0.]
 [   0.    0.    0.    0.    0.]
 [   0.    0.    0.    0.    0.]]

Customizing your NumPy import

import numpy as np
filename = 'MNIST_header.txt'
data = np.loadtxt(filename, delimiter=',', skiprows=1)
print(data)

[[   0.    0.    0.    0.    0.]
 [  86.  250.  254.  254.  254.]
 [   0.    0.    0.    9.  254.]
 ..., 
 [   0.    0.    0.    0.    0.]
 [   0.    0.    0.    0.    0.]
 [   0.    0.    0.    0.    0.]]

skiprows: how many rows (not indices) you wish to skip

Customizing your NumPy import

import numpy as np
filename = 'MNIST_header.txt'
data = np.loadtxt(filename, delimiter=',', skiprows=1, usecols=[0, 2])
print(data)

[[   0.    0.]
 [  86.  254.]
 [   0.    0.]
 ..., 
 [   0.    0.]
 [   0.    0.]
 [   0.    0.]]

usecols: list of the indices of the columns you wish to keep

Customizing your NumPy import

data = np.loadtxt(filename, delimiter=',', dtype=str)

Mixed datatypes

titanic.csv

                        Name      Sex  Cabin   Fare
     Braund, Mr. Owen Harris     male    NaN    7.3
  Cumings, Mrs. John Bradley   female    C85   71.3
      Heikkinen, Miss. Laina   female    NaN    8.0
Futrelle, Mrs. Jacques Heath   female   C123   53.1
    Allen, Mr. William Henry     male    NaN   8.05

¹ Source: Kaggle

Mixed datatypes

titanic.csv

                        Name      Sex  Cabin   Fare
     Braund, Mr. Owen Harris     male    NaN    7.3
  Cumings, Mrs. John Bradley   female    C85   71.3
      Heikkinen, Miss. Laina   female    NaN    8.0
Futrelle, Mrs. Jacques Heath   female   C123   53.1
    Allen, Mr. William Henry     male    NaN   8.05
               ^                                 ^
            strings                           floats

¹ Source: Kaggle

Let's practice!

Introduzione all'importazione dei dati in Python