Explicit indexes

Data Manipulation with pandas

Richie Cotton

Data Evangelist at DataCamp

The dog dataset, revisited

print(dogs)
      name        breed  color  height_cm  weight_kg
0    Bella     Labrador  Brown         56         25
1  Charlie       Poodle  Black         43         23
2     Lucy    Chow Chow  Brown         46         22
3   Cooper    Schnauzer   Gray         49         17
4      Max     Labrador  Black         59         29
5   Stella    Chihuahua    Tan         18          2
6   Bernie  St. Bernard  White         77         74
Data Manipulation with pandas

.columns and .index

dogs.columns
Index(['name', 'breed', 'color', 'height_cm', 'weight_kg'], dtype='object')
dogs.index
RangeIndex(start=0, stop=7, step=1)
Data Manipulation with pandas

Setting a column as the index

dogs_ind = dogs.set_index("name")

print(dogs_ind)
               breed  color  height_cm  weight_kg
name                                             
Bella       Labrador  Brown         56         25
Charlie       Poodle  Black         43         23
Lucy       Chow Chow  Brown         46         22
Cooper     Schnauzer   Grey         49         17
Max         Labrador  Black         59         29
Stella     Chihuahua    Tan         18          2
Bernie   St. Bernard  White         77         74
Data Manipulation with pandas

Removing an index

dogs_ind.reset_index()
      name        breed  color  height_cm  weight_kg
0    Bella     Labrador  Brown         56         25
1  Charlie       Poodle  Black         43         23
2     Lucy    Chow Chow  Brown         46         22
3   Cooper    Schnauzer   Grey         49         17
4      Max     Labrador  Black         59         29
5   Stella    Chihuahua    Tan         18          2
6   Bernie  St. Bernard  White         77         74
Data Manipulation with pandas

Dropping an index

dogs_ind.reset_index(drop=True)
         breed  color  height_cm  weight_kg
0     Labrador  Brown         56         25
1       Poodle  Black         43         23
2    Chow Chow  Brown         46         22
3    Schnauzer   Grey         49         17
4     Labrador  Black         59         29
5    Chihuahua    Tan         18          2
6  St. Bernard  White         77         74
Data Manipulation with pandas

Indexes make subsetting simpler

dogs[dogs["name"].isin(["Bella", "Stella"])]
     name      breed  color  height_cm  weight_kg
0   Bella   Labrador  Brown         56         25
5  Stella  Chihuahua    Tan         18          2
dogs_ind.loc[["Bella", "Stella"]]
            breed  color  height_cm  weight_kg
name                                          
Bella    Labrador  Brown         56         25
Stella  Chihuahua    Tan         18          2
Data Manipulation with pandas

Index values don't need to be unique

dogs_ind2 = dogs.set_index("breed")
print(dogs_ind2)
                name  color  height_cm  weight_kg
breed                                            
Labrador       Bella  Brown         56         25
Poodle       Charlie  Black         43         23
Chow Chow       Lucy  Brown         46         22
Schnauzer     Cooper   Grey         49         17
Labrador         Max  Black         59         29
Chihuahua     Stella    Tan         18          2
St. Bernard   Bernie  White         77         74
Data Manipulation with pandas

Subsetting on duplicated index values

dogs_ind2.loc["Labrador"]
           name  color  height_cm  weight_kg
breed                                       
Labrador  Bella  Brown         56         25
Labrador    Max  Black         59         29
Data Manipulation with pandas

Multi-level indexes a.k.a. hierarchical indexes

dogs_ind3 = dogs.set_index(["breed", "color"])
print(dogs_ind3)
                      name  height_cm  weight_kg
breed       color                               
Labrador    Brown    Bella         56         25
Poodle      Black  Charlie         43         23
Chow Chow   Brown     Lucy         46         22
Schnauzer   Grey    Cooper         49         17
Labrador    Black      Max         59         29
Chihuahua   Tan     Stella         18          2
St. Bernard White   Bernie         77         74
Data Manipulation with pandas

Subset the outer level with a list

dogs_ind3.loc[["Labrador", "Chihuahua"]]
                   name  height_cm  weight_kg
breed     color                              
Labrador  Brown   Bella         56         25
          Black     Max         59         29
Chihuahua Tan    Stella         18          2
Data Manipulation with pandas

Subset inner levels with a list of tuples

dogs_ind3.loc[[("Labrador", "Brown"), ("Chihuahua", "Tan")]]
                   name  height_cm  weight_kg
breed     color                              
Labrador  Brown   Bella         56         25
Chihuahua Tan    Stella         18          2
Data Manipulation with pandas

Sorting by index values

dogs_ind3.sort_index()
                      name  height_cm  weight_kg
breed       color                               
Chihuahua   Tan     Stella         18          2
Chow Chow   Brown     Lucy         46         22
Labrador    Black      Max         59         29
            Brown    Bella         56         25
Poodle      Black  Charlie         43         23
Schnauzer   Grey    Cooper         49         17
St. Bernard White   Bernie         77         74
Data Manipulation with pandas

Controlling sort_index

dogs_ind3.sort_index(level=["color", "breed"], ascending=[True, False])
                      name  height_cm  weight_kg
breed       color                               
Poodle      Black  Charlie         43         23
Labrador    Black      Max         59         29
            Brown    Bella         56         25
Chow Chow   Brown     Lucy         46         22
Schanuzer   Grey    Cooper         49         17
Chihuahua   Tan     Stella         18          2
St. Bernard White   Bernie         77         74
Data Manipulation with pandas

Now you have two problems

  • Index values are just data
  • Indexes violate "tidy data" principles
  • You need to learn two syntaxes
Data Manipulation with pandas

Temperature dataset

date city country avg_temp_c
0 2000-01-01 Abidjan Côte D'Ivoire 27.293
1 2000-02-01 Abidjan Côte D'Ivoire 27.685
2 2000-03-01 Abidjan Côte D'Ivoire 29.061
3 2000-04-01 Abidjan Côte D'Ivoire 28.162
4 2000-05-01 Abidjan Côte D'Ivoire 27.547
Data Manipulation with pandas

Let's practice!

Data Manipulation with pandas

Preparing Video For Download...