Explicit indexes

Data Manipulation with pandas

Richie Cotton

Data Evangelist at DataCamp

The dog dataset, revisited

print(dogs)

      name        breed  color  height_cm  weight_kg
0    Bella     Labrador  Brown         56         25
1  Charlie       Poodle  Black         43         23
2     Lucy    Chow Chow  Brown         46         22
3   Cooper    Schnauzer   Gray         49         17
4      Max     Labrador  Black         59         29
5   Stella    Chihuahua    Tan         18          2
6   Bernie  St. Bernard  White         77         74

.columns and .index

dogs.columns

Index(['name', 'breed', 'color', 'height_cm', 'weight_kg'], dtype='object')

dogs.index

RangeIndex(start=0, stop=7, step=1)

Setting a column as the index

dogs_ind = dogs.set_index("name")

print(dogs_ind)

               breed  color  height_cm  weight_kg
name                                             
Bella       Labrador  Brown         56         25
Charlie       Poodle  Black         43         23
Lucy       Chow Chow  Brown         46         22
Cooper     Schnauzer   Grey         49         17
Max         Labrador  Black         59         29
Stella     Chihuahua    Tan         18          2
Bernie   St. Bernard  White         77         74

Removing an index

dogs_ind.reset_index()

      name        breed  color  height_cm  weight_kg
0    Bella     Labrador  Brown         56         25
1  Charlie       Poodle  Black         43         23
2     Lucy    Chow Chow  Brown         46         22
3   Cooper    Schnauzer   Grey         49         17
4      Max     Labrador  Black         59         29
5   Stella    Chihuahua    Tan         18          2
6   Bernie  St. Bernard  White         77         74

Dropping an index

dogs_ind.reset_index(drop=True)

         breed  color  height_cm  weight_kg
0     Labrador  Brown         56         25
1       Poodle  Black         43         23
2    Chow Chow  Brown         46         22
3    Schnauzer   Grey         49         17
4     Labrador  Black         59         29
5    Chihuahua    Tan         18          2
6  St. Bernard  White         77         74

Indexes make subsetting simpler

dogs[dogs["name"].isin(["Bella", "Stella"])]

     name      breed  color  height_cm  weight_kg
0   Bella   Labrador  Brown         56         25
5  Stella  Chihuahua    Tan         18          2

dogs_ind.loc[["Bella", "Stella"]]

            breed  color  height_cm  weight_kg
name                                          
Bella    Labrador  Brown         56         25
Stella  Chihuahua    Tan         18          2

Index values don't need to be unique

dogs_ind2 = dogs.set_index("breed")
print(dogs_ind2)

                name  color  height_cm  weight_kg
breed                                            
Labrador       Bella  Brown         56         25
Poodle       Charlie  Black         43         23
Chow Chow       Lucy  Brown         46         22
Schnauzer     Cooper   Grey         49         17
Labrador         Max  Black         59         29
Chihuahua     Stella    Tan         18          2
St. Bernard   Bernie  White         77         74

Subsetting on duplicated index values

dogs_ind2.loc["Labrador"]

           name  color  height_cm  weight_kg
breed                                       
Labrador  Bella  Brown         56         25
Labrador    Max  Black         59         29

Multi-level indexes a.k.a. hierarchical indexes

dogs_ind3 = dogs.set_index(["breed", "color"])
print(dogs_ind3)

                      name  height_cm  weight_kg
breed       color                               
Labrador    Brown    Bella         56         25
Poodle      Black  Charlie         43         23
Chow Chow   Brown     Lucy         46         22
Schnauzer   Grey    Cooper         49         17
Labrador    Black      Max         59         29
Chihuahua   Tan     Stella         18          2
St. Bernard White   Bernie         77         74

Subset the outer level with a list

dogs_ind3.loc[["Labrador", "Chihuahua"]]

                   name  height_cm  weight_kg
breed     color                              
Labrador  Brown   Bella         56         25
          Black     Max         59         29
Chihuahua Tan    Stella         18          2

Subset inner levels with a list of tuples

dogs_ind3.loc[[("Labrador", "Brown"), ("Chihuahua", "Tan")]]

                   name  height_cm  weight_kg
breed     color                              
Labrador  Brown   Bella         56         25
Chihuahua Tan    Stella         18          2

Sorting by index values

dogs_ind3.sort_index()

                      name  height_cm  weight_kg
breed       color                               
Chihuahua   Tan     Stella         18          2
Chow Chow   Brown     Lucy         46         22
Labrador    Black      Max         59         29
            Brown    Bella         56         25
Poodle      Black  Charlie         43         23
Schnauzer   Grey    Cooper         49         17
St. Bernard White   Bernie         77         74

Controlling sort_index

dogs_ind3.sort_index(level=["color", "breed"], ascending=[True, False])

                      name  height_cm  weight_kg
breed       color                               
Poodle      Black  Charlie         43         23
Labrador    Black      Max         59         29
            Brown    Bella         56         25
Chow Chow   Brown     Lucy         46         22
Schanuzer   Grey    Cooper         49         17
Chihuahua   Tan     Stella         18          2
St. Bernard White   Bernie         77         74

Now you have two problems

Index values are just data
Indexes violate "tidy data" principles
You need to learn two syntaxes

Temperature dataset

	date	city	country	avg_temp_c
0	2000-01-01	Abidjan	Côte D'Ivoire	27.293
1	2000-02-01	Abidjan	Côte D'Ivoire	27.685
2	2000-03-01	Abidjan	Côte D'Ivoire	29.061
3	2000-04-01	Abidjan	Côte D'Ivoire	28.162
4	2000-05-01	Abidjan	Côte D'Ivoire	27.547

Let's practice!

Data Manipulation with pandas