Generating hypotheses

Exploratory Data Analysis in Python

George Boorman

Curriculum Manager, DataCamp

What do we know?

Countplot showing the number of flights per airline in different price categories, with Jet Airways having the largest number of First Class tickets

sns.heatmap(planes.corr(), annot=True)
plt.show()

Heatmap showing Pearson correlation coefficient scores between variables in the planes dataset

sns.scatterplot(data=planes, x="Duration", y="Price", hue="Total_Stops")
plt.show()

Scatter plot of Price versus Duration, factoring Total Stops

Heatmap with correlation coefficient scores for each number of stops

Typewriter displaying "Fake News"

Would data from a different time give the same results?
Detecting relationships, differences, and patterns:
- We use Hypothesis Testing
Hypothesis testing requires, prior to data collection:
- Generating a hypothesis or question
- A decision on what statistical test to use

¹ Image credit: https://unsplash.com/@markuswinkler

office with a view looking out on to an airport runway

Magnifying glass looking into a bar chart

sns.barplot(data=planes, x="Airline", y="Duration")
plt.show()

Bar plot of duration versus airline

sns.barplot(data=planes, x="Destination", y="Price")
plt.show()

Bar plot showing average proce

Design our experiment
Involves steps such as:
- Choosing a sample
- Calculating how many data points we need
- Deciding what statistical test to run

Exploratory Data Analysis in Python