Experiments

Introduction to Statistics

George Boorman

Curriculum Manager, DataCamp

Experiments, treatment, and control

  • Experiments are a subset of hypothesis testing
    • Experiments are not just conducted in academia

 

Experiments aim to answer: What is the effect of the treatment on the response?

  • Treatment: independent variable
  • Response: dependent variable

scientist_in_laboratory.jpg

1 Image credit: https://unsplash.com/@nci
Introduction to Statistics

Advertising as a treatment

What is the effect of an advertisement on the number of products purchased?

  • Treatment: advertisement
  • Response: number of products purchased

bar_plot_showing_the_number_of_products_purchased_for_treatment_vs_control_advertising.png

Introduction to Statistics

Controlled experiments

  • Participants are assigned to either the treatment group or the control group
    • Treatment group sees the advertisement
    • Control group does not see the advertisement
  • Groups should be comparable to avoid introducing bias
  • If groups are not comparable, this could lead to drawing incorrect conclusions

bar_plot_with_mean_treatment_age_of_50_and_control_age_of_25.png

Introduction to Statistics

The gold standard of experiments

  • Randomization
    • Participants are assigned to treatment/control randomly, not based on any other characteristics
    • Choosing randomly helps ensure that groups are comparable
    • Known as a randomized controlled trial

 

  • Blinding
    • Participants will not know which group they're in
    • Participants receive a placebo, which resembles the treatment but has no effect
    • In clinical trials it is common to use a sugar pill
Introduction to Statistics

The gold standard of experiments

  • Double-blind randomized controlled trial
    • Person administering the treatment/running the study doesn't know whether the treatment is real or a placebo
    • Prevents bias in the response and/or analysis of results

 

Fewer opportunities for bias = more reliable conclusion about causation

Introduction to Statistics

Randomized Controlled Trials vs. A/B testing

  • Randomized controlled trial
    • Can be multiple treatment groups
    • Popular in science, clinical research

 

  • A/B testing
    • Popular in marketing, engineering
    • Only split evenly into two groups

collection_of_pills.jpg

woman_programming_at_a_desk.jpg

1 Image credits: https://unsplash.com/@towfiqu999999; https://unsplash.com/@thisisengineering
Introduction to Statistics

Let's practice!

Introduction to Statistics

Preparing Video For Download...