A/B Testing

Data Science for Business

Kaelen Medeiros

Data Scientist at Medidata Solutions

What is A/B Testing?

Case study: article headlines

A/B Testing Steps

• Picking a metric to track
• Calculating sample size
• Running the experiment
• Checking for significance

Pick a metric to track

Run your experiment

Check for significance

What if the results aren't significant?

• Difference is smaller than the threshold we chose
• Running our test longer won't help
• Still might be a difference; it's just small

Let's practice!

Data Science for Business