The approximation shortcut

Inference for Categorical Data in R

Andrew Bray

Assistant Professor of Statistics at Reed College

Confidence Intervals

SE

0.009998905

SE_small_n

0.03809731

SE_low_p

0.00547912

Standard errors increase when

n is small
p is close to 0.5

The normal distribution

A.K.A the "bell curve".

observations are independent
n is large

Then

$\hat{p}$ follows a normal distribution

Standard deviation

$$\sqrt{\frac{ \hat{p} \times (1 - \hat{p})}{n}}$$

Assessing model assumptions

How do I check "observations are independent"?

This depends upon the data collection method.

What does "n is large" mean?

$n \times \hat{p} \gt 10$
$n \times(1 - \hat{p}) \gt 10$

Calculating standard error: approximation

p_hat <- gss2016 %>%
  summarize(mean(happy == "HAPPY")) %>%
  pull()
n <- nrow(gss2016)

c(n * p_hat, n * (1 - p_hat))

116  35

SE_approx <- sqrt(p_hat * (1 - p_hat) / n)
SE_approx

0.03418468

Calculating standard error: computation

boot <- gss2016 %>%
  specify(response = happy, success = "HAPPY") %>%
  generate(reps = 500, type = "bootstrap") %>%
  calculate(stat = "prop")
SE_boot <- boot %>%
  summarize(sd(stat)) %>%
  pull()
SE_boot

0.03176741

Sampling distributions

ggplot(boot, aes(x = stat)) +
  geom_density()

Sampling distributions

ggplot(boot, aes(x = stat)) +
  geom_density() +
  stat_function(fun = dnorm, 
                color = "purple",
                args = 
                  list(mean = p_hat,
                       sd = SE_approx))

Sampling distributions

ggplot(boot, aes(x = stat)) +
  geom_density() +
  stat_function(fun = dnorm, 
                color = "purple",
                args = 
                  list(mean = p_hat,
                       sd = SE_approx))

Let's practice!

Inference for Categorical Data in R