獨立樣本 t 檢定

R 中的 A/B 測試

Lauryn Burleigh

Data Scientist

A/B 設計中的獨立樣本 t 檢定

平均數差異的顯著性
虛無假設：
- 「Cheese」與「Pepperoni」披薩的食用時間相同（無差異）

前提假設

依變數：
- 區間或比率
- 等距
隨機抽樣
常態分配
相近的組內變異數

library(ggplot2)
ggplot(pizza, aes(x = Time, 
                  fill = Topping)) +
       geom_histogram() + 
       facet_grid(Topping~.)

兩個近似常態的直方圖。粉色為 Pepperoni，平均 8；藍色為 Cheese，平均 6.2。

樣本數

library(pwr)
pwr.t.test(d = 0.73, power = 0.80, 
           sig.level = 0.05, 
           type = "two.sample", 
           alternative = "two.sided")

  Two-sample t test power calculation 

              n = 30.44799
              d = 0.73
      sig.level = 0.05
          power = 0.8
    alternative = two.sided

NOTE: n is number in *each* group

檢查變異數

組內變異數相等
Levene 檢定

不顯著（p > 0.05）＝變異數相等

若顯著（p < 0.05）＝變異數不相等

library(car)
leveneTest(Time ~ Topping, 
           data = Pizza)

Levene's Test for Homogeneity of Variance
         Df F value Pr(>F)
group   1  0.1457 0.7031

檢定

t.test(Time ~ Topping, data = Pizza, 
       paired = FALSE, 
       alternative = "two.sided", 
       var.equal = TRUE)

    Two Sample t-test
data:  Time by Topping
t = 2.3811, df = 198, p-value = 0.01821
alternative hypothesis: true difference 
in means between group Pepperoni and 
group Cheese is not equal to 0
95 percent confidence interval:
 0.0599370 0.6377601

Cohen's d

Cohen's d：t 檢定效果量
- 平均數差異的標準化指標

小：0.2
中：0.5
大：0.8

library(effectsize)
cohens_d(Time ~ Topping, data = Pizza)

Cohen's d |       95% CI
<----------------------
0.34      | [0.06, 0.62]

檢定力（Power）

library(pwr)

pwr.t.test(n = 1000, 
           sig.level = 0.0182, 
           d = 0.34, 
           type = "two.sample")

  Two-sample t test power calculation 
              n = 100
              d = 0.34
      sig.level = 0.0182
          power = 0.510256
    alternative = two.sided
NOTE: n is number in *each* group

接受結果的理想檢定力：0.8
- 錯誤機率：20%
- 100 - 80 = 20

一起來練習吧！

R 中的 A/B 測試