過度離散的問題

Generalized Linear Models in Python

Ita Cirovic Donev

Data Science Consultant

理解資料

衛星數（螃蟹）的分佈圖

# mean of y
y_mean = crab['sat'].mean()

2.919

# variance of y
y_variance = crab['sat'].var()

9.912
Generalized Linear Models in Python

平均不等於變異數

  • $variance > mean$ $\rightarrow$ 過度離散（overdispersion）
  • $variance < mean$ $\rightarrow$ 不足離散（underdispersion）

影響：

  • 標準誤偏小
  • p 值偏小
Generalized Linear Models in Python

如何檢查過度離散？

已配適模型的摘要，標示殘差自由度與 Pearson 卡方統計量。

Generalized Linear Models in Python

計算估計的過度離散

ratio = crab_fit.pearson_chi2 / crab_fit.df_resid
print(ratio)

3.134

  • Ratio $ =1$ $\rightarrow$ 近似 Poisson

  • Ratio $ <1$ $\rightarrow$ 不足離散

  • Ratio $ >1$ $\rightarrow$ 過度離散

Generalized Linear Models in Python

負二項迴歸

  • $E(y)=\lambda$
  • $Var(y) = \lambda+\alpha\lambda^2$
  • $\alpha$－離散參數
Generalized Linear Models in Python

Python 中的 GLM 負二項

import statsmodels.api as sm
from statsmodels.formula.api import glm

model = glm('y ~ x', data = my_data, 
            family = sm.families.NegativeBinomial(alpha = 1)).fit()
Generalized Linear Models in Python

一起來練習吧！

Generalized Linear Models in Python

Preparing Video For Download...