直方图与离群值

R 中的信用风险建模

Lore Dirick

Manager of Data Science Curriculum at Flatiron School

使用 hist() 函数

hist(loan_data$int_rate)

利率直方图

R 中的信用风险建模

使用 hist() 函数

hist(loan_data$int_rate, main = "Histogram of interest rate", xlab = "Interest rate")

利率直方图

R 中的信用风险建模

对 annual_inc 使用 hist()

hist(loan_data$annual_inc, xlab = "Annual Income", main = "Histogram of Annual Income")

2020-06-12 1.55.03 PM 的截图

R 中的信用风险建模

对 annual_inc 使用 hist()

hist_income <- hist(loan_data$annual_inc,
                    xlab = "Annual Income",
                    main = "Histogram of Annual Income")
hist_income$breaks

0  500000 1000000 1500000 2000000 2500000 3000000 3500000 4000000 4500000 ...
R 中的信用风险建模

breaks 参数

n_breaks <- sqrt(nrow(loan_data)) # n_breaks = 170.5638
hist_income_n <- hist(loan_data$annual_inc, breaks = n_breaks, 
                      xlab = "Annual Income", main = "Histogram of Annual Income")

2020-06-12 1.55.58 PM 的截图

R 中的信用风险建模

annual_inc

plot(loan_data$annual_inc, ylab = "Annual Income")

2020-06-12 1.56.24 PM 的截图

R 中的信用风险建模

annual_inc

plot(loan_data$annual_inc, ylab = "Annual Income")

2020-06-12 1.56.53 PM 的截图

R 中的信用风险建模

离群值

  • 何时算离群值？

    • 专家判断

    • 经验法则，如：

      • Q1 - 1.5 × IQR
      • Q3 + 1.5 × IQR
    • 多数情况：两者结合
R 中的信用风险建模

专家判断

"年薪 > $300万 为离群值"

$$

# 查找离群值
index_outlier_expert <- which(loan_data$annual_inc > 3000000)

# 从数据集中移除离群值
loan_data_expert <- loan_data[-index_outlier_expert, ]
R 中的信用风险建模

经验法则

若大于 Q3 + 1.5 × IQR 则为离群值

$$

# 计算 Q3 + 1.5 * IQR
outlier_cutoff <- quantile(loan_data$annual_inc, 0.75) + 1.5 * IQR(loan_data$annual_inc)

# 标识离群值
index_outlier_ROT <- which(loan_data$annual_inc > outlier_cutoff)

# 移除离群值
loan_data_ROT <- loan_data[-index_outlier_ROT, ]
R 中的信用风险建模
hist(loan_data_expert$annual_inc,
     sqrt(nrow(loan_data_expert)), 
     xlab = "Annual income")


hist(loan_data_ROT$annual_inc,
     sqrt(nrow(loan_data_ROT)), 
     xlab = "Annual income")

R 中的信用风险建模

双变量图

plot(loan_data$emp_length, loan_data$annual_inc, 
     xlab= "Employment length", ylab= "Annual income")

2020-06-12 1.58.14 PM 的截图

R 中的信用风险建模

双变量图

plot(loan_data$emp_length, loan_data$annual_inc, 
     xlab= "Employment length", ylab= "Annual income")

2020-06-12 1.58.34 PM 的截图

R 中的信用风险建模

Vamos praticar!

R 中的信用风险建模

Preparing Video For Download...