多輸出模型的評估與損失加權

Intermediate Deep Learning with PyTorch

Michal Oleszak

Machine Learning Engineer

模型評估

acc_alpha = Accuracy(
    task="multiclass", num_classes=30
)
acc_char = Accuracy(
    task="multiclass", num_classes=964
)


net.eval()
with torch.no_grad():
    for images, labels_alpha, labels_char \
    in dataloader_test:
        out_alpha, out_char = net(images)

        _, pred_alpha = torch.max(out_alpha, 1)
        _, pred_char = torch.max(out_char, 1)

        acc_alpha(pred_alpha, labels_alpha)
        acc_char(pred_char, labels_char)

為每個輸出設定評估指標
走訪測試資料載入器並取得輸出
為每個輸出計算預測
更新準確率指標
計算最終準確率

print(f"Alphabet: {acc_alpha.compute()}")
print(f"Character: {acc_char.compute()}")

Alphabet: 0.3166305720806122
Character: 0.24064336717128754

重溫多輸出訓練迴圈

for epoch in range(10):
    for images, labels_alpha, labels_char \
    in dataloader_train:
        optimizer.zero_grad()
        outputs_alpha, outputs_char = net(images)
        loss_alpha = criterion(
          outputs_alpha, labels_alpha
        )
        loss_char = criterion(
          outputs_char, labels_char
        )
        loss = loss_alpha + loss_char
        loss.backward()
        optimizer.step()

兩個損失：字母與字元
最終損失為兩者相加：loss = loss_alpha + loss_char
兩個分類任務同等重要

任務重要性調整

字元分類的重要性是字母分類的 2 倍

作法 1：將較重要的損失乘以 2
```
loss = loss_alpha + loss_char * 2
```

作法 2：指派權重且總和為 1

loss = 0.33 * loss_alpha + 0.67 * loss_char

警告：損失尺度不一致

在加權相加前，損失必須在同一尺度
範例任務：
- 預測房價 -> MSE 損失
- 預測品質：low、medium、high -> CrossEntropy 損失
CrossEntropy 通常是個位數
MSE 可能高達數萬
模型會忽略品質評估任務

解法：先正規化兩個損失再加權相加

loss_price = loss_price / torch.max(loss_price)
loss_quality = loss_quality / torch.max(loss_quality)
loss = 0.7 * loss_price + 0.3 * loss_quality

一起來練習吧！

Intermediate Deep Learning with PyTorch