用於文字分類的循環神經網路

Deep Learning for Text with PyTorch

Shubham Jain

Data Scientist

文字用 RNN

處理可變長度序列
維持內部短期記憶
CNN 擅長在片段中找出模式
RNN 記住先前詞彙以理解語意

文字分類中的 RNN

諷刺範例圖片

為什麼？

RNN 可像人一樣逐字閱讀句子
理解脈絡與順序

範例：偵測推文中的諷刺

「I just love getting stuck in traffic.」

諷刺

複習：實作 Dataset 與 DataLoader

# Import libraries
from torch.utils.data import Dataset, DataLoader

# Create a class
class TextDataset(Dataset):

    def __init__(self, text):
        self.text = text

    def __len__(self):
        return len(self.text)

    def __getitem__(self, idx):
        return self.text[idx]

RNN 實作

sample_tweet = "This movie had a great plot and amazing acting."
# Preprocess the review and convert it to a tensor (not shown for brevity)
# ...
sentiment_prediction = model(sample_tweet_tensor)

訓練 RNN 將推文分類為正向或負向
輸出：「Positive」

RNN 變體：LSTM

情緒分析示意圖

推文：

  「Loved the cinematography,
  hated the dialogue.
  The acting was exceptional,
  but the plot fell flat.」

長短期記憶（LSTM）可處理 RNN 容易受挫的複雜性

LSTM

LSTM 架構：輸入閘、遺忘閘與輸出閘

class LSTMModel(nn.Module):

    def __init__(self, input_size, hidden_size, output_size):
        super(LSTMModel, self).__init__()
        self.lstm = nn.LSTM(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)


    def forward(self, x):
        _, (hidden, _) = self.lstm(x)
        output = self.fc(hidden.squeeze(0))
        return output

RNN 變體：GRU

Email 主旨：

   「Congratulations!
    You've won a free trip 
    to Hawaii!」

閘控循環單元（GRU）可不需完整脈絡就快速識別垃圾樣式

垃圾郵件

GRU

class GRUModel(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(GRUModel, self).__init__()
        self.gru = nn.GRU(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        _, hidden = self.gru(x)
        output = self.fc(hidden.squeeze(0))
        return output

一起來練習吧！

Deep Learning for Text with PyTorch