用卷積神經網路做文字分類

Deep Learning for Text with PyTorch

Shubham Jain

Instructor

用 CNN 做文字分類

將推文分類為
- Positive
- Negative
- Neutral

卷積運算

卷積運算
- 將濾波器（kernel）在輸入資料上滑動
- 濾波器每個位置進行對位計算

對文字：學到詞的結構與語意

¹ 動畫來源：Vincent Dumoulin、Francesco Visin

CNN 中的濾波器與步幅

濾波器：
- 在輸入上滑動的小矩陣

步幅：
- 濾波器每次移動的格數

濾波器與步幅

¹ 動畫來源：Vincent Dumoulin、Francesco Visin

文字用 CNN 的架構

卷積層：對輸入資料套用濾波器
池化層：在保留重點資訊下縮小資料尺寸
全連接層：根據前一層輸出做最終預測

用 CNN 實作文字分類模型

class SentimentAnalysisCNN(nn.Module):

    def __init__(self, vocab_size, embed_dim):

        super().__init__()

        self.embedding = nn.Embedding(vocab_size, 
                                         embed_dim)

        self.conv = nn.Conv1d(embed_dim, embed_dim, 
                               kernel_size=3, stride=1, 
                               padding=1)

        self.fc =  nn.Linear(embed_dim, 2)
    ...

__init__ 方法設定模型架構
super() 初始化基底類別 nn.Module
nn.Embedding 建立稠密詞向量
nn.Conv1d 用於一維資料

用 CNN 實作文字分類模型

    ...
    def forward(self, text):
        embedded = self.embedding(text).permute(0, 2, 1)

        conved = F.relu(self.conv(embedded))

        conved = conved.mean(dim=2)

        return self.fc(conved)

嵌入層將文字轉為 embedding
配對張量形狀以符合卷積層輸入
以 ReLU 擷取關鍵特徵
移除多餘維度與層級

為情感分析模型準備資料

vocab = ["i", "love", "this", "book", "do", "not", "like"]
word_to_idx = {word: i for i, word in enumerate(vocab)}

vocab_size = len(word_to_ix)

embed_dim = 10

book_samples = [
    ("The story was captivating and kept me hooked until the end.".split(),1),
    ("I found the characters shallow and the plot predictable.".split(),0)
]

model = SentimentAnalysisCNN(vocab_size, embed_dim)
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

訓練模型

for epoch in range(10):  
    for sentence, label in data:

        model.zero_grad()

        sentence = torch.LongTensor([word_to_idx.get(w, 0) for w in sentence]).unsqueeze(0)

        outputs = model(sentence)
        label = torch.LongTensor([int(label)])

        loss = criterion(outputs, label)
        loss.backward()

        optimizer.step()

執行情感分析模型

for sample in book_samples:

    input_tensor = torch.tensor([word_to_idx[w] for w in sample], dtype=torch.long).unsqueeze(0)

    outputs = model(input_tensor)

    _, predicted_label = torch.max(outputs.data, 1)

    sentiment = "Positive" if predicted_label.item() == 1 else "Negative"

    print(f"Book Review: {' '.join(sample)}")
    print(f"Sentiment: {sentiment}\n")

Book Review: The story was captivating and kept me hooked until the end
Sentiment: Positive
Book Review: I found the characters shallow and the plot predictable
Sentiment: Negative

一起來練習吧！

Deep Learning for Text with PyTorch