字串正規化

使用 Java 進行資料清理

Dennis Lee

Software Engineer

雜貨庫存資料集

本章全程使用雜貨庫存資料集
修改產品名稱等欄位，示範資料轉換

Product Name	Category	Status	Date Received	Quantity
Eggplant	Fruits & Vegetables	Discontinued	3/1/25	46
VEGETABLE OIL	Oils & Fats	Backordered	4/1/25	51
Cheese	Dairy	Active	6/1/25	78
Fresh (Organic) Carrots	Fruits & Vegetables	Discontinued	5/1/25	51
Bell Pepper Fresh	Fruits & Vegetables	Active	5/2/25	67

為什麼字串正規化很重要

混亂字串：電腦無法可靠辨識文字
範例：雜貨店庫存
解法：正規化字串

String[] messyProducts = {
    "Eggplant ",              // Extra whitespace at the end
    "VEGETABLE OIL",             // Inconsistent case
    "Fresh (Organic) *Carrots*", // Special characters
    "Bell   Pepper    Fresh",    // Extra whitespace between words
};

移除前後空白字元

String[] products = {"Eggplant ", "  Vegetable Oil", " Cheese "};
for (String product : products) {
    String cleaned = product.trim(); // Removes leading/trailing whitespace
    System.out.println(cleaned);
}

Eggplant
Vegetable Oil
Cheese

統一大小寫格式

List<String> products = Arrays.asList("Eggplant", "VEGETABLE OIL", "cheese");


products.stream()

        .map(String::toLowerCase) // Convert all to lowercase

        .forEach(System.out::println); // Print each product

eggplant
vegetable oil
cheese

Regex 樣式

正規表示式（regex）：用來比對文字的特殊樣式$^1$
[^a-zA-Z\\s]：找出所有不是英文字母或空白的字元

// This pattern matches any character that is NOT:
[       // Start a character set
^       // NOT - match anything not in this set
a-z     // any lowercase letter
A-Z     // any uppercase letter
\\s     // any whitespace character (need extra \ for Java to interpret \s)
]       // End character set

¹ https://www.datacamp.com/cheat-sheet/regular-expresso

清理特殊字元

String dirtyName = "Fresh (Organic) *Carrots*"; // Special characters: (, ), *


String cleaned = dirtyName.replaceAll("[^a-zA-Z\\s]", ""); // Remove non-letters


System.out.println(cleaned); // Output: "Fresh Organic Carrots"

Fresh Organic Carrots

清理多餘空白

import java.util.regex.Pattern;

Pattern pattern = Pattern.compile("\\s+"); // Match multiple spaces


String messyProduct = "Bell   Pepper    Fresh"; // Contains extra spaces
// Replace multiple spaces with single space
String cleanedProduct = pattern.matcher(messyProduct).replaceAll(" ");


System.out.println(cleanedProduct);

Bell Pepper Fresh

綜合運用

List<String> messyProducts = Arrays.asList(
        "Eggplant ", "VEGETABLE OIL",
        "Fresh (Organic) *Carrots*", "Bell   Pepper    Fresh"
); // Product names extracted from our grocery inventory dataset

messyProducts.stream()
        .map(s -> s.trim()                       // Fix outer spaces
                .replaceAll("[^a-zA-Z\\s]", "")  // Remove special chars
                .replaceAll("\\s+", " ")         // Fix inner spaces
                .toLowerCase())                  // Standardize case
        .forEach(System.out::println);

綜合運用：輸出

eggplant
vegetable oil
fresh organic carrots
bell pepper fresh

一起來練習吧！

使用 Java 進行資料清理