Strängnormalisering

Datarensning i Java

Dennis Lee

Software Engineer

Lagerregistret för matbutiken

Vi arbetar med ett lagerregister för en matbutik genom hela kapitlet
Vissa kolumner, som produktnamn, ändras för att illustrera datatransformationer

Product Name	Category	Status	Date Received	Quantity
Eggplant	Fruits & Vegetables	Discontinued	3/1/25	46
VEGETABLE OIL	Oils & Fats	Backordered	4/1/25	51
Cheese	Dairy	Active	6/1/25	78
Fresh (Organic) Carrots	Fruits & Vegetables	Discontinued	5/1/25	51
Bell Pepper Fresh	Fruits & Vegetables	Active	5/2/25	67

Varför strängnormalisering spelar roll

Stökiga strängar: datorn kan inte tolka texten
Exempel: matbutikens lagerregister
Lösning: normalisera strängarna

String[] messyProducts = {
    "Eggplant ",              // Extra whitespace at the end
    "VEGETABLE OIL",             // Inconsistent case
    "Fresh (Organic) *Carrots*", // Special characters
    "Bell   Pepper    Fresh",    // Extra whitespace between words
};

Ta bort inledande och avslutande blanksteg

String[] products = {"Eggplant ", "  Vegetable Oil", " Cheese "};
for (String product : products) {
    String cleaned = product.trim(); // Removes leading/trailing whitespace
    System.out.println(cleaned);
}

Eggplant
Vegetable Oil
Cheese

Standardisera textens skiftläge

List<String> products = Arrays.asList("Eggplant", "VEGETABLE OIL", "cheese");


products.stream()

        .map(String::toLowerCase) // Convert all to lowercase

        .forEach(System.out::println); // Print each product

eggplant
vegetable oil
cheese

Regexmönster

Reguljära uttryck (regex): specialmönster som matchar text$^1$
[^a-zA-Z\\s]: Matchar alla tecken som inte är bokstäver eller blanksteg

// This pattern matches any character that is NOT:
[       // Start a character set
^       // NOT - match anything not in this set
a-z     // any lowercase letter
A-Z     // any uppercase letter
\\s     // any whitespace character (need extra \ for Java to interpret \s)
]       // End character set

¹ https://www.datacamp.com/cheat-sheet/regular-expresso

Rensa specialtecken

String dirtyName = "Fresh (Organic) *Carrots*"; // Special characters: (, ), *


String cleaned = dirtyName.replaceAll("[^a-zA-Z\\s]", ""); // Remove non-letters


System.out.println(cleaned); // Output: "Fresh Organic Carrots"

Fresh Organic Carrots

Rensa flera blanksteg

import java.util.regex.Pattern;

Pattern pattern = Pattern.compile("\\s+"); // Match multiple spaces


String messyProduct = "Bell   Pepper    Fresh"; // Contains extra spaces
// Replace multiple spaces with single space
String cleanedProduct = pattern.matcher(messyProduct).replaceAll(" ");


System.out.println(cleanedProduct);

Bell Pepper Fresh

Allt på en gång

List<String> messyProducts = Arrays.asList(
        "Eggplant ", "VEGETABLE OIL",
        "Fresh (Organic) *Carrots*", "Bell   Pepper    Fresh"
); // Product names extracted from our grocery inventory dataset

messyProducts.stream()
        .map(s -> s.trim()                       // Fix outer spaces
                .replaceAll("[^a-zA-Z\\s]", "")  // Remove special chars
                .replaceAll("\\s+", " ")         // Fix inner spaces
                .toLowerCase())                  // Standardize case
        .forEach(System.out::println);

Allt på en gång: utdata

eggplant
vegetable oil
fresh organic carrots
bell pepper fresh

Nu kör vi en övning!

Datarensning i Java