用 Tablesaw 進行資料品質評估

使用 Java 進行資料清理

Dennis Lee

Software Engineer

書籍銷售資料集

書名	作者	語言	初版年份	銷量（百萬）	平均價格
A Tale of Two Cities	Charles Dickens	English	1859	200.0	12.99
The Little Prince (Le Petit Prince)	Antoine de Saint-Exupéry	French	1943	200.0	15.50
Harry Potter and the Philosopher's Stone	J. K. Rowling	English	1997	120.0	19.99
And Then There Were None	Agatha Christie	English	1939	100.0	14.95

檢視資料

import tech.tablesaw.api.Table;
import tech.tablesaw.api.DoubleColumn;
import tech.tablesaw.api.StringColumn;

// Load CSV file into Tablesaw table
Table books = Table.read().csv("bestsellers.csv");

// Get basic table information: row count, column count, column names
System.out.println("Row count: " + books.rowCount());
System.out.println("Column count: " + books.columnCount());
System.out.println("Column names: " + books.columnNames());

檢視資料：輸出

Row count: 290
Column count: 6
Column names: [Books, Authors, Language, First_Published, Sales_in_millions]

檢查遺漏值

for (String columnName : books.columnNames()) {

    // Get column by name and count missing values (nulls) in that column
    int missing = books.column(columnName).countMissing();

    System.out.println(columnName + " missing values: " + missing);
}

Books missing values: 0
Authors missing values: 0
Language missing values: 0
First_Published missing values: 0
Sales_in_millions missing values: 2
Average_Price missing values: 0

檢視類別欄位

// countBy() creates new table with unique values and their frequencies
Table languageCounts = books.countBy("Language");

System.out.println("Language distribution:\n");

// Prints formatted table showing each language and its count
System.out.println(languageCounts);

檢視類別欄位：輸出

Language distribution:

| Language   | Count |
|------------|-------|
| English    | 210   |
| French     | 10    |
| Chinese    | 6     |
| Portuguese | 1     |
| Spanish    | 3     |
| German     | 6     |
| Italian    | 5     |

分析數值欄位

// Get numeric column as DoubleColumn type for statistical operations
DoubleColumn sales = books.doubleColumn("Sales_in_millions");


System.out.println("Sales Statistics (millions of copies):\n");
// Smallest value in column
System.out.println("Min sales: " + sales.min() + " million");
// Largest value in column
System.out.println("Max sales: " + sales.max() + " million");

// Average of all values
System.out.println("Mean sales: " + sales.mean() + " million");
// Measure of spread (standard deviation)
System.out.println("Std. deviation: " + sales.standardDeviation() + " million");

分析數值欄位：輸出

Sales Statistics (millions of copies):

Min sales: 10.0 million
Max sales: 600.0 million
Mean sales: 49.996875 million
Std. deviation: 64.6846320839116 million

綜合應用

System.out.printf("Rows: %d, Columns: %s", books.rowCount(), books.columnCount());

for (String colName : books.columnNames()) // For each column
    System.out.println(books.column(colName).countMissing()); // Count null values

StringColumn language = books.stringColumn("Language"); // Get text column
Table langCounts = books.countBy("Language"); // Count categories

// Calculate numeric statistics
DoubleColumn sales = books.doubleColumn("Sales_in_Millions"); // Get number column
System.out.printf("Mean: %.1f million, Min: %.1f million, Max: %.1f million",
                  sales.mean(), sales.min(), sales.max());

綜合應用：輸出

Rows: 290, Columns: 6

Books missing values: 0
Authors missing values: 0

| Language   | Count |
|------------|-------|
| English    | 210   |
| French     | 10    |

Mean: 50.0 million, Min: 10.0 million, Max: 600.0 million

一起來練習吧！

使用 Java 進行資料清理