Valutare la qualità dei dati con Tablesaw

Pulizia dei dati in Java

Dennis Lee

Software Engineer

Dataset vendite libri

Books	Authors	Language	First_Published	Sales_in_Millions	Average_Price
A Tale of Two Cities	Charles Dickens	English	1859	200.0	12.99
The Little Prince (Le Petit Prince)	Antoine de Saint-Exupéry	French	1943	200.0	15.50
Harry Potter and the Philosopher's Stone	J. K. Rowling	English	1997	120.0	19.99
And Then There Were None	Agatha Christie	English	1939	100.0	14.95

Esaminare i dati

import tech.tablesaw.api.Table;
import tech.tablesaw.api.DoubleColumn;
import tech.tablesaw.api.StringColumn;

// Load CSV file into Tablesaw table
Table books = Table.read().csv("bestsellers.csv");

// Get basic table information: row count, column count, column names
System.out.println("Numero righe: " + books.rowCount());
System.out.println("Numero colonne: " + books.columnCount());
System.out.println("Nomi colonne: " + books.columnNames());

Esaminare i dati: output

Numero righe: 290
Numero colonne: 6
Nomi colonne: [Books, Authors, Language, First_Published, Sales_in_millions]

Verificare i valori mancanti

for (String columnName : books.columnNames()) {

    // Get column by name and count missing values (nulls) in that column
    int missing = books.column(columnName).countMissing();

    System.out.println(columnName + " valori mancanti: " + missing);
}

Books valori mancanti: 0
Authors valori mancanti: 0
Language valori mancanti: 0
First_Published valori mancanti: 0
Sales_in_millions valori mancanti: 2
Average_Price valori mancanti: 0

Esaminare le colonne categoriche

// countBy() creates new table with unique values and their frequencies
Table languageCounts = books.countBy("Language");

System.out.println("Distribuzione delle lingue:\n");

// Prints formatted table showing each language and its count
System.out.println(languageCounts);

Esaminare le colonne categoriche: output

Distribuzione delle lingue:

| Lingua     | Conteggio |
|------------|-----------|
| English    | 210       |
| French     | 10        |
| Chinese    | 6         |
| Portuguese | 1         |
| Spanish    | 3         |
| German     | 6         |
| Italian    | 5         |

Analizzare le colonne numeriche

// Get numeric column as DoubleColumn type for statistical operations
DoubleColumn sales = books.doubleColumn("Sales_in_millions");


System.out.println("Statistiche vendite (milioni di copie):\n");
// Smallest value in column
System.out.println("Vendite min: " + sales.min() + " milioni");
// Largest value in column
System.out.println("Vendite max: " + sales.max() + " milioni");

// Average of all values
System.out.println("Media vendite: " + sales.mean() + " milioni");
// Measure of spread (standard deviation)
System.out.println("Dev. standard: " + sales.standardDeviation() + " milioni");

Analizzare le colonne numeriche: output

Statistiche vendite (milioni di copie):

Vendite min: 10.0 milioni
Vendite max: 600.0 milioni
Media vendite: 49.996875 milioni
Dev. standard: 64.6846320839116 milioni

Mettere tutto insieme

System.out.printf("Righe: %d, Colonne: %s", books.rowCount(), books.columnCount());

for (String colName : books.columnNames()) // Per ogni colonna
    System.out.println(books.column(colName).countMissing()); // Conta i valori null

StringColumn language = books.stringColumn("Language"); // Ottieni colonna testuale
Table langCounts = books.countBy("Language"); // Conta le categorie

// Calcola statistiche numeriche
DoubleColumn sales = books.doubleColumn("Sales_in_Millions"); // Ottieni colonna numerica
System.out.printf("Media: %.1f milioni, Min: %.1f milioni, Max: %.1f milioni",
                  sales.mean(), sales.min(), sales.max());

Mettere tutto insieme: output

Righe: 290, Colonne: 6

Books valori mancanti: 0
Authors valori mancanti: 0

| Lingua     | Conteggio |
|------------|-----------|
| English    | 210       |
| French     | 10        |

Media: 50.0 milioni, Min: 10.0 milioni, Max: 600.0 milioni

Ayo berlatih!

Pulizia dei dati in Java