约束执行

Java 数据清洗

Dennis Lee

Software Engineer

数据质量控制

  • 校验注解:数据质量控制
  • 错误:国家缺失、金额无效、发货箱数为负
  • 笔误:把 100 箱写成 1000 箱

 

 

Salesperson Country Product Date Amount Boxes Shipped
James Rudeforth UK Mint Chip Choco 4-Jan-22 $5320 100
Van Tuxwell India 85% Dark Bars 1-Aug-22 $7896 94
Gigi Bohling US Peanut Butter Cubes 7-Jul-22 $4501 91
Java 数据清洗

基础校验注解

import jakarta.validation.constraints.NotNull;     // Import rules for empty fields
import jakarta.validation.constraints.Size;        // Import rules for string length

public class ChocolateSale {
    @NotNull(message = "Salesperson cannot be empty")
    private String salesperson;


@NotNull(message = "Country cannot be empty") // Multiple rules can be stacked @Size(min = 2, max = 50) // Enforces country name length private String country;
@NotNull(message = "Product cannot be empty") private String product; }
Java 数据清洗

数值约束

import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
// We will show message outputs later
public class ChocolateSale {
    @Min(value = 0, message = "Sales amount must be positive")
    private Double amount;


@Min(value = 1, message = "Must ship at least 1 box") @Max(value = 1000, message = "Cannot ship more than 1000 boxes") private Integer boxesShipped; }
Java 数据清洗

用于执行约束的导入

// Stores unique validation errors (no duplicates)
import java.util.Set;
// Holds details about a single validation error (field, message, etc.)
import jakarta.validation.ConstraintViolation;
// Entry point for creating validators
import jakarta.validation.Validation;
// Checks data against rules
import jakarta.validation.Validator;
// Creates configured validators
import jakarta.validation.ValidatorFactory;
// Thrown when validation fails
import jakarta.validation.ConstraintViolationException;
Java 数据清洗

实现校验器

class SalesValidator {
    // Create tools for checking our data
    private static final ValidatorFactory factory = Validation.buildDefaultValidatorFactory();

// Get a validator to check sales records private static final Validator validator = factory.getValidator();
public static Set<ConstraintViolation<ChocolateSale>> validateSale(ChocolateSale sale) { // Check sale record and return any problems found return validator.validate(sale); } }
Java 数据清洗

处理校验结果

public class Main {
    public static void main(String[] args) {
        // Create sale with some invalid data (null country, negative amount)
        ChocolateSale sale = new ChocolateSale("James Rudeforth", null, "Mint Chip Choco",
                LocalDate.parse("2022-01-04"), -5320.0, 1500);


// Check sale for validation violations Set<ConstraintViolation<ChocolateSale>> violations = SalesValidator.validateSale(sale); // Print each validation error message violations.forEach(violation -> System.out.println(violation.getMessage()));
// If any violations found, throw exception if (!violations.isEmpty()) throw new ConstraintViolationException(violations); } }
Java 数据清洗

校验输出

Country cannot be empty
Sales amount must be positive
Cannot ship more than 1000 boxes

Exception in thread "main" jakarta.validation.ConstraintViolationException
Java 数据清洗

校验可避免高成本错误

  • @NotNull@Size@Min@Max 及早捕获错误
  • 将校验技术应用于通用业务问题
  • 防止代价高昂的数据错误的第一道防线

工厂中筛选可可豆

Java 数据清洗

开始练习!

Java 数据清洗

Preparing Video For Download...