Dealing with nulls

Reporting in SQL

Tyler Pernes

Learning & Development Consultant

What does null really mean?

+--------+-----------------+
| order  | price_per_unit  |
|--------|-----------------|
| 1      | 4.50            |
| 2      | 2.25            |
| 3      | null            |
+--------+-----------------+

Yet to go through?
Free?
Flat price?

Issues with nulls

soccer_games
+---------+-------+-------+
| game_id | home  | away  |
|---------|-------|-------|
| 123     | 3     | 2     |
| 124     | 2     | null  |
| 125     | null  | 1     |
+---------+-------+-------+

SELECT *, home + away AS total_goals
FROM soccer_games;

Issues with nulls

+---------+-------+-------+--------------+
| game_id | home  | away  | total_goals  |
|---------|-------|-------|--------------|
| 123     | 3     | 2     | 5            |
| 124     | 2     | null  | null         |
| 125     | null  | 1     | null         |
+---------+-------+-------+--------------+

Issues with nulls

SELECT 
    region, 
    COUNT(DISTINCT athlete_id) AS athletes
FROM summer_games AS s
JOIN countries AS c
ON s.country_id = c.id
GROUP BY region;

+----------+-----------+
| region   | athletes  |
|----------|-----------|
| BALTICS  | 42        |
| OCEANIA  | 62        |
| null     | 10        |
+----------+-----------+

Unclear what null represents!

Fix 1: Filtering nulls

original_table
+--------+-----------------+
| order  | price_per_unit  |
|--------|-----------------|
| 1      | 4.50            |
| 2      | 2.25            |
| 3      | null            |
+--------+-----------------+

SELECT *
FROM original_table
WHERE price_per_unit IS NOT NULL;

Fix 1: Filtering nulls

+--------+-----------------+
| order  | price_per_unit  |
|--------|-----------------|
| 1      | 4.50            |
| 2      | 2.25            |
+--------+-----------------+

Fix 2: COALESCE()

Syntax:COALESCE(field, null_replacement)

SELECT 
    COALESCE(region,'Independent Athletes') AS region,
    COUNT(DISTINCT athlete_id) AS athletes
FROM summer_games AS s
JOIN countries AS c
ON s.country_id = c.id;

+-----------------------+-----------+
| region                | athletes  |
|-----------------------|-----------|
| BALTICS               | 42        |
| OCEANIA               | 62        |
| Independent Athletes  | 10        |
+-----------------------+-----------+

Fix 2: COALESCE()

soccer_games
+---------+-------+-------+
| game_id | home  | away  |
|---------|-------|-------|
| 123     | 3     | 2     |
| 124     | 2     | null  |
| 125     | null  | 1     |
+---------+-------+-------+

SELECT *, COALESCE(home,0) + COALESCE(away,0) AS total_goals
FROM soccer_games;

Fix 2: COALESCE()

+---------+-------+-------+--------------+
| game_id | home  | away  | total_goals  |
|---------|-------|-------|--------------|
| 123     | 3     | 2     | 5            |
| 124     | 2     | null  | 2            |
| 125     | null  | 1     | 1            |
+---------+-------+-------+--------------+

Nulls as a result of a query

Causes:

LEFT JOIN does not match all rows
No CASE statement conditional is satisfied
Several others!

Measuring the impact of nulls

Ratio of rows that are null

SELECT SUM(CASE when country IS NULL then 1 else 0 end) / SUM(1.00)
FROM orders;

+-------+
| .12   |
+-------+

Ratio of revenue that is null

SELECT SUM(CASE when country IS NULL then revenue else 0 end) / SUM(revenue)
FROM orders;

+-------+
| .25   |
+-------+

Practice time!

Reporting in SQL