Cleaning strings

Reporting in SQL

Tyler Pernes

Learning & Development Consultant

Messy strings

+---------------------+
| country             |
|---------------------|
| US                  |
| U.S.                |
| US (United States)  |
| us                  |
|   US                |
+---------------------+

String functions

Replacing or removing characters

+------------+--------+
| country    | points |
|------------|--------|
| US         | 5      |
| U.S.       | 3      |
+------------+--------+

SELECT REPLACE(country,'.','') AS country_cleaned, SUM(points) as points
FROM original_table
GROUP BY country_cleaned;

+-----------------+--------+
| country_cleaned | points |
|-----------------|--------|
| US              | 8      |
+-----------------+--------+

String functions

Parsing strings

+---------------------+--------+
| country             | points |
|---------------------|--------|
| US                  | 5      |
| US (United States)  | 1      |
+---------------------+--------+

SELECT LEFT(country,2) AS country_cleaned, SUM(points) as points
FROM original_table
GROUP BY country_cleaned;

+------------------+--------+
| country_cleaned  | points |
|------------------|--------|
| US               | 6      |
+------------------+--------+

String functions

Changing case

+----------+--------+
| country  | points |
|----------|--------|
| US       | 5      |
| us       | 4      |
+----------+--------+

SELECT UPPER(country) AS country_cleaned, SUM(points) as points
FROM original_table
GROUP BY country_cleaned;

+------------------+--------+
| country_cleaned  | points |
|------------------|--------|
| US               | 9      |
+------------------+--------+

String functions

Trimming extra spaces

+----------+--------+
| country  | points |
|----------|--------|
| US       | 5      |
|   US     | 2      |
+----------+--------+

SELECT TRIM(country) AS country_cleaned, SUM(points) as points
FROM original_table
GROUP BY country_cleaned;

+------------------+--------+
| country_cleaned  | points |
|------------------|--------|
| US               | 7      |
+------------------+--------+

Nesting functions

original_table
+---------------------+
| country             |
|---------------------|
| US                  |
| U.S.                |
| US (United States)  |
| us                  |
|   US                |
+---------------------+

REPLACE(country,'.','')

TRIM(country)

LEFT(country,2)

UPPER(country)

Take it step-by-step

REPLACE(country,'.','')

TRIM(REPLACE(country,'.',''))

LEFT(TRIM(REPLACE(country,'.','')),2)

UPPER(LEFT(TRIM(REPLACE(country,'.','')),2))

Final Query:

SELECT UPPER(LEFT(TRIM(REPLACE(country,'.','')),2)) AS country_cleaned
FROM original_table
GROUP BY country_cleaned;

Order of nesting matters!

SELECT TRIM(REPLACE(UPPER(LEFT(country,2)),'.','')) AS country_cleaned
FROM original_table
GROUP BY country_cleaned;

+-------------------+
| country_cleaned   |
|-------------------|
| US                |
| U                 |
|                   |
+-------------------+

String function documentation

https://www.postgresql.org/docs/9.1/functions-string.html

Let's practice!

Reporting in SQL