Advanced column-oriented database techniques

Introduction to NoSQL

Jake Roach

Data Engineer

Micro-partitioning data with Snowflake

Micro-partitioning:

  • Creates smaller "chunks" of rows, stored in columnar format
  • Stores metadata about each partition

$$

Allowing for:

  • Query pruning to reduce the amount of data accessed
  • Efficient execution of DML (data manipulation language)
1 https://docs.snowflake.com/en/user-guide/tables-clustering-micropartitions
Introduction to NoSQL

Micro-partitioning data

Multiple Snowflake tables, containing data before and after micro-partitioning.

Introduction to NoSQL

Data clustering with Snowflake

Data clustering:

  • Organizing or grouping similar data points together
  • Automatically performed during data load

$$

Allowing for:

  • Decreasing data accessed during execution
  • Improved query performance
1 https://docs.snowflake.com/en/user-guide/tables-clustering-micropartitions
Introduction to NoSQL

Data clustering

Snowflake tables before and after clustering.

Introduction to NoSQL

Query pruning

SELECT
    title,
    author,
    price
FROM books
WHERE
    price > 25.00;

Micro-partitioning and data clustering allow for:

  • Reducing data scanned
  • Fast time-to-insights

A Snowflake table with micro-partitioning and clustering leveraging query pruning.

Introduction to NoSQL

Let's practice!

Introduction to NoSQL

Preparing Video For Download...