Automação de Pipelines de Dados no Snowflake
Emily Melhuish
Technical Curriculum Developer, Snowflake
Tabelas dinâmicas:

Streams e tasks:

Automação declarativa de pipeline
SELECT — descreva o resultado; a Snowflake cuida do refreshCREATE DYNAMIC TABLE logistics.delivery_summary
TARGET_LAG = '1 hour'
WAREHOUSE = harbr_wh
AS
SELECT region, COUNT(*) AS shipments,
AVG(delivery_days) AS avg_days
FROM logistics.shipments GROUP BY region;
O contrato de frescor entre você e a Snowflake
| Valor de TARGET_LAG | Efeito |
|---|---|
'5 minutes' |
Tabela nunca fica mais de 5 min defasada — atualiza com frequência |
'1 hour' |
Atualiza menos — menor consumo de compute |
DOWNSTREAM |
Infere do dependente a jusante — ideal para tabelas encadeadas |
Refresh incremental
-- Compatível com incremental:
SELECT region, COUNT(*) AS shipments
FROM logistics.shipments
GROUP BY region;
Refresh completo

DOWNSTREAM segue a próxima tabela — evita atualizar em excessoTabelas dinâmicas: declarativas
-- Declare o resultado para a Snowflake
CREATE DYNAMIC TABLE logistics.summary
TARGET_LAG = '1 hour'
WAREHOUSE = harbr_wh
AS SELECT region, COUNT(*)
FROM shipments
GROUP BY 1;
Streams + Tasks: imperativas
-- Você controla cada etapa
CREATE TASK process_events
SCHEDULE = '5 MINUTE'
WHEN SYSTEM$STREAM_HAS_DATA
('events_stream')
AS CALL logistics.process_new_events();
SELECT name, state, refresh_start_time, refresh_end_time
FROM TABLE (
INFORMATION_SCHEMA.DYNAMIC_TABLE_REFRESH_HISTORY (
NAME_PREFIX => 'HARBR_DB.DELIVERY_SUMMARY.', ERROR_ONLY => TRUE
)
)
ORDER BY name, data_timestamp;
| NAME | TRIGGER | STATE | START | END |
|---|---|---|---|---|
| DELIVERY_SUMMARY | SCHEDULED | SUCCEEDED | 2026-03-01 05:00:00 | 2024-03-01 05:00:08 |
| DELIVERY_SUMMARY | SCHEDULED | FAILED | 2026-03-01 04:00:00 | 2024-03-01 04:00:03 |
Automação de Pipelines de Dados no Snowflake