Data integration

Projeto de banco de dados

Lis Sulmont

Curriculum Manager

What is data integration

 

 

 

Data Integration combines data from different sources, formats, technologies to provide users with a translated and unified view of that data.

Projeto de banco de dados

Business case examples

  • 360-degree customer view
  • Acquisition
  • Legacy systems
Projeto de banco de dados

Unified data model

Database icon representing a unified data model

Projeto de banco de dados

Data sources

Three data sources with no connection to the unified data model

Projeto de banco de dados

Data sources format

Three data sources with no connections to the unified data model and with different formats

Projeto de banco de dados

Unified data model format

Three data sources with different formats unconnected to the unified data model which is a Redshift data warehouse

Projeto de banco de dados

Example: DataCamp

Three data sources with different formats and from different departments unconnected to the unified data model which is a Redshift data warehouse

Projeto de banco de dados

Update cadence - sales

The three data sources updating daily

Projeto de banco de dados

Update cadence - air traffic

The three data sources updating in real-time

Projeto de banco de dados

Different update cadences

The three data sources updating at different times

Projeto de banco de dados

So simple?

The three data sources connect to the unified data model

Projeto de banco de dados

Not really

The three data sources updating daily with a big X over it

Projeto de banco de dados

Transformations

The three data sources being transformed to fit the unified data model

Projeto de banco de dados

Transformation - tools

Processing the three data sources via ETL to the unified data model

Projeto de banco de dados

Choosing a data integration tool

  • Flexible
  • Reliable
  • Scalable
Projeto de banco de dados

Automated testing and proactive alerts

ETL diagram with testing icons at each phase

Projeto de banco de dados

Security

ETL diagram showing portions of data that should be restricted from its source to destination

Projeto de banco de dados

Security - credit card anonymization

ETL diagram with anonymization of credit card data

Projeto de banco de dados

Data governance - lineage

ETL diagram with a line representing the flow of data from its source to destination

Projeto de banco de dados

Let's practice!

Projeto de banco de dados

Preparing Video For Download...