Data integration

Conception de la base de données

Lis Sulmont

Curriculum Manager

What is data integration

 

 

 

Data Integration combines data from different sources, formats, technologies to provide users with a translated and unified view of that data.

Conception de la base de données

Business case examples

  • 360-degree customer view
  • Acquisition
  • Legacy systems
Conception de la base de données

Unified data model

Database icon representing a unified data model

Conception de la base de données

Data sources

Three data sources with no connection to the unified data model

Conception de la base de données

Data sources format

Three data sources with no connections to the unified data model and with different formats

Conception de la base de données

Unified data model format

Three data sources with different formats unconnected to the unified data model which is a Redshift data warehouse

Conception de la base de données

Example: DataCamp

Three data sources with different formats and from different departments unconnected to the unified data model which is a Redshift data warehouse

Conception de la base de données

Update cadence - sales

The three data sources updating daily

Conception de la base de données

Update cadence - air traffic

The three data sources updating in real-time

Conception de la base de données

Different update cadences

The three data sources updating at different times

Conception de la base de données

So simple?

The three data sources connect to the unified data model

Conception de la base de données

Not really

The three data sources updating daily with a big X over it

Conception de la base de données

Transformations

The three data sources being transformed to fit the unified data model

Conception de la base de données

Transformation - tools

Processing the three data sources via ETL to the unified data model

Conception de la base de données

Choosing a data integration tool

  • Flexible
  • Reliable
  • Scalable
Conception de la base de données

Automated testing and proactive alerts

ETL diagram with testing icons at each phase

Conception de la base de données

Security

ETL diagram showing portions of data that should be restricted from its source to destination

Conception de la base de données

Security - credit card anonymization

ETL diagram with anonymization of credit card data

Conception de la base de données

Data governance - lineage

ETL diagram with a line representing the flow of data from its source to destination

Conception de la base de données

Let's practice!

Conception de la base de données

Preparing Video For Download...