Introduction to Dataflows Gen2

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Alex Kuntz

Head of Cloud Curriculum, DataCamp

Dataflows Gen2 in Microsoft Fabric

  • What Are Dataflows Gen2?

    • Tool for data transformation and ingestion

    Dataflows Gen 2

  • Key Features:

    • Supports 300 plus Complex Transformations (e.g., merging, pivoting)
    • Flexible schema options for dynamic customization
    • Staging & Fast Copy: Handling large data volumes
    • Seamless integration with Data Pipelines
Acquisizione dei dati e modelli semantici con Microsoft Fabric

Overview of the DataFlows Gen2 Interface

DataFlows Gen2 leverages Power Query Online for an intuitive transformation experience. Dataflow Gen 2 Interface

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Dataflow Gen 2 Interface Components

  • Power Query Ribbon - Connect data sources and apply transformations Power Query Ribbon

  • Queries Pane: Manage, rename, and enable staging for data sources.

Queries

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Dataflow Gen 2 Interface Components

  • Diagram View: Visualize data sources and transformations Diagram View

  • Data Preview Pane: Preview and interact with data

    Data Preview Pane

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Dataflow Gen 2 Interface Components

  • Query Settings Pane: Track and modify applied steps

Query settings

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Supported Data Destinations in DataFlows Gen2

DataFlow Gen2 allows adding destination for each query, offering flexibility in data storage.

Supported Data Destinations:

  • Azure SQL Databases
  • Azure Data Explorer (Kusto)
  • Fabric Lakehouse
  • Fabric Warehouse
  • Fabric KQL Database

Supported Destination for Dataflows Gen2

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Integrating Dataflows Gen2 in Data Pipelines

Why Integrate?
Perform further operations on transformed data and automate workflows

dataflow pipeline activity

  • Add Dataflows Gen2 as a pipeline activity.
  • Ideal for additional operations on transformed data.
  • Automate Dataflows within the pipeline.
  • Monitor performance in real-time.

Integratining DataFlows in Data Pipeline

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Let's practice!

Acquisizione dei dati e modelli semantici con Microsoft Fabric

Preparing Video For Download...