Data Processing in Azure
Understanding Microsoft Azure
Kevin James
Technical Lead and Training Architect
Real-time vs Batch Processing
Consider processing type before choosing a service: real-time or batch.
Real-time: Immediate analytics
Batch: Scheduled or ad-hoc analytics
Real-time vs Batch Processing
Consider processing type before choosing a service
Real-time: Immediate analytics
Batch: Scheduled or ad-hoc analytics
Example in healthcare:
Real-time - hospital emergency dashboards
Batch - weekly-updated dashboards
Different infrastructure and cost implications
ETL Processes
ETL Processes
ETL Processes
Processing tools
Azure Synapse Analytics
Part of Microsoft Fabric
integrates big data and data warehouses
Unified experience for data ingestion, preparation, management, and delivery
Supports real-time insights and batch processing
Acts as a turbocharged analytics engine
Azure Stream Analytics
Enables real-time data access
Sets up real-time analytics with straightforward query definition
Handles data streaming from diverse inputs like blob storage
Essential for immediate insights:
fraud detection in a bank
dynamic pricing on the stock market
Azure Databricks
Microsoft-Databricks collaboration
Analytics platform optimized for Azure
Unified environment for data engineering, analytics, and machine learning
Collaborative workspace for data scientists and engineers
Built-in Data Lake support
Real-time and batch
Azure Data Factory
Cloud-based integration service
Creates, schedules, orchestrates data workflows
Streamlines ETL processes
Handles diverse data sources and formats
Automates workflows for flexibility
Azure HDInsight
Managed service for fast, customizable data processing
Runs on popular open-source platforms:
Hadoop
Spark
Kafka
Easily scales resources based on demand
Seamlessly connects with Azure storage solutions
Let's practice!
Understanding Microsoft Azure
Preparing Video For Download...