Getting started with Databricks

Databricks Concepts

Kevin Barlow

Data Practitioner

Compute cluster refresh

Cloud computing diagram

Databricks Concepts

Create your first cluster

The first step is to create a cluster for your data processing!

Configuration options:

Cluster UI

Databricks Concepts

Create your first cluster

The first step is to create a cluster for your data processing!

Configuration options:

  • Cluster policies and access

Cluster UI - Access

Databricks Concepts

Cluster Access

Cluster Access Diagram

Databricks Concepts

Create your first cluster

The first step is to create a cluster for your data processing!

Configuration options:

  • Cluster policies and access
  • Databricks Runtime
  • Photon Acceleration

Cluster UI - Runtime

Databricks Concepts

Create your first cluster

The first step is to create a cluster for your data processing!

Configuration options:

  • Cluster policies and access
  • Databricks Runtime
  • Photon Acceleration
  • Node instance types and number
  • Auto-scaling / Auto-termination

Cluster UI - Nodes

Databricks Concepts

Data Explorer

Get familiar with the Data Explorer! In this UI, you can:

  1. Browse available catalogs/schemas/tables
  2. Look at sample data and summary statistics
  3. View data lineage and history

You can also upload new data by clicking the "plus" icon!

Data Exploration

1 Photo by Jakub Zerdzicki: https://www.pexels.com/photo/magnifier-loupe-17284804/
Databricks Concepts

Create a notebook

Databricks notebooks:

  • Standard interface for Databricks
  • Improvements on open-source Jupyter
  • Support for many languages
    • Python, R, Scala, SQL
    • Magic commands (%sql)
  • Built-in visualizations
  • Real-time commenting and collaboration

Notebook Creation in Workspace

Databricks Concepts

Let's practice!

Databricks Concepts

Preparing Video For Download...