Deploying a model in Databricks

Databricks Concepts

Kevin Barlow

Data Practitioner

Machine Learning Lifecycle

¹ https://www.datacamp.com/blog/machine-learning-lifecycle-explained

Model Deployment and Operations

Concerns with deploying models

Availability

• How will my end users or application use the model?
• Where do I need to put my model to access it?
• Will the model be easy to understand or use?

Evaluation

• Are my users actually using my model?
• Is my model still performing well?
• Do I need to retrain my model?
• Do I need a new model that is better?

Model Deployment Process

Model Flavors

• MLFlow Models can store a model from any machine learning framework
• Models are stored alongside different configurations and artifacts
• Models can be "translated" into another kind of model based on needs. For example:
  • scikit-learn
  • pyfunc
  • spark
  • tensorflow

Model Registry

Model Registry

Model Registry

Model Registry

Model Serving

Model Serving

Model Serving

Model Serving

¹ https://www.databricks.com/product/model-serving

Let's practice!

Databricks Concepts