Data Engineering Implications of the DevOps Architecture

DevOps Concepts

Cem Sakarya

DevOps Risk Advisor

Microservices architecture

A microservice comprising of the service database, microservice, and API.

Microservices are small-scale software programs
Microservices deployed separately
Each microservice take care of a different functionality
Each microservice has its own data and logic
They store the data in private databases

Monolithic architecture

Monolithic Architecture: User Interface, Single Unit Backend, Single Database

Opposite of the microservices architecture
A big single unit
Much simpler compared to microservices
Maintenance and change is very hard and risky

Monolithic Architecture

Limited number of databases
All application uses the same databases
Could be a viable option for small scale applications

Microservices Architecture

A database for each microservice
Microservices must do API calls to reach another services database
Microservices are effective in large organizations and complex products

Microservices private databases

Some product functionalities require collaboration from multiple microservices
Microservices can not access each others databases freely, so they do API calls
Microservices always communicate with each other.

Two microservices communicating to each other via APIs.

Data Engineering applications in microservices

Microservices communicate to each other via APIs. Also they interact with independent databases through data pipelines.

Data Engineering applications in microservices

Microservices communicate to each other via APIs. Also they interact with independent databases through data pipelines. Data pipelines are highlighted.

Let's practice!

DevOps Concepts