Parallel computing

Capire il Data Engineering

Hadrien Lacroix

Content Developer at DataCamp

Basis of modern data processing tools
Necessary:
- Mainly because of memory
- Also for processing power
How it works:
- Split tasks up into several smaller subtasks
- Distribute these subtasks over several computers

1000 t shirts

senior sales assistant

junior sales assistant

¹ Emojis by Mohamed Hassan

one sales assistant at at time

batching t-shirts

junior sales assistants finishing in one hour and fifteen minutes

senior sales assistants finishing in two hours and thirteen minutes

Benefits and risks of parallel computing

comparing junior and senior sales assistant performance

it takes ten minutes to distribute the one thousand t-shirts to the four junior assistants

it takes five minutes to gather the t-shirts from the four junior assistants into one pile

data pipeline

Capire il Data Engineering