Congratulations!

Pemrosesan Data yang Dapat Diskalakan di R

Michael J. Kane and Simon Urbanek

Instructors, DataCamp

Split-Apply-Combine

  • Break the data into parts
  • Compute on the parts
  • Combine the results
Pemrosesan Data yang Dapat Diskalakan di R

Split-Apply-Combine: Advantages

  • Manageable parts don't overwhelm your computer
  • Approach is easy to parallelize
  • Process sequentially
  • Process on serveral machines in a cluster
Pemrosesan Data yang Dapat Diskalakan di R

Split-Apply-Combine: R

  • split() partitions set of row numbers or data.frame

  • Map() computes on parts

  • Reduce() combines results

Pemrosesan Data yang Dapat Diskalakan di R

bigmemory

bigmemory

  • Good for larger data sets that can be represented as dense matrices and might be too big for RAM
  • Looks like a regular R matrix
Pemrosesan Data yang Dapat Diskalakan di R

iotools

iotools

  • Good for much larger data that can be processed in sequential chunks
  • Supports data.frame and matrix
Pemrosesan Data yang Dapat Diskalakan di R

Pemrosesan Data yang Dapat Diskalakan di R

Good luck!

Pemrosesan Data yang Dapat Diskalakan di R

Preparing Video For Download...