Limitations of bigmemory

Pemrosesan Data yang Dapat Diskalakan di R

Michael Kane

Assistant Professor, Yale University

Where can you use bigmemory?

  • You can use bigmemory when your data are
    • matrices
    • dense
    • numeric
  • Underlying data structures are compatible with low-level linear algebra libraries for fast model fitting
  • If you have different column types, you could try the ff package
Pemrosesan Data yang Dapat Diskalakan di R

Understanding disk access

A big.matrix is a data structure designed for random access

Pemrosesan Data yang Dapat Diskalakan di R

Disadvantages of random access

  • Can't add rows or columns to an existing big.matrix object
  • You need to have enough disk space to hold the entire matrix in one big block
Pemrosesan Data yang Dapat Diskalakan di R

Let's practice!

Pemrosesan Data yang Dapat Diskalakan di R

Preparing Video For Download...