Non-negative matrix factorization (NMF)

Unsupervised Learning in Python

Benjamin Wilson

Director of Research at lateral.io

Non-negative matrix factorization

  • NMF = "non-negative matrix factorization"
  • Dimension reduction technique
  • NMF models are interpretable (unlike PCA)
  • Easy to interpret means easy to explain!
  • However, all sample features must be non-negative (>= 0)
Unsupervised Learning in Python

Interpretable parts

  • NMF expresses documents as combinations of topics (or "themes")

 

Text box: DataCamp is the first and foremost leader in Data Science Education offering skill-based training, pioneering technical innovation... approximately equals sign 0.6 times text box with words program, r, python, function, and method, plus 0.5 times text box with words data, analysis, cluster, statistics, mean plus 0.7 times text box with words teaching, learn, lesson, lessons, and course

Unsupervised Learning in Python

Interpretable parts

  • NMF expresses images as combinations of patterns

 

box with 3 stripes approximately equals 0.98 times box with one stripe plus 0.91 times box with a different stripe plus 0.95 times box with a different stripe

Unsupervised Learning in Python

Using scikit-learn NMF

  • Follows fit() / transform() pattern
  • Must specify number of components e.g. NMF(n_components=2)
  • Works with NumPy arrays and with csr_matrix
Unsupervised Learning in Python

Example word-frequency array

  • Word frequency array, 4 words, many documents
  • Measure presence of words in each document using "tf-idf"
    • "tf" = frequency of word in document
    • "idf" reduces influence of frequent words

Word frequency array

Unsupervised Learning in Python

Example usage of NMF

  • samples is the word-frequency array
from sklearn.decomposition import NMF

model = NMF(n_components=2)

model.fit(samples)

NMF(n_components=2)

nmf_features = model.transform(samples)
Unsupervised Learning in Python

NMF components

  • NMF has components
  • ... just like PCA has principal components
  • Dimension of components = dimension of samples
  • Entries are non-negative
print(model.components_)

[[ 0.01  0.    2.13  0.54]
 [ 0.99  1.47  0.    0.5 ]]
Unsupervised Learning in Python

NMF features

  • NMF feature values are non-negative
  • Can be used to reconstruct the samples
  • ... combine feature values with components
print(nmf_features)

[[ 0.    0.2 ]
 [ 0.19  0.  ]
  ...
 [ 0.15  0.12]]
Unsupervised Learning in Python

Reconstruction of a sample

print(samples[i,:])

[ 0.12  0.18  0.32  0.14]

print(nmf_features[i,:])

[ 0.15  0.12]

Components of NMF multiplied by feature value and added up

Unsupervised Learning in Python

Sample reconstruction

  • Multiply components by feature values, and add up
  • Can also be expressed as a product of matrices
  • This is the "Matrix Factorization" in "NMF"
Unsupervised Learning in Python

NMF fits to non-negative data only

  • Word frequencies in each document
  • Images encoded as arrays
  • Audio spectrograms
  • Purchase histories on e-commerce sites
  • ... and many more!
Unsupervised Learning in Python

Let's practice!

Unsupervised Learning in Python

Preparing Video For Download...