ALS parameters and hyperparameters

Créer des moteurs de recommandation avec PySpark

Jamen Long

Data Scientist at Nike

Example ALS model code

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Column names

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Arguments

userCol: Name of column that contains user id's
itemCol: Name of column that contains item id's
ratingCol: Name of column that contains ratings

blank original ratings matrix with two blank factor matrices

same image as previous with latent dimensions highlighted

Rank

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Hyperparameters

rank, $k$: number of latent features

MaxIter

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Hyperparameters

rank, $k$: number of latent features
maxIter: number of iterations

RegParam

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Hyperparameters

rank, $k$: number of latent features
maxIter: number of iterations
regParam: Lambda

Alpha

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Hyperparameters

rank, $k$: number of latent features
maxIter: number of iterations
regParam: Lambda
alpha: Discussed later. Only used with implicit ratings.

Non-negative

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Additional Arguments

nonnegative = True: Ensures positive numbers

Cold start strategy

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Additional Arguments

nonnegative = True: Ensures positive numbers
coldStartStrategy = "drop": Addresses issues with test/train split

Implicit preferences

als_model = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, alpha=40,
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Additional Arguments

nonnegative = True: Ensures positive numbers
coldStartStrategy = "drop": Addresses issues with test/train split
implicitPrefs = True: True/False depending on ratings type

Sample ALS model build

als = ALS(userCol="userId", itemCol="movieId", ratingCol="rating", 
            rank=25, maxIter=100, regParam=.05, 
            nonnegative=True,
            coldStartStrategy="drop",
            implicitPrefs=False)

Fit and transform methods

# Fit ALS to training dataset
model = als.fit(training_data)

# Generate predictions on test dataset
predictions = model.transform(test_data)

Let's practice!

Créer des moteurs de recommandation avec PySpark