Category embeddings

Advanced Deep Learning with Keras

Zach Deane Mayer

Data Scientist

Category embeddings

Input: integers
Output: floats
Note: Increased dimensionality: output layer flattens back to 2D

Inputs

input_tensor = Input(shape=(1,))

Embedding Layer

from tensorflow.keras.layers import Embedding
input_tensor = Input(shape=(1,))
n_teams = 10887
embed_layer = Embedding(input_dim=n_teams,
                        input_length=1,
                        output_dim=1,
                        name='Team-Strength-Lookup')
embed_tensor = embed_layer(input_tensor)

Flattening

from tensorflow.keras.layers import Flatten
flatten_tensor = Flatten()(embed_tensor)

Put it all together

input_tensor = Input(shape=(1,))
n_teams =  10887
embed_layer = Embedding(input_dim=n_teams,
                        input_length=1,
                        output_dim=1,
                        name='Team-Strength-Lookup')
embed_tensor = embed_layer(input_tensor)
flatten_tensor = Flatten()(embed_tensor)
model = Model(input_tensor, flatten_tensor)

Let's practice!

Advanced Deep Learning with Keras