How many parameters?
Image Modeling with Keras
Ariel Rokem
Senior Data Scientist, University of Washington
Counting parameters
model = Sequential()
model.add(Dense(10, activation='relu',
input_shape=(784,)))
model.add(Dense(10, activation='relu'))
model.add(Dense(3, activation='softmax'))
# Call the summary method
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 10) 7850
_________________________________________________________________
dense_2 (Dense) (None, 10) 110
_________________________________________________________________
dense_3 (Dense) (None, 3) 33
=================================================================
Total params: 7,993
Trainable params: 7,993
Non-trainable params: 0
_________________________________________________________________
Counting parameters
model.add(Dense(
10, activation='relu',
input_shape=(784,)))
model.add(Dense(
10, activation='relu'))
model.add(Dense(
3, activation='softmax'))
$$ parameters~=~ 784 * 10 + 10$$
$$= 7850 $$
$$ parameters~=~10 * 10 + 10$$
$$ = 110 $$
$$parameters~=~ 10 * 3 + 3$$
$$ = 33$$
$$ 7850 + 110 + 33 = 7993$$
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 10) 7850
_________________________________________________________________
dense_2 (Dense) (None, 10) 110
_________________________________________________________________
dense_3 (Dense) (None, 3) 33
=================================================================
Total params: 7,993
Trainable params: 7,993
Non-trainable params: 0
_________________________________________________________________
The number of parameters in a CNN
model = Sequential()
model.add(Conv2D(10, kernel_size=3, activation='relu',
input_shape=(28, 28, 1), padding='same'))
model.add(Conv2D(10, kernel_size=3, activation='relu',
padding='same'))
model.add(Flatten())
model.add(Dense(3, activation='softmax'))
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 28, 28, 10) 100
_________________________________________________________________
conv2d_2 (Conv2D) (None, 28, 28, 10) 910
_________________________________________________________________
flatten_3 (Flatten) (None, 7840) 0
_________________________________________________________________
dense_4 (Dense) (None, 3) 23523
=================================================================
Total params: 24,533
Trainable params: 24,533
Non-trainable params: 0
_________________________________________________________________
The number of parameters in a CNN
model.add( Conv2D(10, kernel_size=3, activation='relu', input_shape=(28, 28, 1), padding='same'))model.add( Conv2D(10, kernel_size=3, activation='relu', padding='same'))model.add(Flatten())
model.add(Dense(
3, activation='softmax'))
$$ parameters~=~ 9 * 10 + 10$$
$$= 100 $$
. $$ parameters~=~ 10 * 9 * 10 + 10 $$
$$ = 910 $$
$$parameters~=~0$$
$$parameters~=~ 7840 * 3 + 3$$
$$ = 23523$$
$$ 100 + 910 + 0 + 23523 = 24533$$
Increasing the number of units in each layer
model = Sequential()
model.add(Dense(5, activation='relu',
input_shape=(784,), padding='same'))
model.add(Dense(15, activation='relu', padding='same'))
model.add(Dense(3, activation='softmax'))
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 5) 3925
_________________________________________________________________
dense_2 (Dense) (None, 15) 90
_________________________________________________________________
dense_3 (Dense) (None, 3) 48
=================================================================
Total params: 4,063
Trainable params: 4,063
Non-trainable params: 0
_________________________________________________________________
Increasing the number of units in each layer
model = Sequential()
model.add(Conv2D(5, kernel_size=3, activation='relu',
input_shape=(28, 28, 1),
padding="same"))
model.add(Conv2D(15, kernel_size=3, activation='relu',
padding="same"))
model.add(Flatten())
model.add(Dense(3, activation='softmax'))
model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_12 (Conv2D) (None, 28, 28, 5) 50
_________________________________________________________________
conv2d_13 (Conv2D) (None, 28, 28, 15) 690
_________________________________________________________________
flatten_6 (Flatten) (None, 11760) 0
_________________________________________________________________
dense_9 (Dense) (None, 3) 35283
=================================================================
Total params: 36,023
Trainable params: 36,023
Non-trainable params: 0
_________________________________________________________________
Let's practice!
Image Modeling with Keras
