Machine Translation with Keras
Thushan Ganegedara
Data Scientist and Author
Defining and using a Dense layer
dense = Dense(3, activation='softmax')
inp = Input(shape=(3,))
pred = dense(inp)
model = Model(inputs=inp, outputs=pred)
Defining a Dense layer with custom initialization
from tensorflow.keras.initializers import RandomNormal
init = RandomNormal(mean=0.0, stddev=0.05, seed=6000)
dense = Dense(3, activation='softmax',
kernel_initializer=init, bias_initializer=init)
(batch size, input size)
arrayx = [[1, 6, 8], [8, 9, 10]] # a 2x3 array
(batch size, num classes)
arrayy = [[0.1, 0.3, 0.4, 0.2], [0.2, 0.5, 0.1, 0.2]] # a 2x4 array
np.argmax(y, axis=-1)
np.argmax(y,axis=-1)
produces [2,1]
Dense
layers to process time-series inputsdense_time = TimeDistributed(Dense(3, activation='softmax'))
inp = Input(shape=(2, 3))
pred = dense_time(inp)
model = Model(inputs=inp, outputs=pred)
(batch size, sequence length, input size)
arrayx = [[[1, 6], [8, 2], [1, 2]],
[[8, 9], [10, 8], [1, 0]]] # a 2x3x2 array
(batch size, sequence length, num classes)
arrayy = [[[0.1, 0.5, 0.4], [0.8, 0.1, 0.1], [0.6, 0.2, 0.2]],
[[0.2, 0.5, 0.3], [0.2, 0.5, 0.3], [0.2, 0.8, 0.0]]] # a 2x3x3 array
np.argmax(y, axis=-1)
y = [[[0.1, 0.5, 0.4], [0.8, 0.1, 0.1], [0.6, 0.2, 0.2]],
[[0.2, 0.5, 0.3], [0.2, 0.5, 0.3], [0.2, 0.8, 0.0]]] # a 2x3x3 array
classes = np.argmax(y, axis=-1) # a 2 x 3 array
Iterating through time-distributed data
for t in range(3):
# Get the t-th time-dimension slice of y and classes
for prob, c in zip(y[:,t,:], classes[:,t]):
print("Prob: ", prob, ", Class: ", c)
Prob: [0.1 0.5 0.4] , Class: 1
Prob: [0.2 0.5 0.3] , Class: 1
Prob: [0.8 0.1 0.1] , Class: 0
...
Machine Translation with Keras