Encoder decoder architecture

Machine Translation with Keras

Thushan Ganegedara

Data Scientist and Author

Encoder decoder model

Machine translation model

Machine Translation with Keras

Encoder

Encoder

Machine Translation with Keras

Encoder and Decoder

Encoder decoder

Machine Translation with Keras

Analogy: Encoder decoder architecture

Encoder analogy

Decoder analogy

Machine Translation with Keras

Reversing sentences - encoder decoder model

Encoder decoder text reverse

Machine Translation with Keras

Writing the encoder

def words2onehot(word_list, word2index):
  word_ids = [word2index[w] for w in word_list]
  onehot = to_categorical(word_ids, 3)
  return onehot

def encoder(onehot):
  word_ids = np.argmax(onehot, axis=1)
  return word_ids
Machine Translation with Keras

Writing the encoder

onehot = words2onehot(["I", "like", "cats"], word2index)
context = encoder(onehot)
print(context)

[0, 1, 2]
Machine Translation with Keras

Writing the decoder

  • Decoder: Word IDs ? Reverse the IDs ? one-hot vectors
def decoder(context_vector):
  word_ids_rev = context_vector[::-1]
  onehot_rev = to_categorical(word_ids_rev, 3)
  return onehot_rev
  • Helper function: convert one-hot vectors to human readable words
def onehot2words(onehot, index2word):
  ids = np.argmax(onehot, axis=1)
  return [index2word[id] for id in ids]
Machine Translation with Keras

Writing the decoder

onehot_rev = decoder(context)
reversed_words = onehot2words(onehot_rev, index2word)

print(reversed_words)

['cats', 'like', 'I']
Machine Translation with Keras

Let's practice!

Machine Translation with Keras

Preparing Video For Download...