Introduction to PyTorch

Deep Learning with PyTorch

Ismail Elezi

Ph.D. Student of Deep Learning

Neural networks

Why PyTorch?

"PyThonic" - easy to use
Strong GPU support - models run fast
Many algorithms are already implemented
Automatic differentiation - more in next lesson
Similar to NumPy

Matrix Multiplication

PyTorch compared to NumPy

import torch
torch.tensor([[2, 3, 5], [1, 2, 9]])

tensor([[ 2,  3,  5],
        [ 1,  2,  9]])

torch.rand(2, 2)

tensor([[ 0.0374, -0.0936],
        [ 0.3135, -0.6961]])

a = torch.rand((3, 5))
a.shape

torch.Size([3, 5])

import numpy as np
np.array([[2, 3, 5], [1, 2, 9]])

array([[ 2,  3,  5],
       [ 1,  2,  9]])

np.random.rand(2, 2)

array([[ 0.0374, -0.0936],
       [ 0.3135, -0.6961]])

a = np.random.randn(3, 5)
a.shape

(3, 5)

Matrix operations

a = torch.rand((2, 2))
b = torch.rand((2, 2))

tensor([[-0.6110,  0.0145],
        [ 1.3583, -0.0921]])
tensor([[ 0.0673,  0.6419],
        [-0.0734,  0.3283]])

torch.matmul(a, b)

tensor([[-0.0422, -0.3875],
        [ 0.0981,  0.8417]])

a = np.random.rand(2, 2)
b = np.random.rand(2, 2)

array([[-0.6110,  0.0145],
        [ 1.3583, -0.0921]])
array([[ 0.0673,  0.6419],
        [-0.0734,  0.3283]])

np.dot(a, b)

array([[-0.0422, -0.3875],
       [ 0.0981,  0.8417]])

Matrix operations

a * b

tensor([[-0.0411,  0.0093],
        [-0.0998, -0.0302]])

np.multiply(a, b)

array([[-0.0411,  0.0093],
       [-0.0998, -0.0302]])

Zeros and Ones

a_torch = torch.zeros(2, 2)

tensor([[0., 0.],
        [0., 0.])

b_torch = torch.ones(2, 2)

tensor([[1., 1.],
        [1., 1.])

c_torch = torch.eye(2)

tensor([[1., 0.],
        [0., 1.]

a_numpy = np.zeros((2, 2))

array([[0., 0.],
       [0., 0.]])

b_numpy = np.ones((2, 2))

array([[1., 1.],
       [1., 1.]])

c_numpy = np.identity(2)

array([[1., 0.],
       [0., 1.]])

PyTorch to NumPy and vice versa

d_torch = torch.from_numpy(c_numpy)

tensor([[1., 0.],
        [0., 1.], 
        dtype=torch.float64)

d = c_torch.numpy()

array([[1., 0.],
       [0., 1.]])

Summary

torch.matmul(a, b)   # multiples torch tensors a and b

*                    # element-wise multiplication between two torch tensors

torch.eye(n)         # creates an identity torch tensor with shape (n, n)

torch.zeros(n, m)    # creates a torch tensor of zeros with shape (n, m)

torch.ones(n, m)     # creates a torch tensor of ones with shape (n, m)

torch.rand(n, m)     # creates a random torch tensor with shape (n, m)

torch.tensor(l)      # creates a torch tensor based on list l

Let's practice

Deep Learning with PyTorch