Neural networks and layers

Introduction to Deep Learning with PyTorch

Jasmin Ludolf

Senior Data Science Content Developer, DataCamp

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Introduction to Deep Learning with PyTorch

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Introduction to Deep Learning with PyTorch

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Introduction to Deep Learning with PyTorch

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Introduction to Deep Learning with PyTorch

Our first neural network

A diagram representing a basic neural network with only input and output

  • Fully connected network
  • Equivalent to a linear model
Introduction to Deep Learning with PyTorch

Designing a neural network

Three nodes representing the input to a neural network

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features input_tensor = torch.tensor( [[0.3471, 0.4547, -0.2356]])
  • Input neurons = features
  • Output neurons = classes
Introduction to Deep Learning with PyTorch

Designing a neural network

Three nodes representing the input to a neural network with arrows pointing forward to represent the linear layer

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features input_tensor = torch.tensor( [[0.3471, 0.4547, -0.2356]])
# Define our linear layer linear_layer = nn.Linear(
in_features=3,
out_features=2
)
Introduction to Deep Learning with PyTorch

Designing a neural network

A diagram representing a basic neural network with only input and output

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features input_tensor = torch.tensor( [[0.3471, 0.4547, -0.2356]])
# Define our linear layer linear_layer = nn.Linear( in_features=3, out_features=2 )
# Pass input through linear layer output = linear_layer(input_tensor) print(output)
Introduction to Deep Learning with PyTorch

Designing a neural network

A diagram representing a basic neural network with only input and output

$$ $$ $$

# Pass input through linear layer
output = linear_layer(input_tensor)
print(output)
tensor([[-0.2415, -0.1604]], 
    grad_fn=<AddmmBackward0>)
Introduction to Deep Learning with PyTorch

Weights and biases

output = linear_layer(input_tensor)

Linear operation representation

Introduction to Deep Learning with PyTorch

Weights and biases

  • .weight
    print(linear_layer.weight)
    
Parameter containing:
tensor([[-0.4799,  0.4996,  0.1123],
        [-0.0365, -0.1855,  0.0432]], 
        requires_grad=True)

$$

  • Reflects the importance of different features
  • .bias
    print(linear_layer.bias)
    
Parameter containing:
tensor([0.0310, 0.1537], requires_grad=True)

$$

$$

  • Provides the neuron with a baseline output
Introduction to Deep Learning with PyTorch

A fully connected network in action

A diagram representing a basic neural network with only input and output

Introduction to Deep Learning with PyTorch

A fully connected network in action

A diagram representing a basic neural network with only input and output

Introduction to Deep Learning with PyTorch

A fully connected network in action

A diagram representing a basic neural network with only input and output

  • Humidity feature will have a more significant weight
  • Bias is to account for baseline information
Introduction to Deep Learning with PyTorch

Let's practice!

Introduction to Deep Learning with PyTorch

Preparing Video For Download...