Neural networks and layers

Introduction to Deep Learning with PyTorch

Jasmin Ludolf

Senior Data Science Content Developer, DataCamp

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Neural network layers

A diagram representing a neural network with input, hidden layer and output

Our first neural network

A diagram representing a basic neural network with only input and output

Fully connected network
Equivalent to a linear model

Designing a neural network

Three nodes representing the input to a neural network

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features
input_tensor = torch.tensor(
    [[0.3471, 0.4547, -0.2356]])

Input neurons = features
Output neurons = classes

Designing a neural network

Three nodes representing the input to a neural network with arrows pointing forward to represent the linear layer

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features
input_tensor = torch.tensor(
    [[0.3471, 0.4547, -0.2356]])


# Define our linear layer
linear_layer = nn.Linear(

                         in_features=3,

                         out_features=2

                         )

Designing a neural network

A diagram representing a basic neural network with only input and output

# Importing as nn to avoid writing torch.nn
import torch.nn as nn


# Create input_tensor with three features
input_tensor = torch.tensor(
    [[0.3471, 0.4547, -0.2356]])


# Define our linear layer
linear_layer = nn.Linear(
                         in_features=3, 
                         out_features=2
                         )


# Pass input through linear layer
output = linear_layer(input_tensor)
print(output)

Designing a neural network

A diagram representing a basic neural network with only input and output

$$ $$ $$

# Pass input through linear layer
output = linear_layer(input_tensor)
print(output)

tensor([[-0.2415, -0.1604]], 
    grad_fn=<AddmmBackward0>)

Weights and biases

output = linear_layer(input_tensor)

Linear operation representation

Weights and biases

.weight
```
print(linear_layer.weight)
```

Parameter containing:
tensor([[-0.4799,  0.4996,  0.1123],
        [-0.0365, -0.1855,  0.0432]], 
        requires_grad=True)

Reflects the importance of different features

.bias
```
print(linear_layer.bias)
```

Parameter containing:
tensor([0.0310, 0.1537], requires_grad=True)

Provides the neuron with a baseline output

A fully connected network in action

A diagram representing a basic neural network with only input and output

A fully connected network in action

A diagram representing a basic neural network with only input and output

A fully connected network in action

A diagram representing a basic neural network with only input and output

Humidity feature will have a more significant weight
Bias is to account for baseline information

Let's practice!

Introduction to Deep Learning with PyTorch