Dense layers

Introduction to TensorFlow in Python

Isaiah Hull

Visiting Associate Professor of Finance, BI Norwegian Business School

The linear regression model

The diagram shows a regression model with two features: marital status and age. The target of the regression model is the default status of a credit card borrower.

What is a neural network?

The diagram shows a neural network with two feature inputs: marital status and age. The target of the neural network is the default status of a credit card borrower.

What is a neural network?

This image shows a more complicated neural network with 10 input features, 3 dense hidden layers, and an output layer.

A dense layer applies weights to all nodes from the previous layer.

A simple dense layer

import tensorflow as tf

# Define inputs (features)
inputs = tf.constant([[1, 35]])

# Define weights
weights = tf.Variable([[-0.05], [-0.01]])

# Define the bias
bias = tf.Variable([0.5])

A simple dense layer

# Multiply inputs (features) by the weights
product = tf.matmul(inputs, weights)

# Define dense layer
dense = tf.keras.activations.sigmoid(product+bias)

This diagram shows a dense layer that takes two input nodes and produces one output.

Defining a complete model

import tensorflow as tf

# Define input (features) layer
inputs = tf.constant(data, tf.float32)

# Define first dense layer
dense1 = tf.keras.layers.Dense(10, activation='sigmoid')(inputs)

Defining a complete model

# Define second dense layer
dense2 = tf.keras.layers.Dense(5, activation='sigmoid')(dense1)

# Define output (predictions) layer
outputs =  tf.keras.layers.Dense(1, activation='sigmoid')(dense2)

This diagram shows a dense layer that takes two input nodes and produces one output.

High-level versus low-level approach

High-level approach
- High-level API operations

dense = keras.layers.Dense(10,\
 activation='sigmoid')

Low-level approach
- Linear-algebraic operations

prod = matmul(inputs, weights)
dense = keras.activations.sigmoid(prod)

Let's practice!

Introduction to TensorFlow in Python