Blogs/Predicting with a Neural Network

Predicting with a Neural Network

peterwashington Nov 02 2021 11 min read 0 views
Neural Networks
NeuralNetPrediction1.png

Neural networks are made up of nodes and edges. Nodes are analogous to neurons in the brain, and edges are analogous to synapses (the structures in the brain connecting neurons and sending electrical or chemical signals from neuron to neuron). Nodes are connected by edges (also called weights), and they are often visualized like this:

The first set of nodes, on the leftmost side of the neural network, are collectively called the input layer. The last set of nodes, on the rightmost side of the neural network, are collectively called the output layer. All other layers in between are called hidden layers.

After a neural network is trained, the model will have learned values for all of its weights. A trained neural network could end up with the following weights, for example:

To calculate the value of a node in a hidden layer or output layer, you multiply the weight of the arrow coming into that node with the value of the node that arrow is coming from, for each arrow pointing to the node, and add up all of these products. For example, let’s say we want to calculate the value of the hidden node below given the prior layer’s node values and weight values:

We calculate the value of the new node as follows:

8 × -4+3 × -1+6 × 2= -23

The edges correspond to the m variables and the nodes correspond to the x variables in mx + b. In this way, the edges are the learned parameters of the neural network.

Nodes also have a value for b, which we call the bias parameter. For each node, we could write the value of the node y as y = m1x1 + m2x2 + … + mNxN + b.

For simplicity in the following examples in this chapter, we will assume that the bias value (the value of b) is set to 0.

The definition of a neural network we have covered so far limits the learned equation to be a linear equation. Let’s say we have this neural network, where the vector [x1, x2, x3] is the input to the neural network and y is the output of the network:

We get the following equation for y (remember that we are setting the bias to 0 for simplicity):

To get the equation for y in terms of the input nodes, we must write out the equation for x4 and x5:

The equation for y in terms of the input nodes is obtained by plugging in these equations for x4 and x5 back into the equation for y (again, remember that we are setting the bias to 0 for simplicity):

This is just a linear equation of the inputs. We might as well just use linear or logistic regression. So why are these more complicated neural networks so popular today? The reason comes down to activation functions. We will cover this topic in the next blog.