3

Forward and Backward Propagation in a Neural Network

Unsolved
Neural Networks

Difficulty: 7 | Problem written by ankita
Problem reported in interviews at

Amazon
Apple
Facebook
Google
Netflix

Neural Networks are trained by a feedforward process to calculate node activations progressively and backpropagation to calculate the derivative of the loss function with respect to each weight.

In this problem, you are required to implement the feed-forward and backpropagation algorithms of a custom-defined fully connected neural network.

The activation function must be sigmoid.

The loss function must be binary cross-entropy loss.

It is mandatory to initialize the weights and biases to zeros prior to training.

Input

X: A vector with training data values

y: Labels of the training data

l: list of number of neurons in each hidden layer

Output:

After one forward and backward pass on the complete data, return the derivative values of:

deriv_w: list of derivatives of the loss function with respect to weights in each hidden layer

deriv_b: list of derivatives of the loss function with respect to bias in each hidden layer

For example:

X = [[2.55337307, 1.52481329], [0.95618789, 1.22932837]]

y = [0,1]

l = [2] (one hidden layers with 2 neurons)

The last hidden layer is not specified but has to specified by the user as number of classed in y.

RESULT

deriv_w = [array([[0., 0.], [0., 0.]]), array([[ 0.25, -0.25], [ 0.25, -0.25]])]

deriv_b = [array([[0., 0.]]), array([[ 0.5, -0.5]])]

 

I would recommend watching this video (https://www.youtube.com/watch?v=x_Eamf8MHwU) by Andrew Ng to understand how to calculate derivatives with respect to weights and biases using delta matrix.

You can also refer given Wikipedia link: https://en.wikipedia.org/wiki/Backpropagation for a better understanding of the algorithm.

Sample Input:
<class 'list'>
X: [[2.55337307, 1.52481329], [0.95618789, 1.22932837], [0.75296472, 3.24716693], [-0.93797213, 1.26415069], [-0.39155179, 2.39860195], [0.71028504, 0.72796597], [-3.63427663, 1.35134052], [-0.24226226, -1.84743763], [2.96405404, -0.68521863], [-2.83764855, -1.61462203]]
<class 'list'>
y: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0]
<class 'list'>
l: [2]

Expected Output:
<class 'tuple'>
([array([[0., 0.], [0., 0.]]), array([[0.05, 0.05], [0.05, 0.05]])], [array([[0., 0.]]), array([[0.1, 0.1]])])

This is a premium problem, to view more details of this problem please sign up for MLPro Premium. MLPro premium offers access to actual machine learning and data science interview questions and coding challenges commonly asked at tech companies all over the world

MLPro Premium also allows you to access all our high quality MCQs which are not available on the free tier.

Not able to solve a problem? MLPro premium brings you access to solutions for all problems available on MLPro

Get access to Premium only exclusive educational content available to only Premium users.

Have an issue, the MLPro support team is available 24X7 to Premium users.

This is a premium feature.
To access this and other such features, click on upgrade below.

Log in to post a comment

Comments
Ready.

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)