Compute cost function for the neural network

ex4/nnCostFunction.m

@@ -8,8 +8,8 @@ function [J grad] = nnCostFunction(nn_params, ...
 %   [J grad] = NNCOSTFUNCTON(nn_params, hidden_layer_size, num_labels, ...
 %   X, y, lambda) computes the cost and gradient of the neural network. The
 %   parameters for the neural network are "unrolled" into the vector
-%   nn_params and need to be converted back into the weight matrices. 
+%   nn_params and need to be converted back into the weight matrices.
-% 
+%
 %   The returned parameter grad should be a "unrolled" vector of the
 %   partial derivatives of the neural network.
 %
@@ -24,8 +24,8 @@ Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):en
 
 % Setup some useful variables
 m = size(X, 1);
-         
+
-% You need to return the following variables correctly 
+% You need to return the following variables correctly
 J = 0;
 Theta1_grad = zeros(size(Theta1));
 Theta2_grad = zeros(size(Theta2));
@@ -39,6 +39,24 @@ Theta2_grad = zeros(size(Theta2));
 %         cost function computation is correct by verifying the cost
 %         computed in ex4.m
 %
+
+X = [ones(m, 1), X];  % add a first colum of ones (bias term)
+A_2 = sigmoid(X*Theta1');
+A_2 = [ones(m, 1), A_2];  % (bias term)
+A_3 = sigmoid(A_2*Theta2');
+h_0 = A_3;
+%disp(round(h_0));
+
+% y is 1x5000 and holds the labels as numbers, turn it into 5000x10,
+% each row holding the label as vectors, e.g.  [0 1 0 0 0 ... ] for 2.
+y = eye(num_labels)(y,:);  % y is used as an index, it gets a row,
+                           % e.g. [0 0 0 1 ... 0 0]
+assert(size(y) == [m num_labels]);
+
+
+J = 1/m * sum(sum(-y.*log(h_0) - (1-y).*log(1-h_0)));
+assert(size(J) == [1 1]);
+
 % Part 2: Implement the backpropagation algorithm to compute the gradients
 %         Theta1_grad and Theta2_grad. You should return the partial derivatives of
 %         the cost function with respect to Theta1 and Theta2 in Theta1_grad and
@@ -46,12 +64,12 @@ Theta2_grad = zeros(size(Theta2));
 %         that your implementation is correct by running checkNNGradients
 %
 %         Note: The vector y passed into the function is a vector of labels
-%               containing values from 1..K. You need to map this vector into a 
+%               containing values from 1..K. You need to map this vector into a
 %               binary vector of 1's and 0's to be used with the neural network
 %               cost function.
 %
 %         Hint: We recommend implementing backpropagation using a for-loop
-%               over the training examples if you are implementing it for the 
+%               over the training examples if you are implementing it for the
 %               first time.
 %
 % Part 3: Implement regularization with the cost function and gradients.