Your solution for Question 1 seems mostly correct, but there are some key points to review and verify for accuracy. I'll break down the process and the components to ensure everything aligns properly.

Breakdown:

1. Objective Function $E$ and its Derivative:

   • Your objective function $E$ is given by: $E(y_i) = \frac{c^2}{2} \log_e \left( \left( \frac{t_i - y_i}{c} + b \right)^2 \right) + \sum_k^w \frac{\left(w_k^2/\theta\right)}{1 + \left(w_k^2/\theta\right)}$
   • You correctly identified that the log property of $\log(x^2)$ simplifies to $2\log(x)$, which you used to rewrite the function.

2. Partial Derivative with Respect to $y_j$:

   • You found the partial derivative of the first term in the objective function with respect to $y_j$: $\frac{\partial E}{\partial y_j} = \frac{-c^2}{t_i - y_i + bc}$
   • This is accurate. Your handling of the constant terms and the application of the chain rule is correct.

3. Derivative of the Second Term:

   • Since the second term involves the weights $w_k$ and does not depend on $y_j$, you correctly concluded that its derivative with respect to $y_j$ is zero.

4. Derivative with Respect to $\text{net}_j$:

   • You applied the quotient rule to derive the function: $f(x) = \frac{x}{1 + |x|}$
   • The derivative you found: $\frac{\delta y_j}{\delta \text{net}_j} = \frac{1}{(1 + \text{net}_j)^2}$ is correct.

5. Derivative of $\text{net}_j$ with Respect to $w_{jk}$:

   • You identified that $\frac{\delta \text{net}_j}{\delta w_{jk}} = y_k$, which is also correct.

6. Putting it All Together:

   • You combined all the partial derivatives to express: $\frac{\delta E}{\delta w_{jk}} = \frac{-c^2 y_k}{(t_i - y_i + bc) (1 + \text{net}_i^{\text{out}})^2}$
   • This expression seems correct, and the final update rule for the weights $w_{jk}$ follows from here.

Final Steps:

You concluded with the weight update rule: $w_{jk}^{\text{new}} = w_{jk}^{\text{old}} + \eta \frac{c^2 y_k}{(t_i - y_i + bc)(1 + \text{net}_i^{\text{out}})^2}$ This result is correct, given the expressions you've derived earlier.

Summary:

• The logical steps and mathematical operations you've performed are mostly correct.
• However, ensure that each step, especially during the application of the chain rule and the quotient rule, is carried out precisely. The resulting formula you derived for updating the weights appears to be correct given the process you've outlined.

Would you like any specific details or further verification on any part of this derivation?

5 Related Questions:

1. How would changing the activation function $f(x)$ affect the derivatives in your solution?
2. Can you derive the weight update rule if the objective function $E$ were a different type of loss function, such as Mean Squared Error?
3. How does the learning rate $\eta$ influence the convergence of the weight update rule?
4. Could you explain how to generalize this derivative for a multi-layer neural network?
5. How would introducing regularization terms affect the derivatives and the weight update rule?

Tip:

Always double-check the differentiation steps, particularly when dealing with complex functions involving multiple variables and constants.