HW3 : Newton Method

2003-10-01

Requirement:

In Newton¡¯s method for local minimization, let the Hessian matrix be approximated by an identity matrix. The method then becomes a so-called gradient method. Consider a onedimensional function f . The gradient method searches for a local minimizer of f with the following iterative formula

x_k+1 = x_k ¨C ?_k g (x_k),

where xk and xk+1 are the iterates in kth and k+1th iterations, g (xk) is the first derivative of f at xk, and ?k is a parameter to be determined in every iteration so that the sequence of the iterates can be guaranteed to converge globally to a minimizer of f . Design a strategy for choosing ?k and use it to write a Matlab code for the one-dimensional version of the gradient method.

Test your program on the following problem

min f (x)
f (x) = 10*x*argtan (x) ¨C 5 ln (x² + 1)

with x₀ = 1 and x₀ = 2.0, and observe the convergence behavior of the algorithm with or without your global convergence strategy. Turn in your program and a short description on your test results.

Program ans Result:

           Main program : hw3.m
           Newton method: newtonmethod.m

the Basic Newton method doexn't converge at x = 2.0, while both Global Strategy 1 and Global Strategy 2 converge. All of them converge at x = 1.0. That proves the global strategies help to converge.

the two methods are discribed in  http://www.math.iastate.edu/wu/OptimizationConvergence.ppt

To test the work, run hw3
For the usage of the newton funtion, type : help newtonmethod