Four matlab m-files are provided that implement the N-GMRES Nonlinear GMRES optimization method that was proposed and analyzed in references [1] and [2] below, and test the method for a standard continuous optimization problem and a CP tensor decomposition problem. (Note that some of these files use functions from Sandia's Tensor and Poblano toolboxes, see below.)

Matlab code

1. ngmres.m: This function contains a baseline implementation of the N-GMRES algorithm (no dependencies on Tensor toolbox or Poblano toolbox).
2. ngmres_test_general.m: A test script that illustrates how ngmres.m (with steepest descent preconditioner) can be used for a general nonlinear optimization problem (with comparison to NCG, LBFGS, and steepest descent) (with dependencies on Poblano toolbox for linesearch and for comparison with NCG and LBFGS).
3. ngmres_test_tensor_CP.m: A test script that illustrates how ngmres.m can be used for a tensor CP decomposition problem (Tomasi & Bro test problem; with comparison to NCG, LBFGS, and steepest descent) (with dependencies on Tensor toolbox and Poblano toolbox).
4. cp_ngmres.m: This function calls ngmres.m (with ALS preconditioner) to compute a CP decomposition of a given tensor (with dependencies on Tensor toolbox and Poblano toolbox). It complements the cp_opt function in the Tensor toolbox, and is used in ngmres_test_tensor_CP.m.

ngmres_opt.zip: zipped folder with the above files (1-4)

April 2014: These files were tested with the current stable/released versions of the Tensor toolbox (version 2.5) and the Poblano toolbox.

References

[1] Hans De Sterck, "A Nonlinear GMRES Optimization Algorithm for Canonical Tensor Decomposition", SIAM J. Sci. Comput. 34, A1351-A1379, 2012. [pdf]

[2] Hans De Sterck, "Steepest Descent Preconditioning for Nonlinear GMRES Optimization", Numerical Linear Algebra with Applications 20, 453-471, 2013. [pdf]