The Rosenbrock Function

Ronny Bergmann 2023-01-03

After loading the necessary packages

using Pkg;
cd(@__DIR__)
Pkg.activate("."); # use the example environment,

using Manifolds, Manopt, ManoptExamples
using Plots

We fix the parameters for the 📖 Rosenbrock (where the wikipedia page has a slightly different parameter naming).

a = 100.0
b = 1.0
p0 = [1/10, 2/10]

which is defined on $\mathbb R^2$, so we need

M = ℝ^2

Euclidean(2; field=ℝ)

and can then generate both the cost and the gradient

f = ManoptExamples.RosenbrockCost(M; a=a, b=b)
grad_f = ManoptExamples.RosenbrockGradient!!(M; a=a, b=b)

ManoptExamples.RosenbrockGradient!!{Float64}(100.0, 1.0)

For comparison, we look at the initial cost

f(M, p0)

4.42

And to illustrate, we run two small solvers with their default settings as a comparison.

Gradient Descent

We start with the gradient descent solver.

Since we need the state anyways to access the record, we also get from the return_state=true a short summary of the solver run.

gd_state = gradient_descent(M, f, grad_f, p0; record = [:Iteration, :Cost], return_state=true)

# Solver state for `Manopt.jl`s Gradient Descent
After 200 iterations

## Parameters
* retraction method: ManifoldsBase.ExponentialRetraction()

## Stepsize
ArmijoLinesearch(;
    initial_stepsize=1.0,
    retraction_method=ManifoldsBase.ExponentialRetraction(),
    contraction_factor=0.95,
    sufficient_decrease=0.1,
)

## Stopping criterion

Stop When _one_ of the following are fulfilled:
  * Max Iteration 200:  reached
  * |grad f| < 1.0e-8: not reached
Overall: reached
This indicates convergence: No

## Record
(Iteration = RecordGroup([RecordIteration(), RecordCost()]),)

From the summary we see, that the gradient is not yet small enough, but we hit the 200 iterations (default) iteration limit. Collecting the cost recording and printing the final cost

gd_x = get_record(gd_state, :Iteration, :Iteration)
gd_y =  get_record(gd_state, :Iteration, :Cost)
f(M, get_solver_result(gd_state))

0.10562873187751265

Quasi Newton

We can improve this using the quasi Newton algorithm

qn_state = quasi_Newton(M, f, grad_f, p0;
    record = [:Iteration, :Cost], return_state=true
)

# Solver state for `Manopt.jl`s Quasi Newton Method
After 26 iterations

## Parameters
* direction update:        limited memory Manopt.InverseBFGS (size 2) initial scaling 1.0and ParallelTransport() as vector transport.
* retraction method:       ManifoldsBase.ExponentialRetraction()
* vector transport method: ParallelTransport()

## Stepsize
WolfePowellLinesearch(;
    sufficient_descrease = 0.0001,
    sufficient_curvature = 0.999,
    retraction_method = ManifoldsBase.ExponentialRetraction(),
    vector_transport_method = ParallelTransport(),
    stop_when_stepsize_less = 1.0e-10,
)

## Stopping criterion

Stop When _one_ of the following are fulfilled:
  * Max Iteration 1000: not reached
  * |grad f| < 1.0e-6: reached
Overall: reached
This indicates convergence: No

## Record
(Iteration = RecordGroup([RecordIteration(), RecordCost()]),)

And we see it stops far earlier, after 45 Iterations. We again collect the recorded values

qn_x = get_record(qn_state, :Iteration, :Iteration)
qn_y =  get_record(qn_state, :Iteration, :Cost)
f(M, get_solver_result(qn_state))

1.4404666436813376e-18

and see that the final value is close to the one of the minimizer

f(M, ManoptExamples.minimizer(f))

0.0

which we also see if we plot the recorded cost.

fig = plot(gd_x, gd_y; linewidth=1, label="Gradient Descent");
plot!(fig, qn_x, qn_y; linewidth=1, label="Quasi Newton")

Technical Details

Status `~/work/ManoptExamples.jl/ManoptExamples.jl/examples/Project.toml`
  [6e4b80f9] BenchmarkTools v1.6.1
  [336ed68f] CSV v0.10.15
  [13f3f980] CairoMakie v0.15.6
  [0ca39b1e] Chairmarks v1.3.1
  [35d6a980] ColorSchemes v3.31.0
  [5ae59095] Colors v0.13.1
  [a93c6f00] DataFrames v1.8.0
  [31c24e10] Distributions v0.25.122
  [682c06a0] JSON v1.1.0
  [8ac3fa9e] LRUCache v1.6.2
  [b964fa9f] LaTeXStrings v1.4.0
  [d3d80556] LineSearches v7.4.0
  [ee78f7c6] Makie v0.24.6
  [af67fdf4] ManifoldDiff v0.4.5
  [1cead3c2] Manifolds v0.11.0
  [3362f125] ManifoldsBase v2.0.0
  [0fc0a36d] Manopt v0.5.25
  [5b8d5e80] ManoptExamples v0.1.17 `..`
  [51fcb6bd] NamedColors v0.2.3
  [91a5bcdd] Plots v1.41.1
  [08abe8d2] PrettyTables v3.1.0
  [6099a3de] PythonCall v0.9.28
  [f468eda6] QuadraticModels v0.9.14
  [1e40b3f8] RipQP v0.7.0

This tutorial was last rendered October 16, 2025, 12:14:3.