Debug output

DebugAction

A DebugAction is a small functor to print/issue debug output. The usual call is given by (p::AbstractManoptProblem, s::AbstractManoptSolverState, i) -> s, where i is the current iterate.

By convention i=0 is interpreted as "For Initialization only," only debug info that prints initialization reacts, i<0 triggers updates of variables internally but does not trigger any output.

Fields (assumed by subtypes to exist)

• print method to perform the actual print. Can for example be set to a file export,

or to @info. The default is the print function on the default Base.stdout.

DebugChange(M=DefaultManifold())

debug for the amount of change of the iterate (stored in get_iterate(o) of the AbstractManoptSolverState) during the last iteration. See DebugEntryChange for the general case

Keyword parameters

• storage: (StoreStateAction( [:Gradient] ) storage of the previous action
• prefix: ("Last Change:") prefix of the debug output (ignored if you set format)
• io: (stdout) default stream to print the debug to.
• format: ( "$prefix %f") format to print the output.
• inverse_retraction_method: (default_inverse_retraction_method(M)) the inverse retraction to be used for approximating distance.

DebugCost <: DebugAction

print the current cost function value, see get_cost.

Constructors

DebugCost()

Parameters

• format: ("$prefix %f") format to print the output
• io: (stdout) default stream to print the debug to.
• long: (false) short form to set the format to f(x): (default) or current cost: and the cost

DebugDivider <: DebugAction

print a small divider (default " | ").

Constructor

DebugDivider(div,print)

DebugEntry <: DebugAction

print a certain fields entry during the iterates, where a format can be specified how to print the entry.

Additional fields

• field: symbol the entry can be accessed with within AbstractManoptSolverState

Constructor

DebugEntry(f; prefix="$f:", format = "$prefix %s", io=stdout)

DebugEntryChange{T} <: DebugAction

print a certain entries change during iterates

Additional fields

• print: (print) function to print the result
• prefix: ("Change of :Iterate") prefix to the print out
• format: ("$prefix %e") format to print (uses the `prefix by default and scientific notation)
• field: Symbol the field can be accessed with within AbstractManoptSolverState
• distance: function (p,o,x1,x2) to compute the change/distance between two values of the entry
• storage: a StoreStateAction to store the previous value of :f

Constructors

DebugEntryChange(f,d)

Keyword arguments

• io: (stdout) an IOStream
• prefix: ("Change of $f")
• storage: (StoreStateAction((f,))) a StoreStateAction
• initial_value: an initial value for the change of o.field.
• format: ("$prefix %e") format to print the change

DebugEvery <: DebugAction

evaluate and print debug only every $i$th iteration. Otherwise no print is performed. Whether internal variables are updates is determined by always_update.

This method does not perform any print itself but relies on it's children's print.

It also sets the subsolvers active parameter, see |DebugWhenActive}(#ref). Here, the activattion_offset can be used to specify whether it refers to this iteration, the ith, when this call is before the iteration, then the offset should be 0, for the next iteration, that is if this is called after an iteration, it has to be set to 1. Since usual debug is happening after the iteration, 1 is the default.

Constructor

DebugEvery(d::DebugAction, every=1, always_update=true, activation_offset=1)

Initialise the DebugEvery.

DebugGradientChange()

debug for the amount of change of the gradient (stored in get_gradient(o) of the AbstractManoptSolverStateo) during the last iteration. See DebugEntryChange for the general case

Keyword parameters

• storage: (StoreStateAction( (:Gradient,) )) storage of the action for previous data
• prefix: ("Last Change:") prefix of the debug output (ignored if you set format)
• io: (stdout) default stream to print the debug to.
• format: ( "$prefix %f") format to print the output

DebugGroup <: DebugAction

group a set of DebugActions into one action, where the internal prints are removed by default and the resulting strings are concatenated

Constructor

DebugGroup(g)

construct a group consisting of an Array of DebugActions g, that are evaluated en bloque; the method does not perform any print itself, but relies on the internal prints. It still concatenates the result and returns the complete string

DebugIfEntry <: DebugAction

Issue a warning, info, or error if a certain field does not pass a the check.

The message is printed in this case. If it contains a @printf argument identifier, that one is filled with the value of the field. That way you can print the value in this case as well.

Fields

• io: an IO stream
• check: a function that takes the value of the field as input and returns a boolean
• field: symbol the entry can be accessed with within AbstractManoptSolverState
• msg: if the check fails, this message is displayed
• type: symbol specifying the type of display, possible values :print, : warn, :info, :error, where :print prints to io.

Constructor

DebugEntry(field, check=(>(0)); type=:warn, message=":$f is nonnegative", io=stdout)

DebugIterate <: DebugAction

debug for the current iterate (stored in get_iterate(o)).

Constructor

DebugIterate()

Parameters

• io: (stdout) default stream to print the debug to.
• format: ("$prefix %s") format how to print the current iterate
• long: (false) whether to have a long ("current iterate:") or a short ("p:") prefix
• prefix (see long for default) set a prefix to be printed before the iterate

DebugIteration <: DebugAction

Constructor

DebugIteration()

Keyword parameters

• format: ("# %-6d") format to print the output
• io: (stdout) default stream to print the debug to.

debug for the current iteration (prefixed with # by )

DebugMessages <: DebugAction

An AbstractManoptSolverState or one of its sub steps like a Stepsize might generate warnings throughout their computations. This debug can be used to :print them display them as :info or :warnings or even :error, depending on the message type.

Constructor

DebugMessages(mode=:Info, warn=:Once; io::IO=stdout)

Initialize the messages debug to a certain mode. Available modes are

• :Error: issue the messages as an error and hence stop at any issue occurring
• :Info: issue the messages as an @info
• :Print: print messages to the steam io.
• :Warning: issue the messages as a warning

The warn level can be set to :Once to only display only the first message, to :Always to report every message, one can set it to :No, to deactivate this, then this DebugAction is inactive. All other symbols are handled as if they were :Always:

DebugSolverState <: AbstractManoptSolverState

The debug state appends debug to any state, they act as a decorator pattern. Internally a dictionary is kept that stores a DebugAction for several occasions using a Symbol as reference.

The original options can still be accessed using the get_state function.

Fields

• options: the options that are extended by debug information
• debugDictionary: a Dict{Symbol,DebugAction} to keep track of Debug for different actions

Constructors

DebugSolverState(o,dA)

construct debug decorated options, where dD can be

• a DebugAction, then it is stored within the dictionary at :Iteration
• an Array of DebugActions.
• a Dict{Symbol,DebugAction}.
• an Array of Symbols, String and an Int for the DebugFactory

DebugStoppingCriterion <: DebugAction

print the Reason provided by the stopping criterion. Usually this should be empty, unless the algorithm stops.

Fields

• prefix: ("") format to print the output
• io: (stdout) default stream to print the debug to.

Constructor

DebugStoppingCriterion(prefix = ""; io::IO=stdout)

DebugTime()

Measure time and print the intervals. Using start=true you can start the timer on construction, for example to measure the runtime of an algorithm overall (adding)

The measured time is rounded using the given time_accuracy and printed after canonicalization.

Keyword parameters

• io: (stdout) default stream to print the debug to.
• format: ( "$prefix %s") format to print the output, where %s is the canonicalized time`.
• mode: (:cumulative) whether to display the total time or reset on every call using :iterative.
• prefix: ("Last Change:") prefix of the debug output (ignored if you set format)
• start: (false) indicate whether to start the timer on creation or not. Otherwise it might only be started on first call.
• time_accuracy: (Millisecond(1)) round the time to this period before printing the canonicalized time

DebugWarnIfCostIncreases <: DebugAction

print a warning if the cost increases.

Note that this provides an additional warning for gradient descent with its default constant step size.

Constructor

DebugWarnIfCostIncreases(warn=:Once; tol=1e-13)

Initialize the warning to warning level (:Once) and introduce a tolerance for the test of 1e-13.

The warn level can be set to :Once to only warn the first time the cost increases, to :Always to report an increase every time it happens, and it can be set to :No to deactivate the warning, then this DebugAction is inactive. All other symbols are handled as if they were :Always:

DebugWarnIfCostNotFinite <: DebugAction

A debug to see when a field (value or array within the AbstractManoptSolverState is or contains values that are not finite, for example Inf or Nan.

Constructor

DebugWarnIfCostNotFinite(field::Symbol, warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the cost is Nan. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always:

DebugWarnIfFieldNotFinite <: DebugAction

A debug to see when a field from the options is not finite, for example Inf or Nan

Constructor

DebugWarnIfFieldNotFinite(field::Symbol, warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the cost is Nan. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always:

Example

DebugWaranIfFieldNotFinite(:Gradient)

Creates a [DebugAction] to track whether the gradient does not get Nan or Inf.

DebugWarnIfGradientNormTooLarge{T} <: DebugAction

A debug to warn when an evaluated gradient at the current iterate is larger than (a factor times) the maximal (recommended) stepsize at the current iterate.

Constructor

DebugWarnIfGradientNormTooLarge(factor::T=1.0, warn=:Once)

Initialize the warning to warn :Once.

This can be set to :Once to only warn the first time the cost is Nan. It can also be set to :No to deactivate the warning, but this makes this Action also useless. All other symbols are handled as if they were :Always:

Example

DebugWaranIfFieldNotFinite(:Gradient)

Creates a [DebugAction] to track whether the gradient does not get Nan or Inf.

DebugWhenActive <: DebugAction

evaluate and print debug only if the active boolean is set. This can be set from outside and is for example triggered by DebugEvery on debugs on the subsolver.

This method does not perform any print itself but relies on it's children's prints.

For now, the main interaction is with DebugEvery which might activate or deactivate this debug

Fields

• active: a boolean that can (de-)activated from outside to turn on/off debug
• always_update: whether or not to call the order debugs with iteration -1 in active state

Constructor

DebugWhenActive(d::DebugAction, active=true, always_update=true)

Initialise the DebugSubsolver.

DebugActionFactory(s)

create a DebugAction where

• a Stringyields the corresponding divider
• a DebugAction is passed through
• a [Symbol] creates DebugEntry of that symbol, with the exceptions of :Change, :Iterate, :Iteration, and :Cost.
• a Tuple{Symbol,String} creates a DebugEntry of that symbol where the String specifies the format.

DebugActionFactory(s::Symbol)

Convert certain Symbols in the debug=[ ... ] vector to DebugActions Currently the following ones are done. Note that the Shortcut symbols should all start with a capital letter.

• :Cost creates a DebugCost
• :Change creates a DebugChange
• :Gradient creates a DebugGradient
• :GradientChange creates a DebugGradientChange
• :GradientNorm creates a DebugGradientNorm
• :Iterate creates a DebugIterate
• :Iteration creates a DebugIteration
• :IterativeTime creates a DebugTime(:Iterative)
• :Stepsize creates a DebugStepsize
• :Stop creates a StoppingCriterion()
• :WarnCost creates a DebugWarnIfCostNotFinite
• :WarnGradient creates a DebugWarnIfFieldNotFinite for the ::Gradient.
• :WarnBundle creates a DebugWarnIfLagrangeMultiplierIncreases
• :Time creates a DebugTime
• :WarningMessagescreates a DebugMessages(:Warning)
• :InfoMessagescreates a DebugMessages(:Info)
• :ErrorMessages creates a DebugMessages(:Error)
• :Messages creates a DebugMessages() (the same as :InfoMessages)

any other symbol creates a DebugEntry(s) to print the entry (o.:s) from the options.

DebugActionFactory(t::Tuple{Symbol,String)

Convert certain Symbols in the debug=[ ... ] vector to DebugActions Currently the following ones are done, where the string in t[2] is passed as the format the corresponding debug. Note that the Shortcut symbols t[1] should all start with a capital letter.

• :Change creates a DebugChange
• :Cost creates a DebugCost
• :Gradient creates a DebugGradient
• :GradientChange creates a DebugGradientChange
• :GradientNorm creates a DebugGradientNorm
• :Iterate creates a DebugIterate
• :Iteration creates a DebugIteration
• :Stepsize creates a DebugStepsize
• :Stop creates a DebugStoppingCriterion
• :Time creates a DebugTime
• :IterativeTime creates a DebugTime(:Iterative)

any other symbol creates a DebugEntry(s) to print the entry (o.:s) from the options.

DebugFactory(a::Vector)

Generate a dictionary of DebugActions.

First all Symbols String, DebugActions and numbers are collected, excluding :Stop and :Subsolver. This collected vector is added to the :Iteration => [...] pair. :Stop is added as :StoppingCriterion to the :Stop => [...] pair. If necessary, these pairs are created

For each Pair of a Symbol and a Vector, the DebugGroupFactory is called for the Vector and the result is added to the debug dictonaries entry with said symbold. This is wrapped into the DebugWhenActive, when the :Subsolver symbol is present

Return value

A dictionary for the different enrty points where debug can happen, each containing a DebugAction to call.

Note that upon the initialisation all dictionaries but the :StartAlgorithm one are called with an i=0 for reset.

Examples

1. Providing a simple vector of symbols, numbers and strings like

[:Iterate, " | ", :Cost, :Stop, 10]

Adds a group to :Iteration of three actions (DebugIteration, DebugDivider(" | "), and[DebugCost](@ref)) as a [DebugGroup](@ref) inside an [DebugEvery](@ref) to only be executed every 10th iteration. It also adds the [DebugStoppingCriterion](@ref) to the:EndAlgorhtm` entry of the dictionary.

1. The same can also be written a bit more precise as

DebugFactory([:Iteration => [:Iterate, " | ", :Cost, 10], :Stop])

1. We can even make the stoping criterion concrete and pass Actions directly,

for example explicitly Making the stop more concrete, we get

DebugFactory([:Iteration => [:Iterate, " | ", DebugCost(), 10], :Stop => [:Stop]])

DebugGroupFactory(a::Vector)

Generate a [DebugGroup] of DebugActions. The following rules are used

1. Any Symbol is passed to DebugActionFactory
2. Any (Symbol, String) generates similar actions as in 1., but the string is used for format=`,

see DebugActionFactory

1. Any String is passed to DebugActionFactory(d::String)](@ref)`
2. Any DebugAction is included as is.

If this results in more than one DebugAction a DebugGroup of these is build.

If any integers are present, the last of these is used to wrap the group in a DebugEvery(k).

If :SubSolver is present, the resulting Action is wrappedn in DebugWhenActive, making it deactivatable by its parent solver.

reset!(d::DebugTime)

reset the internal time of a DebugTime, that is start from now again.

set_manopt_parameter!(ams::DebugSolverState, ::Val{:Debug}, args...)

Set certain values specified by args... into the elements of the debugDictionary

stop!(d::DebugTime)

stop the reset the internal time of a DebugTime, that is set the time to 0 (undefined)

initialize_solver!(amp::AbstractManoptProblem, dss::DebugSolverState)

Extend the initialization of the solver by a hook to run the DebugAction that was added to the :Start entry of the debug lists. All others are triggered (with iteration number 0) to trigger possible resets

step_solver!(amp::AbstractManoptProblem, dss::DebugSolverState, i)

Extend the ith step of the solver by a hook to run debug prints, that were added to the :BeforeIteration and :Iteration entries of the debug lists.

stop_solver!(amp::AbstractManoptProblem, dss::DebugSolverState, i)

Extend the stop_solver!, whether to stop the solver by a hook to run debug, that were added to the :Stop entry of the debug lists.