# Tucker manifold

`Manifolds.Tucker`

โ Type`Tucker{N, R, D, ๐ฝ} <: AbstractManifold{๐ฝ}`

The manifold of $N_1 \times \dots \times N_D$ real-valued or complex-valued tensors of fixed multilinear rank $(R_1, \dots, R_D)$ . If $R_1 = \dots = R_D = 1$, this is the Segre manifold, i.e., the set of rank-1 tensors.

**Representation in HOSVD format**

Let $\mathbb{F}$ be the real or complex numbers. Any tensor $p$ on the Tucker manifold can be represented as a multilinear product in HOSVD ^{[DeLathauwer2000]} form

\[p = (U_1,\dots,U_D) \cdot \mathcal{C}\]

where $\mathcal C \in \mathbb{F}^{R_1 \times \dots \times R_D}$ and, for $d=1,\dots,D$, the matrix $U_d \in \mathbb{F}^{N_d \times R_d}$ contains the singular vectors of the $d$th unfolding of $\mathcal{A}$

**Tangent space**

The tangent space to the Tucker manifold at $p = (U_1,\dots,U_D) \cdot \mathcal{C}$ is ^{[Koch2010]}

\[T_p \mathcal{M} = \bigl\{ (U_1,\dots,U_D) \cdot \mathcal{C}^\prime + \sum_{d=1}^D \bigl( (U_1, \dots, U_{d-1}, U_d^\prime, U_{d+1}, \dots, U_D) \cdot \mathcal{C} \bigr) \bigr\}\]

where $\mathcal{C}^\prime$ is arbitrary, $U_d^{\mathrm{H}}$ is the Hermitian adjoint of $U_d$, and $U_d^{\mathrm{H}} U_d^\prime = 0$ for all $d$.

**Constructor**

`Tucker(N::NTuple{D, Int}, R::NTuple{D, Int}[, field = โ])`

Generate the manifold of `field`

-valued tensors of dimensions `N[1] ร โฆ ร N[D]`

and multilinear rank `R = (R[1], โฆ, R[D])`

.

`Manifolds.TuckerPoint`

โ Type`TuckerPoint{T,D}`

An order `D`

tensor of fixed multilinear rank and entries of type `T`

, which makes it a point on the `Tucker`

manifold. The tensor is represented in HOSVD form.

**Constructors:**

`TuckerPoint(core::AbstractArray{T,D}, factors::Vararg{<:AbstractMatrix{T},D}) where {T,D}`

Construct an order `D`

tensor of element type `T`

that can be represented as the multilinear product `(factors[1], โฆ, factors[D]) โ
core`

. It is assumed that the dimensions of the core are the multilinear rank of the tensor and that the matrices `factors`

each have full rank. No further assumptions are made.

`TuckerPoint(p::AbstractArray{T,D}, mlrank::NTuple{D,Int}) where {T,D}`

The low-multilinear rank tensor arising from the sequentially truncated the higher-order singular value decomposition of the `D`

-dimensional array `p`

of type `T`

. The singular values are truncated to get a multilinear rank `mlrank`

^{[Vannieuwenhoven2012]}.

`Manifolds.TuckerTVector`

โ Type`TuckerTVector{T, D} <: TVector`

Tangent vector to the `D`

-th order `Tucker`

manifold at $p = (U_1,\dots,U_D) โ
\mathcal{C}$. The numbers are of type `T`

and the vector is represented as

\[X = (U_1,\dots,U_D) \cdot \mathcal{C}^\prime + \sum_{d=1}^D (U_1,\dots,U_{d-1},U_d^\prime,U_{d+1},\dots,U_D) \cdot \mathcal{C}\]

where $U_d^\mathrm{H} U_d^\prime = 0$.

**Constructor**

`TuckerTVector(Cโฒ::Array{T,D}, Uโฒ::NTuple{D,Matrix{T}}) where {T,D}`

Constructs a `D`

th order `TuckerTVector`

of number type `T`

with $C^\prime$ and $U^\prime$, so that, together with a `TuckerPoint`

$p$ as above, the tangent vector can be represented as $X$ in the above expression.

`Base.convert`

โ Method```
Base.convert(::Type{Matrix{T}}, basis::CachedBasis{๐ฝ,DefaultOrthonormalBasis{๐ฝ, TangentSpaceType},HOSVDBasis{T, D}}) where {๐ฝ, T, D}
Base.convert(::Type{Matrix}, basis::CachedBasis{๐ฝ,DefaultOrthonormalBasis{๐ฝ, TangentSpaceType},HOSVDBasis{T, D}}) where {๐ฝ, T, D}
```

Convert a HOSVD-derived cached basis from ^{[Dewaele2021]} of the `D`

th order `Tucker`

manifold with number type `T`

to a matrix. The columns of this matrix are the vectorisations of the `embed`

dings of the basis vectors.

`Base.foreach`

โ Function`Base.foreach(f, M::Tucker, p::TuckerPoint, basis::AbstractBasis, indices=1:manifold_dimension(M))`

Let `basis`

be and `AbstractBasis`

at a point `p`

on `M`

. Suppose `f`

is a function that takes an index and a vector as an argument. This function applies `f`

to `i`

and the `i`

th basis vector sequentially for each `i`

in `indices`

. Using a `CachedBasis`

may speed up the computation.

**NOTE**: The i'th basis vector is overwritten in each iteration. If any information about the vector is to be stored, `f`

must make a copy.

`Base.ndims`

โ Method`Base.ndims(p::TuckerPoint{T,D}) where {T,D}`

The order of the tensor corresponding to the `TuckerPoint`

`p`

, i.e., `D`

.

`Base.size`

โ Method`Base.size(p::TuckerPoint)`

The dimensions of a `TuckerPoint`

`p`

, when regarded as a full tensor (see `embed`

).

`ManifoldsBase.check_point`

โ Method`check_point(M::Tucker{N,R,D}, p; kwargs...) where {N,R,D}`

Check whether the multidimensional array or `TuckerPoint`

`p`

is a point on the `Tucker`

manifold, i.e. it is a `D`

th order `N[1] ร โฆ ร N[D]`

tensor of multilinear rank `(R[1], โฆ, R[D])`

. The keyword arguments are passed to the matrix rank function applied to the unfoldings. For a `TuckerPoint`

it is checked that the point is in correct HOSVD form.

`ManifoldsBase.check_vector`

โ Method`check_vector(M::Tucker{N,R,D}, p::TuckerPoint{T,D}, X::TuckerTVector) where {N,R,T,D}`

Check whether a `TuckerTVector`

`X`

is is in the tangent space to the `D`

th order `Tucker`

manifold `M`

at the `D`

th order `TuckerPoint`

`p`

. This is the case when the dimensions of the factors in `X`

agree with those of `p`

and the factor matrices of `X`

are in the orthogonal complement of the HOSVD factors of `p`

.

`ManifoldsBase.embed`

โ Method`embed(::Tucker{N,R,D}, p::TuckerPoint) where {N,R,D}`

Convert a `TuckerPoint`

`p`

on the rank `R`

`Tucker`

manifold to a full `N[1] ร โฆ ร N[D]`

-array by evaluating the Tucker decomposition.

`embed(::Tucker{N,R,D}, p::TuckerPoint, X::TuckerTVector) where {N,R,D}`

Convert a tangent vector `X`

with base point `p`

on the rank `R`

`Tucker`

manifold to a full tensor, represented as an `N[1] ร โฆ ร N[D]`

-array.

`ManifoldsBase.get_basis`

โ Method`get_basis(:: Tucker, p::TuckerPoint, basisType::DefaultOrthonormalBasis{๐ฝ, TangentSpaceType}) where ๐ฝ`

An implicitly stored basis of the tangent space to the Tucker manifold. Assume $p = (U_1,\dots,U_D) \cdot \mathcal{C}$ is in HOSVD format and that, for $d=1,\dots,D$, the singular values of the $d$'th unfolding are $\sigma_{dj}$, with $j = 1,\dots,R_d$. The basis of the tangent space is as follows: ^{[Dewaele2021]}

\[\bigl\{ (U_1,\dots,U_D) e_i \bigr\} \cup \bigl\{ (U_1,\dots, \sigma_{dj}^{-1} U_d^{\perp} e_i e_j^T,\dots,U_D) \cdot \mathcal{C} \bigr\}\]

for all $d = 1,\dots,D$ and all canonical basis vectors $e_i$ and $e_j$. Every $U_d^\perp$ is such that $[U_d \quad U_d^{\perp}]$ forms an orthonormal basis of $\mathbb{R}^{N_d}$.

`ManifoldsBase.inner`

โ Method`inner(M::Tucker, p::TuckerPoint, X::TuckerTVector, Y::TuckerTVector)`

The Euclidean inner product between tangent vectors `X`

and `X`

at the point `p`

on the Tucker manifold. This is equal to `embed(M, p, X) โ
embed(M, p, Y)`

.

```
inner(::Tucker, A::TuckerPoint, X::TuckerTVector, Y)
inner(::Tucker, A::TuckerPoint, X, Y::TuckerTVector)
```

The Euclidean inner product between `X`

and `Y`

where `X`

is a vector tangent to the Tucker manifold at `p`

and `Y`

is a vector in the ambient space or vice versa. The vector in the ambient space is represented as a full tensor, i.e., a multidimensional array.

`ManifoldsBase.inverse_retract`

โ Method`inverse_retract(M::Tucker, p::TuckerPoint, q::TuckerPoint, ::ProjectionInverseRetraction)`

The projection inverse retraction on the Tucker manifold interprets `q`

as a point in the ambient Euclidean space (see `embed`

) and projects it onto the tangent space at to `M`

at `p`

.

`ManifoldsBase.manifold_dimension`

โ Method`manifold_dimension(::Tucker{N,R,D}) where {N,R,D}`

The dimension of the manifold of $N_1 \times \dots \times N_D$ tensors of multilinear rank $(R_1, \dots, R_D)$, i.e.

\[\mathrm{dim}(\mathcal{M}) = \prod_{d=1}^D R_d + \sum_{d=1}^D R_d (N_d - R_d).\]

`ManifoldsBase.project`

โ Method`project(M::Tucker, p::TuckerPoint, X)`

The least-squares projection of a dense tensor `X`

onto the tangent space to `M`

at `p`

.

`ManifoldsBase.retract`

โ Method`retract(::Tucker, p::TuckerPoint, X::TuckerTVector, ::PolarRetraction)`

The truncated HOSVD-based retraction ^{[Kressner2014]} to the Tucker manifold, i.e. the result is the sequentially tuncated HOSVD approximation of $p + X$.

In the exceptional case that the multilinear rank of $p + X$ is lower than that of $p$, this retraction produces a boundary point, which is outside the manifold.

`ManifoldsBase.zero_vector`

โ Method`zero_vector(::Tucker, p::TuckerPoint)`

The zero element in the tangent space to `p`

on the `Tucker`

manifold, represented as a `TuckerTVector`

.

## Literature

- DeLathauwer2000
Lieven De Lathauwer, Bart De Moor, Joos Vandewalle: "A multilinear singular value decomposition" SIAM Journal on Matrix Analysis and Applications, 21(4), pp. 1253-1278, 2000 doi: 10.1137/S0895479896305696

- Koch2010
Othmar Koch, Christian Lubic, "Dynamical Tensor approximation" SIAM Journal on Matrix Analysis and Applications, 31(5), pp. 2360-2375, 2010 doi: 10.1137/09076578X

- Vannieuwenhoven2012
Nick Vannieuwenhoven, Raf Vandebril, Karl Meerbergen: "A new truncation strategy for the higher-order singular value decomposition" SIAM Journal on Scientific Computing, 34(2), pp. 1027-1052, 2012 doi: 10.1137/110836067

- Dewaele2021
Nick Dewaele, Paul Breiding, Nick Vannieuwenhoven, "The condition number of many tensor decompositions is invariant under Tucker compression" arxiv: 2106.13034

- Kressner2014
Daniel Kressner, Michael Steinlechner, Bart Vandereycken: "Low-rank tensor completion by Riemannian optimization" BIT Numerical Mathematics, 54(2), pp. 447-468, 2014 doi: 10.1007/s10543-013-0455-z