# Tucker manifold

`Manifolds.Tucker`

— Type`Tucker{T, D, 𝔽} <: AbstractManifold{𝔽}`

The manifold of $N_1×\dots×N_D$ real-valued or complex-valued tensors of fixed multilinear rank $(R_1, \dots, R_D)$ . If $R_1 = \dots = R_D = 1$, this is the Segre manifold, i.e., the set of rank-1 tensors.

**Representation in HOSVD format**

Let $𝔽$ be the real or complex numbers. Any tensor $p$ on the Tucker manifold can be represented as a multilinear product in HOSVD [LMV00] form

\[p = (U_1,\dots,U_D) ⋅ \mathcal{C}\]

where $\mathcal C \in 𝔽^{R_1×\dots×R_D}$ and, for $d=1,\dots,D$, the matrix $U_d \in 𝔽^{N_d×R_d}$ contains the singular vectors of the $d$th unfolding of $\mathcal{A}$

**Tangent space**

The tangent space to the Tucker manifold at $p = (U_1,\dots,U_D) ⋅ \mathcal{C}$ is [KL10]

\[T_p \mathcal{M} = \bigl\{ (U_1,\dots,U_D) ⋅ \mathcal{C}^\prime + \sum_{d=1}^D \bigl( (U_1, \dots, U_{d-1}, U_d^\prime, U_{d+1}, \dots, U_D) ⋅ \mathcal{C} \bigr) \bigr\}\]

where $\mathcal{C}^\prime$ is arbitrary, $U_d^{\mathrm{H}}$ is the Hermitian adjoint of $U_d$, and $U_d^{\mathrm{H}} U_d^\prime = 0$ for all $d$.

**Constructor**

`Tucker(N::NTuple{D, Int}, R::NTuple{D, Int}[, field=ℝ]; parameter::Symbol=:type)`

Generate the manifold of `field`

-valued tensors of dimensions `N[1] × … × N[D]`

and multilinear rank `R = (R[1], …, R[D])`

.

`Manifolds.TuckerPoint`

— Type`TuckerPoint{T,D}`

An order `D`

tensor of fixed multilinear rank and entries of type `T`

, which makes it a point on the `Tucker`

manifold. The tensor is represented in HOSVD form.

**Constructors:**

`TuckerPoint(core::AbstractArray{T,D}, factors::Vararg{<:AbstractMatrix{T},D}) where {T,D}`

Construct an order `D`

tensor of element type `T`

that can be represented as the multilinear product `(factors[1], …, factors[D]) ⋅ core`

. It is assumed that the dimensions of the core are the multilinear rank of the tensor and that the matrices `factors`

each have full rank. No further assumptions are made.

`TuckerPoint(p::AbstractArray{T,D}, mlrank::NTuple{D,Int}) where {T,D}`

The low-multilinear rank tensor arising from the sequentially truncated the higher-order singular value decomposition of the `D`

-dimensional array `p`

of type `T`

. The singular values are truncated to get a multilinear rank `mlrank`

[VVM12].

`Manifolds.TuckerTVector`

— Type`TuckerTVector{T, D} <: TVector`

Tangent vector to the `D`

-th order `Tucker`

manifold at $p = (U_1,\dots,U_D) ⋅ \mathcal{C}$. The numbers are of type `T`

and the vector is represented as

\[X = (U_1,\dots,U_D) ⋅ \mathcal{C}^\prime + \sum_{d=1}^D (U_1,\dots,U_{d-1},U_d^\prime,U_{d+1},\dots,U_D) ⋅ \mathcal{C}\]

where $U_d^\mathrm{H} U_d^\prime = 0$.

**Constructor**

`TuckerTVector(C′::Array{T,D}, U′::NTuple{D,Matrix{T}}) where {T,D}`

Constructs a `D`

th order `TuckerTVector`

of number type `T`

with $C^\prime$ and $U^\prime$, so that, together with a `TuckerPoint`

$p$ as above, the tangent vector can be represented as $X$ in the above expression.

`Base.convert`

— Method```
Base.convert(::Type{Matrix{T}}, basis::CachedBasis{𝔽,DefaultOrthonormalBasis{𝔽, TangentSpaceType},HOSVDBasis{T, D}}) where {𝔽, T, D}
Base.convert(::Type{Matrix}, basis::CachedBasis{𝔽,DefaultOrthonormalBasis{𝔽, TangentSpaceType},HOSVDBasis{T, D}}) where {𝔽, T, D}
```

Convert a HOSVD-derived cached basis from [DBV21] of the `D`

th order `Tucker`

manifold with number type `T`

to a matrix. The columns of this matrix are the vectorisations of the `embed`

dings of the basis vectors.

`Base.foreach`

— Function`Base.foreach(f, M::Tucker, p::TuckerPoint, basis::AbstractBasis, indices=1:manifold_dimension(M))`

Let `basis`

be and `AbstractBasis`

at a point `p`

on `M`

. Suppose `f`

is a function that takes an index and a vector as an argument. This function applies `f`

to `i`

and the `i`

th basis vector sequentially for each `i`

in `indices`

. Using a `CachedBasis`

may speed up the computation.

**NOTE**: The i'th basis vector is overwritten in each iteration. If any information about the vector is to be stored, `f`

must make a copy.

`Base.ndims`

— Method`Base.ndims(p::TuckerPoint{T,D}) where {T,D}`

The order of the tensor corresponding to the `TuckerPoint`

`p`

, i.e., `D`

.

`Base.size`

— Method`Base.size(p::TuckerPoint)`

The dimensions of a `TuckerPoint`

`p`

, when regarded as a full tensor (see `embed`

).

`ManifoldsBase.check_point`

— Method`check_point(M::Tucker, p; kwargs...)`

Check whether the multidimensional array or `TuckerPoint`

`p`

is a point on the `Tucker`

manifold, i.e. it is a `D`

th order `N[1] × … × N[D]`

tensor of multilinear rank `(R[1], …, R[D])`

. The keyword arguments are passed to the matrix rank function applied to the unfoldings. For a `TuckerPoint`

it is checked that the point is in correct HOSVD form.

`ManifoldsBase.check_vector`

— Method`check_vector(M::Tucker{<:Any,D}, p::TuckerPoint{T,D}, X::TuckerTVector) where {T,D}`

Check whether a `TuckerTVector`

`X`

is is in the tangent space to the `D`

th order `Tucker`

manifold `M`

at the `D`

th order `TuckerPoint`

`p`

. This is the case when the dimensions of the factors in `X`

agree with those of `p`

and the factor matrices of `X`

are in the orthogonal complement of the HOSVD factors of `p`

.

`ManifoldsBase.embed`

— Method`embed(::Tucker, p::TuckerPoint, X::TuckerTVector)`

Convert a tangent vector `X`

with base point `p`

on the rank `R`

`Tucker`

manifold to a full tensor, represented as an `N[1] × … × N[D]`

-array.

`ManifoldsBase.embed`

— Method`embed(::Tucker, p::TuckerPoint)`

Convert a `TuckerPoint`

`p`

on the rank `R`

`Tucker`

manifold to a full `N[1] × … × N[D]`

-array by evaluating the Tucker decomposition.

`ManifoldsBase.get_basis`

— Method`get_basis(:: Tucker, p::TuckerPoint, basisType::DefaultOrthonormalBasis{𝔽, TangentSpaceType}) where 𝔽`

An implicitly stored basis of the tangent space to the Tucker manifold. Assume $p = (U_1,\dots,U_D) ⋅ \mathcal{C}$ is in HOSVD format and that, for $d=1,\dots,D$, the singular values of the $d$'th unfolding are $\sigma_{dj}$, with $j = 1,\dots,R_d$. The basis of the tangent space is as follows: [DBV21]

\[\bigl\{ (U_1,\dots,U_D) e_i \bigr\} \cup \bigl\{ (U_1,\dots, \sigma_{dj}^{-1} U_d^{\perp} e_i e_j^T,\dots,U_D) ⋅ \mathcal{C} \bigr\}\]

for all $d = 1,\dots,D$ and all canonical basis vectors $e_i$ and $e_j$. Every $U_d^\perp$ is such that $[U_d \quad U_d^{\perp}]$ forms an orthonormal basis of $ℝ^{N_d}$.

`ManifoldsBase.inner`

— Method`inner(M::Tucker, p::TuckerPoint, X::TuckerTVector, Y::TuckerTVector)`

The Euclidean inner product between tangent vectors `X`

and `X`

at the point `p`

on the Tucker manifold. This is equal to `embed(M, p, X) ⋅ embed(M, p, Y)`

.

```
inner(::Tucker, A::TuckerPoint, X::TuckerTVector, Y)
inner(::Tucker, A::TuckerPoint, X, Y::TuckerTVector)
```

The Euclidean inner product between `X`

and `Y`

where `X`

is a vector tangent to the Tucker manifold at `p`

and `Y`

is a vector in the ambient space or vice versa. The vector in the ambient space is represented as a full tensor, i.e., a multidimensional array.

`ManifoldsBase.inverse_retract`

— Method`inverse_retract(M::Tucker, p::TuckerPoint, q::TuckerPoint, ::ProjectionInverseRetraction)`

The projection inverse retraction on the Tucker manifold interprets `q`

as a point in the ambient Euclidean space (see `embed`

) and projects it onto the tangent space at to `M`

at `p`

.

`ManifoldsBase.is_flat`

— Method`is_flat(::Tucker)`

Return false. `Tucker`

is not a flat manifold.

`ManifoldsBase.manifold_dimension`

— Method`manifold_dimension(::Tucker)`

The dimension of the manifold of $N_1×\dots×N_D$ tensors of multilinear rank $(R_1, \dots, R_D)$, i.e.

\[\mathrm{dim}(\mathcal{M}) = \prod_{d=1}^D R_d + \sum_{d=1}^D R_d (N_d - R_d).\]

`ManifoldsBase.project`

— Method`project(M::Tucker, p::TuckerPoint, X)`

The least-squares projection of a dense tensor `X`

onto the tangent space to `M`

at `p`

.

`ManifoldsBase.retract`

— Method`retract(::Tucker, p::TuckerPoint, X::TuckerTVector, ::PolarRetraction)`

The truncated HOSVD-based retraction [KSV13] to the Tucker manifold, i.e. the result is the sequentially tuncated HOSVD approximation of $p + X$.

In the exceptional case that the multilinear rank of $p + X$ is lower than that of $p$, this retraction produces a boundary point, which is outside the manifold.

`ManifoldsBase.zero_vector`

— Method`zero_vector(::Tucker, p::TuckerPoint)`

The zero element in the tangent space to `p`

on the `Tucker`

manifold, represented as a `TuckerTVector`

.

## Literature

- [DBV21]
- N. Dewaele, P. Breiding and N. Vannieuwenhoven.
*The condition number of many tensor decompositions is invariant under Tucker compression*, arXiv Preprint (2021), arXiv:2106.13034. - [KL10]
- O. Koch and C. Lubich.
*Dynamical Tensor Approximation*. SIAM Journal on Matrix Analysis and Applications**31**, 2360–2375 (2010). - [KSV13]
- D. Kressner, M. Steinlechner and B. Vandereycken.
*Low-rank tensor completion by Riemannian optimization*. BIT Numerical Mathematics**54**, 447–468 (2013). - [LMV00]
- L. D. Lathauwer, B. D. Moor and J. Vandewalle.
*A Multilinear Singular Value Decomposition*. SIAM Journal on Matrix Analysis and Applications**21**, 1253–1278 (2000). - [VVM12]
- N. Vannieuwenhoven, R. Vandebril and K. Meerbergen.
*A New Truncation Strategy for the Higher-Order Singular Value Decomposition*. SIAM Journal on Scientific Computing**34**, A1027–A1052 (2012).