# Symmetric positive definite matrices

Manifolds.SymmetricPositiveDefinite β Type
SymmetricPositiveDefinite{T} <: AbstractDecoratorManifold{β}

The manifold of symmetric positive definite matrices, i.e.

$$$\mathcal P(n) = \bigl\{ p β β^{nΓn}\ \big|\ a^\mathrm{T}pa > 0 \text{ for all } a β β^{n}\backslash\{0\} \bigr\}$$$

The tangent space at $T_p\mathcal P(n)$ reads

$$$T_p\mathcal P(n) = \bigl\{ X \in \mathbb R^{nΓn} \big|\ X=X^\mathrm{T} \bigr\},$$$

i.e. the set of symmetric matrices,

Constructor

SymmetricPositiveDefinite(n; parameter::Symbol=:type)

generates the manifold $\mathcal P(n) \subset β^{nΓn}$

source

This manifold can β for example β be illustrated as ellipsoids: since the eigenvalues are all positive they can be taken as lengths of the axes of an ellipsoids while the directions are given by the eigenvectors.

The manifold can be equipped with different metrics

## Common and metric independent functions

Base.convert β Method
convert(::Type{AbstractMatrix}, p::SPDPoint)

return the point p as a matrix. The matrix is either stored within the SPDPoint or reconstructed from p.eigen.

source
Base.rand β Method
rand(M::SymmetricPositiveDefinite; Ο::Real=1)

Generate a random symmetric positive definite matrix on the SymmetricPositiveDefinite manifold M.

source
ManifoldsBase.check_point β Method
check_point(M::SymmetricPositiveDefinite, p; kwargs...)

checks, whether p is a valid point on the SymmetricPositiveDefinite M, i.e. is a matrix of size (N,N), symmetric and positive definite. The tolerance for the second to last test can be set using the kwargs....

source
ManifoldsBase.check_vector β Method
check_vector(M::SymmetricPositiveDefinite, p, X; kwargs... )

Check whether X is a tangent vector to p on the SymmetricPositiveDefinite M, i.e. atfer check_point(M,p), X has to be of same dimension as p and a symmetric matrix, i.e. this stores tangent vectors as elements of the corresponding Lie group. The tolerance for the last test can be set using the kwargs....

source
ManifoldsBase.injectivity_radius β Method
injectivity_radius(M::SymmetricPositiveDefinite[, p])
injectivity_radius(M::MetricManifold{SymmetricPositiveDefinite,LogCholeskyMetric}[, p])

Return the injectivity radius of the SymmetricPositiveDefinite. Since M is a Hadamard manifold with respect to the AffineInvariantMetric and the LogCholeskyMetric, the injectivity radius is globally $β$.

source

## Default metric: the affine invariant metric

Manifolds.AffineInvariantMetric β Type
AffineInvariantMetric <: AbstractMetric

The linear affine metric is the metric for symmetric positive definite matrices, that employs matrix logarithms and exponentials, which yields a linear and affine metric.

source

This metric is also the default metric, i.e. any call of the following functions with P=SymmetricPositiveDefinite(3) will result in MetricManifold(P,AffineInvariantMetric())and hence yield the formulae described in this section.

Base.exp β Method
exp(M::SymmetricPositiveDefinite, p, X)
exp(M::MetricManifold{<:SymmetricPositiveDefinite,AffineInvariantMetric}, p, X)

Compute the exponential map from p with tangent vector X on the SymmetricPositiveDefinite M with its default MetricManifold having the AffineInvariantMetric. The formula reads

$$$\exp_p X = p^{\frac{1}{2}}\operatorname{Exp}(p^{-\frac{1}{2}} X p^{-\frac{1}{2}})p^{\frac{1}{2}},$$$

where $\operatorname{Exp}$ denotes to the matrix exponential.

source
Base.log β Method
log(M::SymmetricPositiveDefinite, p, q)
log(M::MetricManifold{SymmetricPositiveDefinite,AffineInvariantMetric}, p, q)

Compute the logarithmic map from p to q on the SymmetricPositiveDefinite as a MetricManifold with AffineInvariantMetric. The formula reads

$$$\log_p q = p^{\frac{1}{2}}\operatorname{Log}(p^{-\frac{1}{2}}qp^{-\frac{1}{2}})p^{\frac{1}{2}},$$$

where $\operatorname{Log}$ denotes to the matrix logarithm.

source
ManifoldDiff.riemannian_Hessian β Method
riemannian_Hessian(M::SymmetricPositiveDefinite, p, G, H, X)

The Riemannian Hessian can be computed as stated in Eq. (7.3) [Ngu23]. Let $\nabla f(p)$ denote the Euclidean gradient G, $\nabla^2 f(p)[X]$ the Euclidean Hessian H, and $\operatorname{sym}(X) = \frac{1}{2}\bigl(X^{\mathrm{T}}+X\bigr)$ the symmetrization operator. Then the formula reads

$$$\operatorname{Hess}f(p)[X] = p\operatorname{sym}(β^2 f(p)[X])p + \operatorname{sym}\bigl( X\operatorname{sym}\bigl(β f(p)\bigr)p)$$$
source
ManifoldsBase.change_metric β Method
change_metric(M::SymmetricPositiveDefinite, E::EuclideanMetric, p, X)

Given a tangent vector $X β T_p\mathcal P(n)$ with respect to the EuclideanMetric g_E, this function changes into the AffineInvariantMetric (default) metric on the SymmetricPositiveDefinite M.

To be precise we are looking for $c\colon T_p\mathcal P(n) β T_p\mathcal P(n)$ such that for all $Y,Z β T_p\mathcal P(n)$ it holds

$$$β¨Y,Zβ© = \operatorname{tr}(YZ) = \operatorname{tr}(p^{-1}c(Y)p^{-1}c(Z)) = g_p(c(Z),c(Y))$$$

and hence $c(X) = pX$ is computed.

source
ManifoldsBase.change_representer β Method
change_representer(M::SymmetricPositiveDefinite, E::EuclideanMetric, p, X)

Given a tangent vector $X β T_p\mathcal M$ representing a linear function on the tangent space at p with respect to the EuclideanMetric g_E, this is turned into the representer with respect to the (default) metric, the AffineInvariantMetric on the SymmetricPositiveDefinite M.

To be precise we are looking for $ZβT_p\mathcal P(n)$ such that for all $YβT_p\mathcal P(n)$ it holds

$$$β¨X,Yβ© = \operatorname{tr}(XY) = \operatorname{tr}(p^{-1}Zp^{-1}Y) = g_p(Z,Y)$$$

and hence $Z = pXp$.

source
ManifoldsBase.distance β Method
distance(M::SymmetricPositiveDefinite, p, q)
distance(M::MetricManifold{SymmetricPositiveDefinite,AffineInvariantMetric}, p, q)

Compute the distance on the SymmetricPositiveDefinite manifold between p and q, as a MetricManifold with AffineInvariantMetric. The formula reads

$$$d_{\mathcal P(n)}(p,q) = \lVert \operatorname{Log}(p^{-\frac{1}{2}}qp^{-\frac{1}{2}})\rVert_{\mathrm{F}}.,$$$

where $\operatorname{Log}$ denotes the matrix logarithm and $\lVertβ \rVert_{\mathrm{F}}$ denotes the matrix Frobenius norm.

source
ManifoldsBase.get_basis β Method
[Ξ,ΞΊ] = get_basis(M::SymmetricPositiveDefinite, p, B::DefaultOrthonormalBasis)
[Ξ,ΞΊ] = get_basis(M::MetricManifold{<:SymmetricPositiveDefinite,AffineInvariantMetric}, p, B::DefaultOrthonormalBasis)

Return a default ONB for the tangent space $T_p\mathcal P(n)$ of the SymmetricPositiveDefinite with respect to the AffineInvariantMetric.

$$$g_p(X,Y) = \operatorname{tr}(p^{-1} X p^{-1} Y),$$$

The basis constructed here is based on the ONB for symmetric matrices constructed as follows. Let

$$$\Delta_{i,j} = (a_{k,l})_{k,l=1}^n \quad \text{ with } a_{k,l} = \begin{cases} 1 & \mbox{ for } k=l \text{ if } i=j\\ \frac{1}{\sqrt{2}} & \mbox{ for } k=i, l=j \text{ or } k=j, l=i\\ 0 & \text{ else.} \end{cases}$$$

which forms an ONB for the space of symmetric matrices.

We then form the ONB by

$$$\Xi_{i,j} = p^{\frac{1}{2}}\Delta_{i,j}p^{\frac{1}{2}},\qquad i=1,\ldots,n, j=i,\ldots,n.$$$
source
ManifoldsBase.get_basis_diagonalizing β Method
[Ξ,ΞΊ] = get_basis_diagonalizing(M::SymmetricPositiveDefinite, p, B::DiagonalizingOrthonormalBasis)
[Ξ,ΞΊ] = get_basis_diagonalizing(M::MetricManifold{<:SymmetricPositiveDefinite,AffineInvariantMetric}, p, B::DiagonalizingOrthonormalBasis)

Return a orthonormal basis Ξ as a vector of tangent vectors (of length manifold_dimension of M) in the tangent space of p on the MetricManifold of SymmetricPositiveDefinite manifold M with AffineInvariantMetric that diagonalizes the curvature tensor $R(u,v)w$ with eigenvalues ΞΊ and where the direction B.frame_direction $V$ has curvature 0.

The construction is based on an ONB for the symmetric matrices similar to get_basis(::SymmetricPositiveDefinite, p, ::DefaultOrthonormalBasis just that the ONB here is build from the eigen vectors of $p^{\frac{1}{2}}Vp^{\frac{1}{2}}$.

source
ManifoldsBase.get_coordinates β Method
get_coordinates(::SymmetricPositiveDefinite, p, X, ::DefaultOrthonormalBasis)

Using the basis from get_basis the coordinates with respect to this ONB can be simplified to

$$$c_k = \mathrm{tr}(p^{-\frac{1}{2}}\Delta_{i,j} X)$$$

where $k$ is trhe linearized index of the $i=1,\ldots,n, j=i,\ldots,n$.

source
ManifoldsBase.get_vector β Method
get_vector(::SymmetricPositiveDefinite, p, c, ::DefaultOrthonormalBasis)

Using the basis from get_basis the vector reconstruction with respect to this ONB can be simplified to

$$$X = p^{\frac{1}{2}} \Biggl( \sum_{i=1,j=i}^n c_k \Delta_{i,j} \Biggr) p^{\frac{1}{2}}$$$

where $k$ is the linearized index of the $i=1,\ldots,n, j=i,\ldots,n$.

source
ManifoldsBase.inner β Method
inner(M::SymmetricPositiveDefinite, p, X, Y)
inner(M::MetricManifold{SymmetricPositiveDefinite,AffineInvariantMetric}, p, X, Y)

Compute the inner product of X, Y in the tangent space of p on the SymmetricPositiveDefinite manifold M, as a MetricManifold with AffineInvariantMetric. The formula reads

$$$g_p(X,Y) = \operatorname{tr}(p^{-1} X p^{-1} Y),$$$
source
ManifoldsBase.parallel_transport_to β Method
parallel_transport_to(M::SymmetricPositiveDefinite, p, X, q)
parallel_transport_to(M::MetricManifold{SymmetricPositiveDefinite,AffineInvariantMetric}, p, X, y)

Compute the parallel transport of X from the tangent space at p to the tangent space at q on the SymmetricPositiveDefinite as a MetricManifold with the AffineInvariantMetric. The formula reads

$$$\mathcal P_{qβp}X = p^{\frac{1}{2}} \operatorname{Exp}\bigl( \frac{1}{2}p^{-\frac{1}{2}}\log_p(q)p^{-\frac{1}{2}} \bigr) p^{-\frac{1}{2}}X p^{-\frac{1}{2}} \operatorname{Exp}\bigl( \frac{1}{2}p^{-\frac{1}{2}}\log_p(q)p^{-\frac{1}{2}} \bigr) p^{\frac{1}{2}},$$$

where $\operatorname{Exp}$ denotes the matrix exponential and log the logarithmic map on SymmetricPositiveDefinite (again with respect to the AffineInvariantMetric).

source

## Bures-Wasserstein metric

ManifoldsBase.change_representer β Method
change_representer(M::MetricManifold{β,<:SymmetricPositiveDefinite,BuresWassersteinMetric}, E::EuclideanMetric, p, X)

Given a tangent vector $X β T_p\mathcal M$ representing a linear function on the tangent space at p with respect to the EuclideanMetric g_E, this is turned into the representer with respect to the (default) metric, the BuresWassersteinMetric on the SymmetricPositiveDefinite M.

To be precise we are looking for $ZβT_p\mathcal P(n)$ such that for all $YβT_p\mathcal P(n)$ it holds

$$$β¨X,Yβ© = \operatorname{tr}(XY) = β¨Z,Yβ©_{\mathrm{BW}}$$$

for all $Y$ and hence we get $Z$= 2(A+A^{\mathrm{T}})$with$A=Xp.

source
ManifoldsBase.distance β Method
distance(::MatricManifold{SymmetricPositiveDefinite,BuresWassersteinMetric}, p, q)

Compute the distance with respect to the BuresWassersteinMetric on SymmetricPositiveDefinite matrices, i.e.

$$$d(p,q) = \operatorname{tr}(p) + \operatorname{tr}(q) - 2\operatorname{tr}\Bigl( (p^{\frac{1}{2}}qp^{\frac{1}{2}} \bigr)^\frac{1}{2} \Bigr),$$$

where the last trace can be simplified (by rotating the matrix products in the trace) to $\operatorname{tr}(pq)$.

source

## Generalized Bures-Wasserstein metric

Manifolds.GeneralizedBuresWassersteinMetric β Type
GeneralizedBurresWassertseinMetric{T<:AbstractMatrix} <: AbstractMetric

The generalized Bures Wasserstein metric for symmetric positive definite matrices, see [HMJG21].

This metric internally stores the symmetric positive definite matrix $M$ to generalise the metric, where the name also follows the mentioned preprint.

source
Base.exp β Method
exp(::MatricManifold{β,<:SymmetricPositiveDefinite,<:GeneralizedBuresWassersteinMetric}, p, X)

Compute the exponential map on SymmetricPositiveDefinite with respect to the GeneralizedBuresWassersteinMetric given by

$$$\exp_p(X) = p+X+\mathcal ML_{p,M}(X)pML_{p,M}(X)$$$

where $q=L_{M,p}(X)$ denotes the generalized Lyapunov operator, i.e. it solves $pqM + Mqp = X$.

source
ManifoldsBase.change_representer β Method
change_representer(M::MetricManifold{β,<:SymmetricPositiveDefinite,<:GeneralizedBuresWassersteinMetric}, E::EuclideanMetric, p, X)

Given a tangent vector $X β T_p\mathcal M$ representing a linear function on the tangent space at p with respect to the EuclideanMetric g_E, this is turned into the representer with respect to the (default) metric, the GeneralizedBuresWassersteinMetric on the SymmetricPositiveDefinite M.

To be precise we are looking for $ZβT_p\mathcal P(n)$ such that for all $YβT_p\mathcal P(n)$ it holds

$$$β¨X,Yβ© = \operatorname{tr}(XY) = β¨Z,Yβ©_{\mathrm{BW}}$$$

for all $Y$ and hence we get $Z = 2pXM + 2MXp$.

source
ManifoldsBase.distance β Method
distance(::MatricManifold{SymmetricPositiveDefinite,GeneralizedBuresWassersteinMetric}, p, q)

Compute the distance with respect to the BuresWassersteinMetric on SymmetricPositiveDefinite matrices, i.e.

$$$d(p,q) = \operatorname{tr}(M^{-1}p) + \operatorname{tr}(M^{-1}q) - 2\operatorname{tr}\bigl( (p^{\frac{1}{2}}M^{-1}qM^{-1}p^{\frac{1}{2}} \bigr)^{\frac{1}{2}},$$$
source
ManifoldsBase.inner β Method
inner(::MetricManifold{β,<:SymmetricPositiveDefinite,<:GeneralizedBuresWassersteinMetric}, p, X, Y)

Compute the inner product SymmetricPositiveDefinite with respect to the GeneralizedBuresWassersteinMetric given by

$$$β¨X,Yβ© = \frac{1}{2}\operatorname{tr}(L_{p,M}(X)Y)$$$

where $q=L_{M,p}(X)$ denotes the generalized Lyapunov operator, i.e. it solves $pqM + Mqp = X$.

source

## Log-Euclidean metric

Manifolds.LogEuclideanMetric β Type
LogEuclideanMetric <: RiemannianMetric

The LogEuclidean Metric consists of the Euclidean metric applied to all elements after mapping them into the Lie Algebra, i.e. performing a matrix logarithm beforehand.

source
ManifoldsBase.distance β Method
distance(M::MetricManifold{β,<:SymmetricPositiveDefinite,LogEuclideanMetric}, p, q)

Compute the distance on the SymmetricPositiveDefinite manifold between p and q as a MetricManifold with LogEuclideanMetric. The formula reads

$$$d_{\mathcal P(n)}(p,q) = \lVert \operatorname{Log} p - \operatorname{Log} q \rVert_{\mathrm{F}}$$$

where $\operatorname{Log}$ denotes the matrix logarithm and $\lVertβ \rVert_{\mathrm{F}}$ denotes the matrix Frobenius norm.

source

## Log-Cholesky metric

Base.exp β Method
exp(M::MetricManifold{SymmetricPositiveDefinite,LogCholeskyMetric}, p, X)

Compute the exponential map on the SymmetricPositiveDefinite M with LogCholeskyMetric from p into direction X. The formula reads

$$$\exp_p X = (\exp_y W)(\exp_y W)^\mathrm{T}$$$

where $\exp_xW$ is the exponential map on CholeskySpace, $y$ is the cholesky decomposition of $p$, $W = y(y^{-1}Xy^{-\mathrm{T}})_\frac{1}{2}$, and $(β )_\frac{1}{2}$ denotes the lower triangular matrix with the diagonal multiplied by $\frac{1}{2}$.

source
Base.log β Method
log(M::MetricManifold{β,<:SymmetricPositiveDefinite,LogCholeskyMetric}, p, q)

Compute the logarithmic map on SymmetricPositiveDefinite M with respect to the LogCholeskyMetric emanating from p to q. The formula can be adapted from the CholeskySpace as

$$$\log_p q = xW^{\mathrm{T}} + Wx^{\mathrm{T}},$$$

where $x$ is the cholesky factor of $p$ and $W=\log_x y$ for $y$ the cholesky factor of $q$ and the just mentioned logarithmic map is the one on CholeskySpace.

source
ManifoldsBase.distance β Method
distance(M::MetricManifold{SymmetricPositiveDefinite,LogCholeskyMetric}, p, q)

Compute the distance on the manifold of SymmetricPositiveDefinite nmatrices, i.e. between two symmetric positive definite matrices p and q with respect to the LogCholeskyMetric. The formula reads

$$$d_{\mathcal P(n)}(p,q) = \sqrt{ \lVert β x β - β y β \rVert_{\mathrm{F}}^2 + \lVert \log(\operatorname{diag}(x)) - \log(\operatorname{diag}(y))\rVert_{\mathrm{F}}^2 }\ \ ,$$$

where $x$ and $y$ are the cholesky factors of $p$ and $q$, respectively, $ββ β$ denbotes the strictly lower triangular matrix of its argument, and $\lVertβ \rVert_{\mathrm{F}}$ the Frobenius norm.

source
ManifoldsBase.inner β Method
inner(M::MetricManifold{β,<:SymmetricPositiveDefinite,LogCholeskyMetric}, p, X, Y)

Compute the inner product of two matrices X, Y in the tangent space of p on the SymmetricPositiveDefinite manifold M, as a MetricManifold with LogCholeskyMetric. The formula reads

$$$g_p(X,Y) = β¨a_z(X),a_z(Y)β©_z,$$$

where $β¨β ,β β©_x$ denotes inner product on the CholeskySpace, $z$ is the cholesky factor of $p$, $a_z(W) = z (z^{-1}Wz^{-\mathrm{T}})_{\frac{1}{2}}$, and $(β )_\frac{1}{2}$ denotes the lower triangular matrix with the diagonal multiplied by $\frac{1}{2}$

source
ManifoldsBase.parallel_transport_to β Method
vector_transport_to(
M::MetricManifold{β,<:SymmetricPositiveDefinite,LogCholeskyMetric},
p,
X,
q,
::ParallelTransport,
)

Parallel transport the tangent vector X at p along the geodesic to q with respect to the SymmetricPositiveDefinite manifold M and LogCholeskyMetric. The parallel transport is based on the parallel transport on CholeskySpace: Let $x$ and $y$ denote the cholesky factors of p and q, respectively and $W = x(x^{-1}Xx^{-\mathrm{T}})_\frac{1}{2}$, where $(β )_\frac{1}{2}$ denotes the lower triangular matrix with the diagonal multiplied by $\frac{1}{2}$. With $V$ the parallel transport on CholeskySpace from $x$ to $y$. The formula hear reads

$$$\mathcal P_{qβp}X = yV^{\mathrm{T}} + Vy^{\mathrm{T}}.$$$
source

## Efficient representation

When a point p is used in several occasions, it might be beneficial to store the eigenvalues and vectors of p and optionally its square root and the inverse of the square root. The SPDPoint can be used for exactly that.

Manifolds.SPDPoint β Type
SPDPoint <: AbstractManifoldsPoint

Store the result of eigen(p) of an SPD matrix and (optionally) $p^{1/2}$ and $p^{-1/2}$ to avoid their repeated computations.

This result only has the result of eigen as a mandatory storage, the other three can be stored. If they are not stored they are computed and returned (but then still not stored) when required.

Constructor

SPDPoint(p::AbstractMatrix; store_p=true, store_sqrt=true, store_sqrt_inv=true)

Create an SPD point using an symmetric positive defincite matrix p, where you can optionally store p, sqrt and sqrt_inv

source

and there are three internal functions to be able to use SPDPoint interchangeably with the default representation as a matrix.

Manifolds.spd_sqrt β Function
spd_sqrt(p::AbstractMatrix)
spd_sqrt(p::SPDPoint)

return $p^{\frac{1}{2}}$ by either computing it (if it is missing or for the AbstractMatrix) or returning the stored value from within the SPDPoint.

This method assumes that p represents an spd matrix.

source
Manifolds.spd_sqrt_inv β Function
spd_sqrt_inv(p::SPDPoint)

return $p^{-\frac{1}{2}}$ by either computing it (if it is missing or for the AbstractMatrix) or returning the stored value from within the SPDPoint.

This method assumes that p represents an spd matrix.

source
Manifolds.spd_sqrt_and_sqrt_inv β Function
spd_sqrt_and_sqrt_inv(p::AbstractMatrix)
spd_sqrt_and_sqrt_inv(p::SPDPoint)

return $p^{\frac{1}{2}}$ and $p^{-\frac{1}{2}}$ by either computing them (if they are missing or for the AbstractMatrix) or returning their stored value from within the SPDPoint.

Compared to calling single methods spd_sqrt and spd_sqrt_inv this method only computes the eigenvectors once for the case of the AbstractMatrix or if both are missing.

This method assumes that p` represents an spd matrix.

source