DimensionReduction

DimensionReduction is the module for UQpy to perform the dimensionality reduction of high-dimensional data.

This module contains the classes and methods to perform pointwise and multi point data-based dimensionality reduction via projection onto Grassmann manifolds and Diffusion Maps, respectively. Further, interpolation in the tangent space centered at a given point on the Grassmann manifold can be performed, as well as the computation of kernels defined on the Grassmann manifold to be employed in techniques using subspaces.

The module currently contains the following classes:

  • Grassmann: Class for for analysis of samples on the Grassmann manifold.

  • DiffusionMaps: Class for multi point data-based dimensionality reduction.

This module contains the classes and methods to perform the point-wise and multi point data-based dimensionality reduction via projection onto the Grassmann manifold and Diffusion Maps, respectively. Further, interpolation in the tangent space centered at a given point on the Grassmann manifold can be performed.

The module UQpy.DimensionReduction currently contains the following classes:

  • Grassmann: Class for for analysis of samples on the Grassmann manifold.

  • DiffusionMaps: Class for multi point data-based dimensionality reduction.

Grassmann

In differential geometry the Grassmann manifold \(\mathcal{G}_{n,p}\) refers to a collection of \(p\)-dimensional subspaces embedded in a \(n\)-dimensional vector space. The Grassmann class contains methods to perform the projection of matrices onto the Grassmann manifold via singular value decomposition (SVD), where their dimensionality are reduced. Further, a tangent space, where standard interpolation can be performed, is constructed at a given reference point. Therefore, the mapping from the Grassmann manifold to the tangent space and from the tangent space to the manifold are performed via the logarithmic and exponential mapping, respectively. Moreover, additional quantities such as the Karcher mean, which correspond to the point on the Grassmann manifold minimizing the squared distances to the other points on the same manifold. Further, the kernel defined on the Grassmann manifold is implemented to estimate an affinity matrix to be used in kernel-based machine learning techniques.

A tangent space \(\mathcal{T}_{\mathcal{X}}\mathcal{G}(p,n)\), which is a flat inner-product space, is defined as a set of all tangent vectors at \(\mathcal{X}\) 1, 2, 3; such as

\[\mathcal{T}_{\mathcal{X}}\mathcal{G}(p,n) = \{\mathbf{\Gamma} \in \mathbb{R}^{n \times p} : \mathbf{\Gamma}^T\mathbf{\Psi}=\mathbf{0}\}\]

Where a point \(\mathcal{X} = \mathrm{span}\left(\mathbf{\Psi}\right) \in \mathcal{G}(p,n)\) is invariant to the choice of basis such that \(\mathrm{span}\left(\mathbf{\Psi}\right) = \mathrm{span}\left(\mathbf{R\Psi}\right)\), with \(\mathbf{R} \in SO(p)\), where \(SO(p)\) is the special orthogonal group.

One can write the exponential map (from the tangent space to the manifold) locally as (4, 5)

\[\mathrm{exp}_{\mathcal{X}_0}(\mathbf{\Gamma}) = \mathbf{\Psi}_1\]

Denoting \(\mathbf{\Gamma}\) by its singular value decomposition \(\mathbf{\Gamma} = \mathbf{U}\mathbf{S}\mathbf{V}^T\) one can write a point on the Grassmann manifold \(\mathbf{\Psi}_1\), considering a reference point \(\mathbf{\Psi}_0\), as

\[\mathbf{\Psi}_1 = \mathrm{exp}_{\mathcal{X}_0}(\mathbf{U}\mathbf{S}\mathbf{V}^T) = \mathbf{\Psi}_0\mathbf{V}\mathrm{cos}\left(\mathbf{S}\right)\mathbf{Q}^T+\mathbf{U}\mathrm{sin}\left(\mathbf{S}\right)\mathbf{Q}^T\]

Equivalently, the logarithmic map \(\mathrm{log}_\mathcal{X}:\mathcal{G}(p,n) \rightarrow \mathcal{T}_{\mathcal{X}}\mathcal{G}(p,n)\) is defined locally as

\[\mathrm{log}_\mathcal{X}(\mathbf{\Psi}_1) = \mathbf{U}\mathrm{tan}^{-1}\left(\mathbf{S}\right)\mathbf{V}^T\]

One can write the geodesic as

\[\gamma(t)=\mathrm{span}\left[\left(\mathbf{\Psi}_0\mathbf{V}\mathrm{cos}(t\mathbf{S})+\mathbf{U}\mathrm{sin}(t\mathbf{S})\right)\mathbf{V}^T\right]\]

where \(\mathbf{\Psi}_0\), if \(t=0\), and \(\mathbf{\Psi}_1\), \(t=1\).

The geodesic distance \(d_{\mathcal{G}(p,n)}\left(\mathbf{\Psi}_0,\mathbf{\Psi}_1\right)\) between two points on $mathcal{G}(p,n)$ corresponds to the distance over the geodesic \(\gamma(t)\) and it is given by

\[d_{\mathcal{G}(p,n)}\left(\mathbf{\Psi}_0,\mathbf{\Psi}_1\right) = ||\mathbf{\Theta}||_2\]

where \(\mathbf{\Theta} = \left(\theta_1, \theta_2, \dots, \theta_p \right)\) contains the principal angles. Several definitions of distance on \(\mathcal{G}(p,n)\) can be found in the literature.

In several applications the use of subspaces is essential to describe the underlying geometry of data. However, it is well-known that subspaces do not follow the Euclidean geometry because they lie on the Grassmann manifold. Therefore, working with subspaces requires the definition of an embedding structure of the Grassmann manifold into a Hilbert space. Thus, using positive definite kernels is studied as a solution to this problem. In this regard, a real-valued positive definite kernel is defined as a symmetric function \(k:\mathcal{X}\times \mathcal{X} \rightarrow \mathbb{R}\) if and only if \(\sum^n_{I,j=1}c_i c_j k(x_i,x_j) \leq 0\) for \(n \in \mathbb{N}\), \(x_i in \mathcal{X}\) and \(c_i \in \mathbb{R}\). Further, the Grassmann kernel can be defined as a function \(k:\mathcal{G}(p,n)\times \mathcal{G}(p,n) \rightarrow \mathbb{R}\) if it is well-defined and positive definite 6.

Grassmann Class Descriptions

The Grassmann class is imported using the following command:

>>> from DimensionReduction import Grassmann

One can use the following command to instantiate the class Grassmann

class UQpy.DimensionReduction.Grassmann(distance_method=None, kernel_method=None, interp_object=None, karcher_method=None)[source]

Mathematical analysis on the Grassmann manifold.

The Grassmann class contains methods of data analysis on the Grassmann manifold, which is a special case of flag manifold. The projection of matrices onto the Grassmann manifold is performed via singular value decomposition(SVD), where their dimensionality are reduced. Further, the mapping from the Grassmann manifold to a tangent space constructed at a given reference point (logarithmic mapping), as well as the mapping from the tangent space to the manifold (exponential mapping) are implemented as methods. Moreover, an interpolation can be performed on the tangent space taking advantage of the implemented logarithmic and exponential maps. Additional quantities such as the Karcher mean, distance, and kernel, all defined on the Grassmann manifold, can be obtained using implemented methods. When the class Grassmann is instantiated some attributes are set using the method manifold, where the dimension of the manifold as well as the samples are input arguments.

Input:

  • distance_method (callable)

    Defines the distance metric on the manifold. The user can pass a callable object defining the distance metric using two different ways. First, the user can pass one of the methods implemented in the class Grassmann, they are:

    • grassmann_distance;

    • chordal_distance;

    • procrustes_distance;

    • projection_distance

    • binet_cauchy_distance.

    In this regard, the class Grassmann is instantiated and the attributes are set using the method manifold. Thus, an object containing the attribute distance_method is passed as Grassmann.distance_method. Second, the user can pass either a method of a class or a function. For example, if the user wish to use grassmann_distance to compute the distance, one can use the following command:

    On the other hand, if the user implemented a function (e.g., user_distance) to compute the distance, distance_method must assume the following value distance_method = user_distance, which must be pre-loaded using import. In this regard, the function input must contain the first (x0) and second (x1) matrices as arguments (e.g, user_distance(x0,x1))

  • kernel_method (callable)

    Object of the kernel function defined on the Grassmann manifold. The user can pass a object using two different ways. First, the user can pass one of the methods implemented in the class Grassmann, they are:

    • projection_kernel;

    • binet_cauchy_kernel.

    In this regard, the object is passed as Grassmann.kernel_method. Second, the user can pass callable objects either as a method of a class or as a function. For example, if the user wish to use projection_kernel to estimate the kernel matrix, one can use the following command:

    On the other hand, if the user implemented a function (e.g., user_kernel) to compute the kernel matrix, kernel_method must assume the following value kernel_object = user_kernel, which must be pre-loaded using import. In this regard, the function input must contain the first (x0) and second (x1) matrices as arguments (e.g, user_kernel(x0,x1))

  • interp_object (object)

    Interpolator to be used in the Tangent space. The user can pass an object defining the interpolator via four different ways.

    • Using the Grassmann method linear_interp as Grassmann(interp_object=Grassmann.linear_interp).

    • Using an object of UQpy.Kriging as Grassmann(interp_object=Kriging_Object)

    • Using an object of sklearn.gaussian_process as Grassmann(interp_object=Sklearn_Object)

    • Using an user defined object (e.g., user_interp). In this case, the function must contain the following arguments: coordinates, samples, and point.

  • karcher_method (callable)

    Optimization method used in the estimation of the Karcher mean. The user can pass a callable object via two different ways. First, the user can pass one of the methods implemented in the class Grassmann, they are:

    • gradient_descent;

    • stochastic_gradient_descent.

    Second, the user can pass callable objects either as a method of a class or as a function. It is worth mentioning that the method gradient_descent also allows the accelerated descent method due to Nesterov.

Attributes:

  • p (int or str)

    Dimension of the p-planes defining the Grassmann manifold G(n,p).

  • ranks (list)

    Dimension of the embedding dimension for the manifolds G(n,p) of each sample.

  • samples (list of list or ndarray)

    Input samples defined as a list of matrices.

  • nargs (int)

    Number of matrices in samples.

  • max_rank (int)

    Maximum value of ranks.

  • psi (list)

    Left singular eigenvectors from the singular value decomposition of each sample in samples representing a point on the Grassmann manifold.

  • sigma (list)

    Singular values from the singular value decomposition of each sample in samples.

  • phi (list)

    Right singular eigenvector from the singular value decomposition of each sample in samples representing a point on the Grassmann manifold.

Methods:

static binet_cauchy_distance(x0, x1)[source]

Estimate the Binet-Cauchy distance.

One of the distances defined on the Grassmann manifold is the projection distance.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Projection distance between x0 and x1.

static binet_cauchy_kernel(x0, x1)[source]

Estimate the value of the Binet-Cauchy kernel between x0 and x1.

One of the kernels defined on a manifold is the Binet-Cauchy kernel.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Kernel value for x0 and x1.

static chordal_distance(x0, x1)[source]

Estimate the chordal distance.

One of the distances defined on the Grassmann manifold is the chordal distance.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Chordal distance between x0 and x1.

distance(points_grassmann=None)[source]

Estimate the distance between points on the Grassmann manifold.

This method computes the pairwise distance of points projected on the Grassmann manifold. The input arguments are passed through a list of list or a list of ndarray. When the user call this method a list containing the pairwise distances is returned as an output argument where the distances are stored as [{0,1},{0,2},…,{1,0},{1,1},{1,2},…], where {a,b} corresponds to the distance between the points ‘a’ and ‘b’. Further, users are asked to provide the distance definition when the class Grassmann is instatiated. The current built-in options are the grassmann_distance, chordal_distance, procrustes_distance, projection_distance, and binet_cauchy_distance, but the users have also the option to implement their own distance definition. In this case, the user must be aware that the matrices in points_grassmann must represent points on the Grassmann manifold. For example, given the points on the Grassmann manifold one can compute the pairwise distances in the following way:

Input:

  • points_grassmann (list or NoneType)

    Matrices (at least 2) corresponding to the points on the Grassmann manifold. If points_grassmann is not provided it means that the samples in manifold are employed, and the pairwise distances of the points on the manifold defined by the left and right singular eigenvectors are computed.

Output/Returns:

  • points_distance (list)

    Pairwise distances if points_grassmann is provided.

  • points_distance_psi (list)

    Pairwise distance of points on the manifold defined by the left singular eigenvectors if points_grassmann is not provided.

  • points_distance_phi (list)

    Pairwise distance of points on the manifold defined by the right singular eigenvectors if points_grassmann is not provided.

static exp_map(points_tangent=None, ref=None)[source]

Map points on the tangent space onto the Grassmann manifold.

It maps the points on the tangent space, passed to the method using points_tangent, onto the Grassmann manifold. It is mandatory that the user pass a reference point where the tangent space was created.

Input:

  • points_tangent (list)

    Matrices (at least 2) corresponding to the points on the Grassmann manifold.

  • ref (list or ndarray)

    A point on the Grassmann manifold used as reference to construct the tangent space.

Output/Returns:

  • points_manifold: (list)

    Point on the tangent space.

static frechet_variance(point_grassmann, points_grassmann, distance_fun)[source]

Compute the Frechet variance.

The Frechet variance corresponds to the summation of the square distances, on the manifold, to a given point also on the manifold. This method is employed in the minimization scheme used to find the Karcher mean.

Input:

  • point_grassmann (list or ndarray)

    Point on the Grassmann manifold where the Frechet variance is computed.

  • points_grassmann (list or ndarray)

    Points on the Grassmann manifold.

  • distance_fun (callable)

    Distance function.

Output/Returns:

  • frechet_var (list)

    Frechet variance.

static gradient_descent(data_points, distance_fun, kwargs)[source]

Compute the Karcher mean using the gradient descent method.

This method computes the Karcher mean given a set of points on the Grassmann manifold. In this regard, the gradient_descent method is implemented herein also considering the acceleration scheme due to Nesterov. Further, this method is called by the method karcher_mean.

Input:

  • data_points (list)

    Points on the Grassmann manifold.

  • distance_fun (callable)

    Distance function.

  • kwargs (dictionary)

    Contains the keywords for the used in the optimizers to find the Karcher mean.

Output/Returns:

  • mean_element (list)

    Karcher mean.

static grassmann_distance(x0, x1)[source]

Estimate the Grassmann distance.

One of the distances defined on the Grassmann manifold is the Grassmann distance.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Grassmann distance between x0 and x1.

interpolate(coordinates, point, element_wise=True)[source]

Interpolate a point on the Grassmann manifold given the samples in the ambient space (sample space).

Interpolate a point on the Grassmann manifold given the coordinates, support points, and the samples. Further, the user must select the option element_wise to perform the interpolation in the entries of the input matrices, if point and samples are matrices. The samples related to coordinates are set using manifold. For example, the following command is used to perform the interpolation.

On the other hand, if a scikit learn gaussian_process object is provided, one can use the following commands:

Input:

  • coordinates (list or ndarray)

    Coordinate of the support samples.

  • point (list or ndarray)

    Point to be interpolated.

  • element_wise (bool)

    Element wise interpolation.

Output/Returns:

  • interpolated (list)

    Interpolated point.

interpolate_sample(coordinates, samples, point, element_wise=True)[source]

Interpolate a point on the tangent space.

Once the points on the Grassmann manifold are projected onto the tangent space standard interpolation can be performed. In this regard, the user should provide the data points, the coordinates of each input data point, and the point to be interpolated. Furthermore, additional parameters, depending on the selected interpolation method, can be provided via kwargs. In comparison to interpolate, here the samples prodived are points on the TANGENT SPACE.

Input:

  • coordinates (list or ndarray)

    Coordinates of the input data points.

  • samples (list or ndarray)

    Matrices corresponding to the points on the tangent space.

  • point (list or ndarray)

    Coordinates of the point to be interpolated.

  • element_wise (bool)

    Boolean variable for the element wise intepolation of a matrix.

Output/Returns:

  • interp_point (ndarray)

    Interpolated point on the tangent space.

karcher_mean(points_grassmann=None, **kwargs)[source]

Compute the Karcher mean.

This method computes the Karcher mean given a set of points on the Grassmann manifold. The Karcher mean is estimated by the minimization of the Frechet variance, where the Frechet variance corresponds to the sum of the square distances, defined on the Grassmann manifold, to a given point. The command to compute the Karcher mean given a seto of points on the Grassmann manifold is.

In this case two values are returned corresponding to the ones related to the manifolds defined by the left and right singular eigenvectors.

Input:

  • points_grassmann (list or ndarray)

    Matrices (at least 2) corresponding to the points on the Grassmann manifold.

  • kwargs (dictionary)

    Contains the keywords for the used in the optimizers to find the Karcher mean. If gradient_descent is employed the keywords are acc, a boolean variable for the accelerated method; tol, tolerance with default value equal to 1e-3; and maxiter, maximum number of iterations with default value equal to 1000. If stochastic_gradient_descent is employed instead, acc is not used.

Output/Returns:

  • kr_mean (list)

    Karcher mean.

  • kr_mean_psi (list)

    Karcher mean for left singular eigenvectors if points_grassmann is not provided.

  • kr_mean_phi (list)

    Karcher mean for right singular eigenvectors if points_grassmann is not provided.

kernel(points_grassmann=None)[source]

Compute a kernel matrix on the Grassmann manifold.

This method computes the kernel matrix of points on the Grassmann manifold. The input arguments are passed through a list. When the user call this method a ndarray containing the kernel matrix is returned as an output argument. Further, users are asked to provide the kernel method when the class Grassmann is instatiated. The current built-in options are the projection_kernel and the binet_cauchy_kernel, but the users have the option to implement their own kernel definition. However, the user must be aware that the points used to compute the kernel must be the points on the same manifold; therefore, it is important to ensure that the dimension of all the input matrices are the same. Therefore, the following commands can be used:

Input:

  • points_grassmann (list or NoneType)

    Matrices (at least 2) corresponding to the points on the Grassmann manifold. If points_grassmann is not provided it means that the samples set by manifold is employed and kernel matrices are computed for the points on the manifold defined by the left and right singular eigenvectors are computed.

Output/Returns:

  • kernel_matrix (list)

    Kernel matrix if points_grassmann is provided.

  • kernel_matrix_psi (list)

    Kernel matrix on the manifold defined by the left singular eigenvectors if points_grassmann is not provided.

  • kernel_matrix_phi (list)

    Kernel matrix on the manifold defined by the right singular eigenvectors if points_grassmann is not provided.

static linear_interp(coordinates, samples, point)[source]

Interpolate a point using the linear interpolation.

For the linear interpolation the user are asked to provide the data points, the coordinates of the data points, and the coordinate of the point to be interpolated.

Input:

  • coordinates (ndarray)

    Coordinates of the input data points.

  • samples (ndarray)

    Matrices corresponding to the points on the Grassmann manifold.

  • point (ndarray)

    Coordinates of the point to be interpolated.

Output/Returns:

  • interp_point (ndarray)

    Interpolated point.

static log_map(points_grassmann=None, ref=None)[source]

Mapping points from the Grassmann manifold on the tangent space.

It maps the points on the Grassmann manifold, passed to the method using the input argument points_grassmann, onto the tangent space constructed on ref (a reference point on the Grassmann manifold). It is mandatory that the user pass a reference point to the method. Further, the reference point and the points in points_grassmann must belong to the same manifold.

Input:

  • points_grassmann (list)

    Matrices (at least 2) corresponding to the points on the Grassmann manifold.

  • ref (list or ndarray)

    A point on the Grassmann manifold used as reference to construct the tangent space.

Output/Returns:

  • points_tan: (list)

    Point on the tangent space.

manifold(p=None, samples=None, append_samples=False)[source]

Set the grassmann manifold and project the samples on it via singular value decomposition.

This method project samples onto the Grassmann manifold via singular value decomposition. The input arguments are passed through the argument samples containing a list of lists or a list of ndarrays. Moreover, this method serves to set the manifold and to verify the consistency of the input data. This method can be called using the following command:

In this case, append_samples is a boolean variable used to define if new sample will replace the previous ones or will just get appended (append_samples=True).

Input:

  • p (int or str or NoneType)

    Dimension of the p-planes defining the Grassmann manifold G(n,p). This parameter can assume an integer value larger than 0 or the strings max, when p assumes the maximum rank of the input matrices, or min when it assumes the minimum one. If p is not provided ranks will store the ranks of each input matrix and each sample will lie on a distinct manifold.

  • samples (list)

    Input samples defined as a list of matrices. In this regard, samples is a collection of matrices stored as a list. Moreover, The shape of the input matrices stored in samples are verified and compared for consistency with p.

  • append_samples (bool)

    The attributes are replaced when manifold is called if append_samples is False, otherwise the lists are appended.

static procrustes_distance(x0, x1)[source]

Estimate the Procrustes distance.

One of the distances defined on the Grassmann manifold is the Procrustes distance.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Procrustes distance between x0 and x1.

static projection_distance(x0, x1)[source]

Estimate the projection distance.

One of the distances defined on the Grassmann manifold is the projection distance.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Projection distance between x0 and x1.

static projection_kernel(x0, x1)[source]

Estimate the value of the projection kernel between x0 and x1.

One of the kernels defined on a manifold is the projection kernel.

Input:

  • x0 (list or ndarray)

    Point on the Grassmann manifold.

  • x1 (list or ndarray)

    Point on the Grassmann manifold.

Output/Returns:

  • distance (float)

    Kernel value for x0 and x1.

static stochastic_gradient_descent(data_points, distance_fun, kwargs)[source]

Compute the Karcher mean using the stochastic gradient descent method.

This method computes the Karcher mean given a set of points on the Grassmann manifold. In this regard, the stochastic_gradient_descent method is implemented herein. Further, this method is called by the method karcher_mean.

Input:

  • data_points (list)

    Points on the Grassmann manifold.

  • distance_fun (callable)

    Distance function.

  • kwargs (dictionary)

    Contains the keywords for the used in the optimizers to find the Karcher mean.

Output/Returns:

  • mean_element (list)

    Karcher mean.

DiffusionMaps

In nonlinear dimensionality reduction Diffusion Maps corresponds to a technique used to reveal the intrinsic structure of data sets based on a diffusion process over the data. In particular, the eigenfunctions of Markov matrices defining a random walk on the data are used to obtain a coordinate system represented by the diffusion coordinates revealing the embedded geometry of the data. Moreover, the diffusion coordinates are defined on a Euclidean space where usual metrics define the distances between pairs of data points. Thus, the diffusion maps create a connection between the spectral properties of the diffusion process and the intrinsic geometry of the data resulting in a multiscale representation of the data.

To present this method let’s assume measure space \((X, \mathcal{A}, \mu)\), where \(X\) is the dataset, \(\mathcal{A}\) is a \(\sigma-\), and \(\mu\) a measure; and a non-negative symmetric kernel \(k: X \times X \rightarrow \mathbb{R}\) representing the pairwise affinity of the data points in a symmetric graph; one can define the connectivity between two points as the transition probability in a random walk using the kernel \(k\). Therefore, the diffusion maps technique can be based on a normalized graph Laplacian construction 7, 8, where

\[p(x, y) = \frac{k(x,y)}{\int_X k(x,y)d\mu(y)}\]

with

\[\int_X p(x,y)d\mu(y) = 1\]

can be viewed as the one-step transition probability . Therefore, to construct the transition probability one can resort to the graph Laplacian normalization. In this regard, one can consider that \(L_{i,j} = k(x_i,x_j)\) must be normalized such that \(\tilde{L}_{i,j} = D^{-\alpha}LD^{-\alpha}\), where

is a diagonal matrix. Next, a new matrix $D$ is obtained from \(\tilde{L}\), thus

\[\tilde{D}_{i,i} = \sum_j \tilde{L}_{i,j}\]

Therefore, the transition probability \(M_{i,j} = p(x_j,t|x_i)\) can be obtained after the graph Laplacian normalization of \(\tilde{L}\) such as

\[M = \tilde{D}^{-1}\tilde{L}\]

From the eigendecomposition of \(M\), one can obtain the eigenvectors \((\psi_0, \psi_1, \dots, \psi_N)\) and their respective eigenvalues \((\lambda_0, \lambda_1, \dots, \lambda_N)\). However, only \(k\) eigenvectors and eigenvalues suffice. Thus, the diffusion coordinates are given by \(\Psi_i = \lambda_i \psi_i\) with \(i=1,\dots,k\).

Diffusion Maps Class Descriptions

The DiffusionMaps class is imported using the following command:

>>> from DimensionReduction import DiffusionMaps

One can use the following command to instantiate the class DiffusionMaps

class UQpy.DimensionReduction.DiffusionMaps(alpha=0.5, n_evecs=2, sparse=False, k_neighbors=1, kernel_object=None, kernel_grassmann=None)[source]

Perform the diffusion maps on the input data to reveal its lower dimensional embedded geometry.

In this class, the diffusion maps create a connection between the spectral properties of the diffusion process and the intrinsic geometry of the data resulting in a multiscale representation of it. In this regard, an affinity matrix containing the degree of similarity of the data points is either estimated based on the euclidean distance, using a Gaussian kernel, or it is computed using any other Kernel definition passed to the main method (e.g., defining a kernel on the Grassmann manifold).

Input:

  • alpha (float)

    Assumes a value between 0 and 1 and corresponding to different diffusion operators. In this regard, one can use this parameter to take into consideration the distribution of the data points on the diffusion process. It happens because the distribution of the data is not necessarily dependent on the geometry of the manifold. Therefore, if alpha` is equal to 1, the Laplace-Beltrami operator is approximated and the geometry of the manifold is recovered without taking the distribution of the points into consideration. On the other hand, when alpha is equal to 0.5 the Fokker-Plank operator is approximated and the distribution of points is taken into consideration. Further, when alpha is equal to zero the Laplace normalization is recovered.

  • n_evecs (int)

    The number of eigenvectors and eigenvalues used in the representation of the diffusion coordinates.

  • sparse (bool)

    Is a boolean variable to activate the sparse mode of the method.

  • k_neighbors (int)

    Used when sparse is True to select the k samples close to a given sample in the construction of an sparse graph defining the affinity of the input data. For instance, if k_neighbors is equal to 10, only the closest ten points of a given point are connect to a given point in the graph. As a consequence, the obtained affinity matrix is sparse which reduces the computational effort of the eigendecomposition of the transition kernel of the Markov chain.

  • kernel_object (function)

    An object of a callable object used to compute the kernel matrix. Three different options are provided:

    • Using the DiffusionMaps method gaussian_kernel as DiffusionMaps(kernel_object=DiffusionMaps.gaussian_kernel);

    • Using an user defined function as DiffusionMaps(kernel_object=user_kernel);

    • Passing a Grassmann class object DiffusionMaps(kernel_object=Grassmann_Object). In this case, the user has to select kernel_grassmann in order to define which kernel matrix will be used because when the the Grassmann class is used in a dataset a kernel matrix can be constructed with both the left and right singular eigenvectors.

  • kernel_grassmann (str)

    It assumes the values ‘left’ and ‘right’ for the left and right singular eigenvectors used to compute the kernel matrix, respectively. Moreover, if ‘sum’ is selected, it means that the kernel matrix is composed by the sum of the kernel matrices estimated using the left and right singular eigenvectors. On the other hand, if ‘prod’ is used instead, it means that the kernel matrix is composed by the product of the matrices estimated using the left and right singular eigenvectors.

Attributes:

  • kernel_matrix (ndarray)

    Kernel matrix.

  • transition_matrix (ndarray)

    Transition kernel of a Markov chain on the data.

  • dcoords (ndarray)

    Diffusion coordinates

  • evecs (ndarray)

    Eigenvectors of the transition kernel of a Markov chanin on the data.

  • evals (ndarray)

    Eigenvalues of the transition kernel of a Markov chanin on the data.

Methods:

gaussian_kernel(data, epsilon=None)[source]

Compute the Gaussian Kernel matrix.

Estimate the affinity matrix using the Gaussian kernel. If no epsilon is provided the method estimates a suitable value taking the median of the square value of the pairwise euclidean distances of the points in the input dataset.

Input:

  • data (list)

    Input data.

  • epsilon (float)

    Parameter of the Gaussian kernel.

Output/Returns:

  • Kernel matrix (ndarray)

    Kernel matrix.

mapping(data=None, epsilon=None)[source]

Perform diffusion maps to reveal the embedded geometry of datasets.

In this method, the users have the option to work with input data defined by subspaces obtained via projection of input data points on the Grassmann manifold, or directly with the input data points. For example, considering that a Grassmann object is provided using the following command:

one can instantiate the DiffusionMaps class and run the diffusion maps as follows:

On the other hand, if the user wish to pass a dataset (samples) to compute the diffusion coordinates using the Gaussian kernel, one can use the following commands:

In the latest case, if epsilon is not provided it is estimated based on the median of the square of the euclidian distances between data points.

Input:

  • data (list)

    Data points in the ambient space.

  • epsilon (floar)

    Parameter of the Gaussian kernel.

Output/Returns:

  • dcoords (ndarray)

    Diffusion coordinates.

  • evals (ndarray)

    eigenvalues.

  • evecs (ndarray)

    eigenvectors.

1
  1. Maruskin, Introduction to Dynamical Systems and Geometric Mechanics, Solar Crest Publishing, LLC, 2012:p.165.

2
  1. Wang, Y. Hu, J. Gao, Y. Sun, B. Yin, Low rank represen6tation on Grassmann manifolds: An extrinsic perspective, 2015, 167arXiv:1504.01807.168.

3
  1. Sommer, T. Fletcher, X. Pennec, 1 - introduction to differential and Riemannian geometry, in: X. Pennec, S. Sommer, T. Fletcher (Eds.), Riemannian Geometric Statistics in Medical Image Analysis, Academic Press, 2020, p.3–37.

4
  1. Giovanis, M. Shields, Uncertainty quantification for complex systems with very high dimensional response using Grassmann manifold variations, Journal of Computational Physics, 2018, 364, p.393–415.

5
  1. Wang, Y. Hu, J. Gao, Y. Sun, B. Yin, Low rank representation on Grassmann manifolds: An extrinsic perspective, 2015, 167arXiv:1504.01807.

6
    1. Harandi, M. Salzmann, S. Jayasumana, R. Hartley, H. Li, Expanding the family of Grassmannian kernels: An embedding perspective, 2014, 1622014.arXiv:1407.1123.

7
    1. Coifman, S. Lafon. Diffusion maps. Applied Computational Harmonic Analysis, 2006, 21(1), p.5–30.

8
    1. Coifman, I. G. Kevrekidis, S. Lafon, M. Maggioni, and B. Nadler, Diffusionmaps, reduction coordinates, and low dimensional representation of stochastic systems, Multiscale Modeling and Simulation, 2008, 7(2), p.842–864.