Surrogates

This module contains functionality for all the surrogate methods supported in UQpy.

The module currently contains the following classes:

  • SROM: Class to estimate a discrete approximation for a continuous random variable using Stochastic Reduced Order

    Model.

  • Kriging: Class to generate an approximate surrogate model using Kriging.

Stochatic Reduced Order Models - SROMs

An SROM is a sample-based surrogate for probability models. An SROM takes a set of samples and attributes of a distribution and optimizes the sample probability weights according to the method in 1. More specifically, an SROM constructs a reduce order model for arbitrary random variables X as follows.

\[\begin{split}\tilde{X} = \begin{cases} x_1 & probability \text{ }p_1^{(opt)} \\ & \vdots \\ x_m & probability \text{ }p_m^{(opt)} \end{cases}\end{split}\]

where \(\tilde{X}\) is defined by an arbitrary set of samples \(x_1, \dots, x_m\) defined over the same support as \(X\) (but not necessarily drawn from its probability distribution) and their assigned probability weights. The probability weights are defined such that the total error between the sample empirical probability distribution, moments and correlation of \(\tilde{X}\) and those of the random variable X is minimized. This optimization problem can be express as:

\[\begin{split}& \min_{\mathbf{p}} \sum_{u=1}^3 \alpha_u e_u(\mathbf{p}) \\ & \text{s.t.} \sum_{k=1}^m p_k =1 \quad and \quad p_k \geq 0, \quad k=1,2,\dots,m\end{split}\]

where \(\alpha_1\), \(\alpha_2\), \(\alpha_3 \geq 0\) are constants defining the relative importance of the marginal distribution, moments and correlation error between the reduce order model and actual random variables in the objective function.

\[\begin{split}& e_{1}(p)=\sum\limits_{i=1}^d \sum\limits_{k=1}^m w_{F}(x_{k,i};i)(\hat{F}_{i}(x_{k,i})-F_{i}(x_{k,i}))^2 \\ & e_{2}(p)=\sum\limits_{i=1}^d \sum\limits_{r=1}^2 w_{\mu}(r;i)(\hat{\mu}(r;i)-\mu(r;i))^2 \\ & e_{3}(p)=\sum\limits_{i,j=1,...,d ; j>i} w_{R}(i,j)(\hat{R}(i,j)-R(i,j))^2\end{split}\]

Here, \(F(x_{k,i})\) and \(\hat{F}(x_{k,i})\) denote the marginal cumulative distributions of \(\mathbf{X}\) and \(\mathbf{\tilde{X}}\) (reduced order model) evaluated at point \(x_{k,i}\), \(\mu(r; i)\) and \(\hat{\mu}(r; i)\) are the marginal moments of order r for variable i, and \(R(i,j)\) and \(\hat{R}(i,j)\) are correlation matrices of \(\mathbf{X}\) and \(\mathbf{\tilde{X}}\) evaluted for components \(x_i\) and \(x_j\). Note also that m is the number of sample points and d is the number of random variables.

SROM Class Descriptions

class UQpy.Surrogates.SROM(samples, target_dist_object, moments=None, weights_errors=None, weights_distribution=None, weights_moments=None, weights_correlation=None, properties=None, correlation=None, verbose=False)[source]

Stochastic Reduced Order Model(SROM) provide a low-dimensional, discrete approximation of a given random quantity.

Inputs:

  • samples (ndarray):

    An array/list of samples corresponding to each random variables.

  • target_dist_object ((list of) Distribution object(s)):

    A list of distribution objects of random variables.

  • moments (list of float):

    A list containing first and second order moment about origin of all random variables.

  • weights_errors (list of float):

    Weights associated with error in distribution, moments and correlation.

    Default: weights_errors = [1, 0.2, 0]

  • properties (list of booleans):

    A list of booleans representing properties, which are required to match in reduce order model. This class focus on reducing errors in distribution, first order moment about origin, second order moment about origin and correlation of samples. Example: properties = [True, True, False, False] will minimize errors in distribution and errors in first order moment about origin in reduce order model.

    Default: properties = [True, True, True, False]

  • weights_distribution (ndarray or list of float):

    An list or array containing weights associated with different samples.

    Options:

    If weights_distribution is None, then default value is assigned. If size of weights_distribution is 1xd, then it is assigned as dot product of weights_distribution and default value. Otherwise size of weights_distribution should be equal to Nxd.

    Default: weights_distribution = An array of shape Nxd with all elements equal to 1.

  • weights_moments (ndarray or list of float):

    An array of dimension 2xd, where ‘d’ is number of random variables. It contain weights associated with moments.

    Options:

    If weights_moments is None, then default value is assigned. If size of weights_moments is 1xd, then it is assigned as dot product of weights_moments and default value. Otherwise size of weights_distribution should be equal to 2xd.

    Default: weights_moments = Square of reciprocal of elements of moments.

  • weights_correlation (ndarray or list of float):

    An array of dimension dxd, where ‘d’ is number of random variables. It contain weights associated with correlation of random variables.

    Default: weights_correlation = dxd dimensional array with all elements equal to 1.

  • correlation (ndarray or list of floats):

    Correlation matrix between random variables.

Attribute:

  • sample_weights (ndarray):

    The probabilities/weights defining discrete approximation of continuous random variables.

Methods:

run(weights_errors=None, weights_distribution=None, weights_moments=None, weights_correlation=None, properties=None)[source]

Execute the stochastic reduced order model in the SROM class.

The run method is the function that computes the probability weights corresponding to the sample. If properties is provided, the run method is automatically called when the SROM object is defined. The user may also call the run method directly to generate samples. The run method of the SROM class can be invoked many times with different weights parameters and each time computed probability weights are overwritten.

Inputs:

  • weights_errors (list of float):

    Weights associated with error in distribution, moments and correlation.

    Default: weights_errors = [1, 0.2, 0]

  • properties (list of booleans):

    A list of booleans representing properties, which are required to match in reduce order model. This class focus on reducing errors in distribution, first order moment about origin, second order moment about origin and correlation of samples. Example: properties = [True, True, False, False] will minimize errors in distribution and errors in first order moment about origin in reduce order model.

    Default: weights_errors = [True, True, True, False]

  • weights_distribution (ndarray or list of float):

    An list or array containing weights associated with different samples.

    Options:

    If weights_distribution is None, then default value is assigned. If size of weights_distribution is 1xd, then it is assigned as dot product of weights_distribution and default value. Otherwise size of weights_distribution should be equal to Nxd.

    Default: weights_distribution = An array of shape Nxd with all elements equal to 1.

  • weights_moments (ndarray or list of float):

    An array of dimension 2xd, where ‘d’ is number of random variables. It contain weights associated with moments.

    Options:

    If weights_moments is None, then default value is assigned. If size of weights_moments is 1xd, then it is assigned as dot product of weights_moments and default value. Otherwise size of weights_distribution should be equal to 2xd.

    Default: weights_moments = Square of reciprocal of elements of moments.

  • weights_correlation (ndarray or list of float):

    An array of dimension dxd, where ‘d’ is number of random variables. It contain weights associated with correlation of random variables.

    Default: weights_correlation = dxd dimensional array with all elements equal to 1.

Gaussian Process Regression / Kriging

The Kriging class defines an approximate surrogate model or response surface which can be used to predict the model response and its uncertainty at points wehre the model has not been previously evaluated. Kriging gives the best unbiased linear predictor at the interpolated points. This class generates a model \(\hat{y}\) that express the response as a realization of regression model and Gaussian random process as:

\[\hat{y}(x) = \mathcal{F}(\beta, x) + z(x).\]

The regression model (\(\mathcal{F}\)) is given as a linear combination of ‘\(p\)’ chosen scalar basis functions as:

\[\mathcal{F}(\beta, x) = \beta_1 f_1(x) + \dots + \beta_p f_p(x) = f(x)^T \beta.\]

The random process \(z(x)\) has zero mean and its covariance is defined through the separable correlation function:

\[E\big[z(s)z(x)] = \sigma^2 \mathcal{R}(\theta, s, x)\]

where,

\[\mathcal{R}(s, x; \theta) = \prod_{i=1}^d \mathcal{R}_i(s_i, x_i; \theta_i),\]

and \(\theta\) are a set of hyperparameters generally governing the correlation length of the model determined by maximixing the log-likelihood function

\[\text{log}(p(y|x, \theta)) = -\frac{1}{2}y^T \mathcal{R}^{-1} y - \frac{1}{2}\text{log}(|\mathcal{R}|) - \frac{n}{2}\text{log}(2\pi)\]

The correlation is evaluated between a set of existing sample points \(s\) and points \(x\) in the domain of interest to form the correlation matrix \(R\), and the basis functions are evaluated at the sample points \(s\) to form the matrix \(F\). Using these matrices, the regression coefficients, \(\beta\), and process variance, \(\sigma^2\) are computed as

\[\begin{split}(F^T R^{-1} F)\beta^* & = F^T R^{-1} Y \\ \sigma^2 & = \frac{1}{m} (Y - F\beta^*)^T R{-1}(Y - F\beta^*)\end{split}\]

The final predictor function is then given by:

\[\hat{y}(x) = f(x)^T \beta^* + r(x)^T R^{-1}(Y - F\beta^*)\]

Regression Models

The Kriging class offers a variety of built-in regression models, specified by the reg_model input described below.

Ordinary Kriging

In ordinary Kriging, the regression model is assumed to take a constant value such that

\[\mathcal{F}(\beta, x) = \beta_0\]

Universal Kriging

In universal Kriging, the regression model is assumed to take a general functional form. The Kriging class currenly supports two univeral Kriging models, the linear regression model given by:

\[\mathcal{F}(\beta, x) = \beta_0 = \sum_{i=1}^d \beta_i x_i\]

and the quadratic regression model given by:

\[\mathcal{F}(\beta, x) = \beta_0 = \sum_{i=1}^d \beta_i x_i + \sum_{i=1}^d \sum_{j=1}^d \beta_{ij} x_i x_j\]

User-Defined Regression Model

Adding a new regression model to the Kriging class is straightforward. This is done by creating a new method that evaluates the basis functions and the Jacobian. This method may be passed directly as a callable to the reg_model input of the Kriging class. This new method should takes as input the samples points at which to evaluate the model and return two arrays containing the value of the basis functions and the Jacobian at these sample points.

The first output of this function should be a two dimensional numpy array with the first dimension being the number of samples and the second dimension being the number of basis functions.

The second output (i.e. Jacobian of basis function) is a three dimensional numpy array with the first dimension being the number of samples, the second dimension being the number of variables and the third dimension being the number of basis functions.

An example user-defined model is given below:

>>> def constant(points):
>>>     fx = np.ones([points.shape[0], 1])
>>>     jf = np.zeros([points.shape[0], points.shape[1], 1])
>>>     return fx, jf

Correlation Models

The Kriging class offers a variety of built-in correlation models, specified by the corr_model input described below.

Exponential Correlation

The exponential correlation model takes the following form:

\[\mathcal{R}_i(h_i, \theta_i) = \exp\bigg[ -\dfrac{|h_i|}{\theta_i}\bigg]\]

where \(h_i = s_i-x_i\).

Gaussian Correlation

The Gaussian correlation model takes the following form:

\[\mathcal{R}_i(h_i, \theta_i) = \exp\bigg[ -\bigg(\dfrac{h_i}{\theta_i}\bigg)^2\bigg]\]

where \(h_i = s_i-x_i\).

Linear Correlation

The linear correlation model takes the following form:

\[\mathcal{R}_i(h_i, \theta_i) = \max \bigg(0, 1-\dfrac{|h_i|}{\theta_i}\bigg)\]

where \(h_i = s_i-x_i\).

Spherical Correlation

The spherical correlation model takes the following form:

\[\mathcal{R}_i(h_i, \theta_i) = 1 - 1.5\xi_i + 0.5\xi_i^3\]

where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).

Cubic Correlation

The cubic correlation model takes the following form:

\[\mathcal{R}_i(h_i, \theta_i) = 1 - 3\xi_i^2 + 2\xi_i^3\]

where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).

Spline Correlation

The spline correlation model takes the following form:

\[\begin{split}\mathcal{R}_i(h_i, \theta_i) = \begin{cases} 1-1.5\xi_i^2+30\xi_i^3, & 0\leq \xi_i \leq 0.02 \\ 1.25(1-\xi_i)^3, & 0.2<\xi_i<1 \\ 0, & \xi_i \geq 1\end{cases}\end{split}\]

where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).

User-Defined Correlation

Adding a new correlation model to the Kriging class is straightforward. This is done by creating a new method that evaluates the correlation matrix, it’s derivative with respect to the variables and it’s derivative with respect to the hyperparameters. This method takes as input the new points, training points, hyperparameters and two indicators for the computation of the derivative of correlation matrix (i.e. dt and dx).

If both indicators are false, then the method should return correlation matrix, i.e. a 2-D array with first dimension being the number of points and second dimension being the number of training points.

If dx parameter is True, the method should return the derivative of the correlation matrix respect to the variables, i.e. a 3-D array with first dimension being the number of points, second dimension being the number of training points and third dimension being the number of variables.

If dt is True, then the method should return the correlation matrix and it’s derivative with respect to the hyperparameters, i.e. a 3-D array with first dimension being the number of points, second dimension being the number of training points and third dimension being the number of variables.

An example user-defined model is given below:

>>> def Gaussian(x, s, params, dt=False, dx=False):
>>>     x, s = np.atleast_2d(x), np.atleast_2d(s)
>>>     # Create stack matrix, where each block is x_i with all s
>>>     stack = - np.tile(np.swapaxes(np.atleast_3d(x), 1, 2), (1, np.size(s, 0), 1)) + np.tile(s, (np.size(x, 0), 1, 1))
>>>     rx = np.exp(np.sum(-params * (stack ** 2), axis=2))
>>>     if dt:
>>>         drdt = -(stack ** 2) * np.transpose(np.tile(rx, (np.size(x, 1), 1, 1)), (1, 2, 0))
>>>         return rx, drdt
>>>     if dx:
>>>         drdx = 2 * params * stack * np.transpose(np.tile(rx, (np.size(x, 1), 1, 1)), (1, 2, 0))
>>>         return rx, drdx
>>>     return rx

Kriging Class Descriptions

class UQpy.Surrogates.Kriging(reg_model='Linear', corr_model='Exponential', bounds=None, op=True, nopt=1, normalize=True, verbose=False, corr_model_params=None, optimizer=None, random_state=None, **kwargs_optimizer)[source]

Kriging generates an Gaussian process regression-based surrogate model to predict the model output at new sample points.

Inputs:

  • reg_model (str or function):

    reg_model specifies and evaluates the basis functions and their coefficients, which defines the trend of the model.

    Built-in options (string input): ‘Constant’, ‘Linear’, ‘Quadratic’

    The user may also pass a callable function as defined in User-Defined Regression Model above.

  • corr_model (str or function):

    corr_model specifies and evaluates the correlation function.

    Built-in options (string input): ‘Exponential’, ‘Gaussian’, ‘Linear’, ‘Spherical’, ‘Cubic’, ‘Spline’

    The user may also pass a callable function as defined in User-Defined Correlation above.

  • corr_model_params (ndarray or list of floats):

    List or array of initial values for the correlation model hyperparameters/scale parameters.

  • bounds (list of float):

    Bounds on the hyperparameters used to solve optimization problem to estimate maximum likelihood estimator. This should be a closed bound.

    Default: [0.001, 10**7] for each hyperparameter.

  • op (boolean):

    Indicator to solve MLE problem or not. If ‘True’ corr_model_params will be used as initial solution for optimization problem. Otherwise, corr_model_params will be directly use as the hyperparamters.

    Default: True.

  • nopt (int):

    Number of times optimization problem is to be solved with a random starting point.

    Default: 1.

  • verbose (Boolean):

    A boolean declaring whether to write text to the terminal.

    Default value: False

Attributes:

  • beta (ndarray):

    Regression coefficients

  • err_var (ndarray):

    Variance in the error (assumed to be gaussian process)

  • C_inv (ndarray):

    Inverse of cholesky decomposition of the Correlation matrix

Methods:

fit(samples, values, nopt=None, corr_model_params=None)[source]

Fit the surrogate model using the training samples and the corresponding model values.

The user can run this method multiple time after initiating the Kriging class object.

This method updates the samples and parameters of the Kriging object. This method uses corr_model_params from previous run as the starting point for MLE problem unless user provides a new starting point.

Inputs:

  • samples (ndarray):

    ndarray containing the training points.

  • values (ndarray):

    ndarray containing the model evaluations at the training points.

Output/Return:

The fit method has no returns, although it creates the beta, err_var and C_inv attributes of the Kriging class.

jacobian(x)[source]

Predict the gradient of the model at new points.

This method evaluates the regression and correlation model at new sample point. Then, it predicts the gradient using the regression coefficients and the training data.

Input:

  • x (list or numpy array):

    Points at which to evaluate the gradient.

Output:

  • grad_x (list or numpy array):

    Gradient of the surrogate model evaluated at the new points.

predict(x, return_std=False)[source]

Predict the model response at new points.

This method evaluates the regression and correlation model at new sample points. Then, it predicts the function value and standard deviation.

Inputs:

  • x (list or numpy array):

    Points at which to predict the model response.

  • return_std (list or numpy array):

    Indicator to estimate standard deviation.

Outputs:

  • f_x (numpy array):

    Predicted values at the new points.

  • std_f_x (numpy array):

    Standard deviation of predicted values at the new points.

1
  1. Grigoriu, “Reduced order models for random functions. Application to stochastic problems”, Applied Mathematical Modelling, Volume 33, Issue 1, Pages 161-175, 2009.