Surrogates¶
This module contains functionality for all the surrogate methods supported in UQpy.
The module currently contains the following classes:
SROM
: Class to estimate a discrete approximation for a continuous random variable using Stochastic Reduced OrderModel.
Kriging
: Class to generate an approximate surrogate model using Kriging.
Stochatic Reduced Order Models - SROMs¶
An SROM is a sample-based surrogate for probability models. An SROM takes a set of samples and attributes of a distribution and optimizes the sample probability weights according to the method in 1. More specifically, an SROM constructs a reduce order model for arbitrary random variables X as follows.
where \(\tilde{X}\) is defined by an arbitrary set of samples \(x_1, \dots, x_m\) defined over the same support as \(X\) (but not necessarily drawn from its probability distribution) and their assigned probability weights. The probability weights are defined such that the total error between the sample empirical probability distribution, moments and correlation of \(\tilde{X}\) and those of the random variable X is minimized. This optimization problem can be express as:
where \(\alpha_1\), \(\alpha_2\), \(\alpha_3 \geq 0\) are constants defining the relative importance of the marginal distribution, moments and correlation error between the reduce order model and actual random variables in the objective function.
Here, \(F(x_{k,i})\) and \(\hat{F}(x_{k,i})\) denote the marginal cumulative distributions of \(\mathbf{X}\) and \(\mathbf{\tilde{X}}\) (reduced order model) evaluated at point \(x_{k,i}\), \(\mu(r; i)\) and \(\hat{\mu}(r; i)\) are the marginal moments of order r for variable i, and \(R(i,j)\) and \(\hat{R}(i,j)\) are correlation matrices of \(\mathbf{X}\) and \(\mathbf{\tilde{X}}\) evaluted for components \(x_i\) and \(x_j\). Note also that m is the number of sample points and d is the number of random variables.
SROM Class Descriptions¶
-
class
UQpy.Surrogates.
SROM
(samples, target_dist_object, moments=None, weights_errors=None, weights_distribution=None, weights_moments=None, weights_correlation=None, properties=None, correlation=None, verbose=False)[source]¶ Stochastic Reduced Order Model(SROM) provide a low-dimensional, discrete approximation of a given random quantity.
Inputs:
- samples (ndarray):
An array/list of samples corresponding to each random variables.
- target_dist_object ((list of)
Distribution
object(s)): A list of distribution objects of random variables.
- target_dist_object ((list of)
- moments (list of float):
A list containing first and second order moment about origin of all random variables.
- weights_errors (list of float):
Weights associated with error in distribution, moments and correlation.
Default: weights_errors = [1, 0.2, 0]
- properties (list of booleans):
A list of booleans representing properties, which are required to match in reduce order model. This class focus on reducing errors in distribution, first order moment about origin, second order moment about origin and correlation of samples. Example: properties = [True, True, False, False] will minimize errors in distribution and errors in first order moment about origin in reduce order model.
Default: properties = [True, True, True, False]
- weights_distribution (ndarray or list of float):
An list or array containing weights associated with different samples.
Options:
If weights_distribution is None, then default value is assigned. If size of weights_distribution is 1xd, then it is assigned as dot product of weights_distribution and default value. Otherwise size of weights_distribution should be equal to Nxd.
Default: weights_distribution = An array of shape Nxd with all elements equal to 1.
- weights_moments (ndarray or list of float):
An array of dimension 2xd, where ‘d’ is number of random variables. It contain weights associated with moments.
Options:
If weights_moments is None, then default value is assigned. If size of weights_moments is 1xd, then it is assigned as dot product of weights_moments and default value. Otherwise size of weights_distribution should be equal to 2xd.
Default: weights_moments = Square of reciprocal of elements of moments.
- weights_correlation (ndarray or list of float):
An array of dimension dxd, where ‘d’ is number of random variables. It contain weights associated with correlation of random variables.
Default: weights_correlation = dxd dimensional array with all elements equal to 1.
- correlation (ndarray or list of floats):
Correlation matrix between random variables.
Attribute:
- sample_weights (ndarray):
The probabilities/weights defining discrete approximation of continuous random variables.
Methods:
-
run
(weights_errors=None, weights_distribution=None, weights_moments=None, weights_correlation=None, properties=None)[source]¶ Execute the stochastic reduced order model in the
SROM
class.The
run
method is the function that computes the probability weights corresponding to the sample. If properties is provided, therun
method is automatically called when theSROM
object is defined. The user may also call therun
method directly to generate samples. Therun
method of theSROM
class can be invoked many times with different weights parameters and each time computed probability weights are overwritten.Inputs:
- weights_errors (list of float):
Weights associated with error in distribution, moments and correlation.
Default: weights_errors = [1, 0.2, 0]
- properties (list of booleans):
A list of booleans representing properties, which are required to match in reduce order model. This class focus on reducing errors in distribution, first order moment about origin, second order moment about origin and correlation of samples. Example: properties = [True, True, False, False] will minimize errors in distribution and errors in first order moment about origin in reduce order model.
Default: weights_errors = [True, True, True, False]
- weights_distribution (ndarray or list of float):
An list or array containing weights associated with different samples.
Options:
If weights_distribution is None, then default value is assigned. If size of weights_distribution is 1xd, then it is assigned as dot product of weights_distribution and default value. Otherwise size of weights_distribution should be equal to Nxd.
Default: weights_distribution = An array of shape Nxd with all elements equal to 1.
- weights_moments (ndarray or list of float):
An array of dimension 2xd, where ‘d’ is number of random variables. It contain weights associated with moments.
Options:
If weights_moments is None, then default value is assigned. If size of weights_moments is 1xd, then it is assigned as dot product of weights_moments and default value. Otherwise size of weights_distribution should be equal to 2xd.
Default: weights_moments = Square of reciprocal of elements of moments.
- weights_correlation (ndarray or list of float):
An array of dimension dxd, where ‘d’ is number of random variables. It contain weights associated with correlation of random variables.
Default: weights_correlation = dxd dimensional array with all elements equal to 1.
Gaussian Process Regression / Kriging¶
The Kriging
class defines an approximate surrogate model or response surface which can be used to predict the model response and its uncertainty at points wehre the model has not been previously evaluated. Kriging gives the best unbiased linear predictor at the interpolated points. This class generates a model \(\hat{y}\) that express the response as a realization of regression model and Gaussian random process as:
The regression model (\(\mathcal{F}\)) is given as a linear combination of ‘\(p\)’ chosen scalar basis functions as:
The random process \(z(x)\) has zero mean and its covariance is defined through the separable correlation function:
where,
and \(\theta\) are a set of hyperparameters generally governing the correlation length of the model determined by maximixing the log-likelihood function
The correlation is evaluated between a set of existing sample points \(s\) and points \(x\) in the domain of interest to form the correlation matrix \(R\), and the basis functions are evaluated at the sample points \(s\) to form the matrix \(F\). Using these matrices, the regression coefficients, \(\beta\), and process variance, \(\sigma^2\) are computed as
The final predictor function is then given by:
Regression Models¶
The Kriging
class offers a variety of built-in regression models, specified by the reg_model input described below.
Ordinary Kriging¶
In ordinary Kriging, the regression model is assumed to take a constant value such that
Universal Kriging¶
In universal Kriging, the regression model is assumed to take a general functional form. The Kriging
class currenly supports two univeral Kriging models, the linear regression model given by:
and the quadratic regression model given by:
User-Defined Regression Model¶
Adding a new regression model to the Kriging
class is straightforward. This is done by creating a new method that evaluates the basis functions and the Jacobian. This method may be passed directly as a callable to the reg_model input of the Kriging
class. This new method should takes as input the samples points at which to evaluate the model and return two arrays containing the value of the basis functions and the Jacobian at these sample points.
The first output of this function should be a two dimensional numpy array with the first dimension being the number of samples and the second dimension being the number of basis functions.
The second output (i.e. Jacobian of basis function) is a three dimensional numpy array with the first dimension being the number of samples, the second dimension being the number of variables and the third dimension being the number of basis functions.
An example user-defined model is given below:
>>> def constant(points):
>>> fx = np.ones([points.shape[0], 1])
>>> jf = np.zeros([points.shape[0], points.shape[1], 1])
>>> return fx, jf
Correlation Models¶
The Kriging
class offers a variety of built-in correlation models, specified by the corr_model input described below.
Exponential Correlation¶
The exponential correlation model takes the following form:
where \(h_i = s_i-x_i\).
Gaussian Correlation¶
The Gaussian correlation model takes the following form:
where \(h_i = s_i-x_i\).
Linear Correlation¶
The linear correlation model takes the following form:
where \(h_i = s_i-x_i\).
Spherical Correlation¶
The spherical correlation model takes the following form:
where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).
Cubic Correlation¶
The cubic correlation model takes the following form:
where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).
Spline Correlation¶
The spline correlation model takes the following form:
where \(\xi_i = \min \bigg(1, \dfrac{|h_i|}{\theta_i}\bigg)\) and \(h_i = s_i-x_i\).
User-Defined Correlation¶
Adding a new correlation model to the Kriging
class is straightforward. This is done by creating a new method that evaluates the correlation matrix, it’s derivative with respect to the variables and it’s derivative with respect to the hyperparameters. This method takes as input the new points, training points, hyperparameters and two indicators for the computation of the derivative of correlation matrix (i.e. dt and dx).
If both indicators are false, then the method should return correlation matrix, i.e. a 2-D array with first dimension being the number of points and second dimension being the number of training points.
If dx parameter is True, the method should return the derivative of the correlation matrix respect to the variables, i.e. a 3-D array with first dimension being the number of points, second dimension being the number of training points and third dimension being the number of variables.
If dt is True, then the method should return the correlation matrix and it’s derivative with respect to the hyperparameters, i.e. a 3-D array with first dimension being the number of points, second dimension being the number of training points and third dimension being the number of variables.
An example user-defined model is given below:
>>> def Gaussian(x, s, params, dt=False, dx=False):
>>> x, s = np.atleast_2d(x), np.atleast_2d(s)
>>> # Create stack matrix, where each block is x_i with all s
>>> stack = - np.tile(np.swapaxes(np.atleast_3d(x), 1, 2), (1, np.size(s, 0), 1)) + np.tile(s, (np.size(x, 0), 1, 1))
>>> rx = np.exp(np.sum(-params * (stack ** 2), axis=2))
>>> if dt:
>>> drdt = -(stack ** 2) * np.transpose(np.tile(rx, (np.size(x, 1), 1, 1)), (1, 2, 0))
>>> return rx, drdt
>>> if dx:
>>> drdx = 2 * params * stack * np.transpose(np.tile(rx, (np.size(x, 1), 1, 1)), (1, 2, 0))
>>> return rx, drdx
>>> return rx
Kriging Class Descriptions¶
-
class
UQpy.Surrogates.
Kriging
(reg_model='Linear', corr_model='Exponential', bounds=None, op=True, nopt=1, normalize=True, verbose=False, corr_model_params=None, optimizer=None, random_state=None, **kwargs_optimizer)[source]¶ Kriging generates an Gaussian process regression-based surrogate model to predict the model output at new sample points.
Inputs:
- reg_model (str or function):
reg_model specifies and evaluates the basis functions and their coefficients, which defines the trend of the model.
Built-in options (string input): ‘Constant’, ‘Linear’, ‘Quadratic’
The user may also pass a callable function as defined in User-Defined Regression Model above.
- corr_model (str or function):
corr_model specifies and evaluates the correlation function.
Built-in options (string input): ‘Exponential’, ‘Gaussian’, ‘Linear’, ‘Spherical’, ‘Cubic’, ‘Spline’
The user may also pass a callable function as defined in User-Defined Correlation above.
- corr_model_params (ndarray or list of floats):
List or array of initial values for the correlation model hyperparameters/scale parameters.
- bounds (list of float):
Bounds on the hyperparameters used to solve optimization problem to estimate maximum likelihood estimator. This should be a closed bound.
Default: [0.001, 10**7] for each hyperparameter.
- op (boolean):
Indicator to solve MLE problem or not. If ‘True’ corr_model_params will be used as initial solution for optimization problem. Otherwise, corr_model_params will be directly use as the hyperparamters.
Default: True.
- nopt (int):
Number of times optimization problem is to be solved with a random starting point.
Default: 1.
- verbose (Boolean):
A boolean declaring whether to write text to the terminal.
Default value: False
Attributes:
- beta (ndarray):
Regression coefficients
- err_var (ndarray):
Variance in the error (assumed to be gaussian process)
- C_inv (ndarray):
Inverse of cholesky decomposition of the Correlation matrix
Methods:
-
fit
(samples, values, nopt=None, corr_model_params=None)[source]¶ Fit the surrogate model using the training samples and the corresponding model values.
The user can run this method multiple time after initiating the
Kriging
class object.This method updates the samples and parameters of the
Kriging
object. This method uses corr_model_params from previous run as the starting point for MLE problem unless user provides a new starting point.Inputs:
- samples (ndarray):
ndarray containing the training points.
- values (ndarray):
ndarray containing the model evaluations at the training points.
Output/Return:
The
fit
method has no returns, although it creates the beta, err_var and C_inv attributes of theKriging
class.
-
jacobian
(x)[source]¶ Predict the gradient of the model at new points.
This method evaluates the regression and correlation model at new sample point. Then, it predicts the gradient using the regression coefficients and the training data.
Input:
- x (list or numpy array):
Points at which to evaluate the gradient.
Output:
- grad_x (list or numpy array):
Gradient of the surrogate model evaluated at the new points.
-
predict
(x, return_std=False)[source]¶ Predict the model response at new points.
This method evaluates the regression and correlation model at new sample points. Then, it predicts the function value and standard deviation.
Inputs:
- x (list or numpy array):
Points at which to predict the model response.
- return_std (list or numpy array):
Indicator to estimate standard deviation.
Outputs:
- f_x (numpy array):
Predicted values at the new points.
- std_f_x (numpy array):
Standard deviation of predicted values at the new points.
- 1
Grigoriu, “Reduced order models for random functions. Application to stochastic problems”, Applied Mathematical Modelling, Volume 33, Issue 1, Pages 161-175, 2009.