Linear System Solver

At each iteration \(k\) SCS solves the following set of linear equations:

\[\begin{split}\begin{bmatrix} \rho_x I + P & A^\top \\ A & -\mathrm{diag}(\rho_y) \\ \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} z^k_x \\ z^k_y \end{bmatrix}\end{split}\]

for a particular right-hand side \(z^k \in \mathbf{R}^{n+m}\) (the presence of the diagonal scaling \(\rho\) terms is explained in Non-identity DR Scaling). Note that the matrix does not change from iteration to iteration, which is a major advantage of this approach. Moreover, this system can be solved approximately so long as the errors satisfy a summability condition, which permits the use of approximate solvers that can scale to very large problems.

Available linear solvers

Each of the below linear solvers is included in their own binary. If linking against SCS directly, then to switch between them you must compile and link against the right binary. If using SCS via one of the interfaces then you can choose between the different linear solvers using the appropriate settings.

Sparse direct method

The direct method is the default linear system solver in SCS and factorizes the above matrix using a sparse permuted LDL factorization. Then it solves the linear system at each iteration using the cached factors. Since the linear system is quasidefinite we have strong existence guarantees about the factorization. It relies on the external AMD and QDLDL packages.

Sparse indirect method

The indirect method solves the above linear system approximately with a ‘matrix-free’ method. To do this it first reduces the system to solving

\[\begin{split}\begin{align} (\rho_x I + P + A^\top \mathrm{diag}(\rho_y)^{-1} A) r_x & = q_x - A^\top \mathrm{diag}(\rho_y)^{-1} q_y \\ r_y & = \mathrm{diag}(\rho_y)^{-1}(A z_x + q_y). \end{align}\end{split}\]

then solves the positive definite system using using conjugate gradients. Each iteration of CG requires one multiply each of sparse matrices \(P, A, A^\top\). The system is solved up to some tolerance, which is tuned to ensure that the overall algorithm converges. The tolerance decays with iteration \(k\) like \(O(1/k^\gamma)\) where \(\gamma > 1\) and is determined by the constant CG_RATE (defaults to \(1.5\)).

The indirect method has the advantage of not requiring an expensive factorization but typically is slower on a per-iteration basis. In most cases the factorization is relatively cheap so the direct method is the default, however for very large problems the indirect solver can be faster.

Sparse GPU indirect method

The above linear solvers all run on CPU. We also have support for a GPU version of the indirect solver, where the matrix multiplies are all performed on the GPU.

Implementing a new linear solver

In order to implement you own linear system solver, you need to implement the struct ScsLinSysWork that contains the workspace your solver requires, and implement the functions in include/linsys.h as detailed below. See linsys directory for examples.

typedef struct SCS_LIN_SYS_WORK ScsLinSysWork

Struct containing linear system workspace. Implemented by linear solver.

Functions

ScsLinSysWork *scs_init_lin_sys_work(const ScsMatrix *A, const ScsMatrix *P, scs_float *rho_y_vec, scs_float rho_x)

Initialize ScsLinSysWork structure and perform any necessary preprocessing.

Parameters
  • A – A data matrix.

  • P – P data matrix.

  • rho_y_vecrho_y > 0 diagonal entries.

  • rho_xrho_x > 0 float.

Returns

Linear system solver workspace.

void scs_free_lin_sys_work(ScsLinSysWork *w)

Frees ScsLinSysWork structure and associated allocated memory.

Parameters

w – Linear system private workspace.

scs_int scs_solve_lin_sys(ScsLinSysWork *w, scs_float *b, const scs_float *s, scs_float tol)

Solves the linear system required by SCS at each iteration:

\[\begin{split} \begin{bmatrix} (\rho_x I + P) & A^\top \\ A & -\mathrm{diag}(\rho_y) \\ \end{bmatrix} x = b \end{split}\]

for x. Overwrites b with result.

Parameters
  • w – Linear system private workspace.

  • b – Right hand side, contains solution at the end.

  • s – Contains warm-start (may be NULL).

  • tol – Tolerance required for the system solve.

Returns

status < 0 indicates failure.

void scs_update_lin_sys_rho_y_vec(ScsLinSysWork *w, scs_float *rho_y_vec)

Update the linsys workspace when rho_y_vec is changed. For example, a direct method for solving the linear system might need to update the factorization of the matrix.

Parameters
  • w – Linear system private workspace.

  • rho_y_vecrho_y diagonal entries.

const char *scs_get_lin_sys_method(void)

Name of the linear solver.

Returns

name of method.