Solvers

diamond_logistic

Diamond solver for logistic regression

class diamond.solvers.diamond_logistic.FixedHessianSolverMulti[source]

Bases: object

This class wraps the fit method

static fit(Y, designs, H_invs, sparse_inv_covs, **kwargs)[source]

Fit the model. Outer iterations loop over main effects and each grouping factor

Parameters:
  • Y – array_like. Vector of binary responses in {0, 1}
  • designs – dict. Design matrices for main effects and grouping factors
  • H_invs – dict. dictionary of inverse Hessian matrix for each grouping factor
  • sparse_inv_covs – dict. dictionary of sparse regularization matrices, one for each grouping factor
Keyword Arguments:
 
  • min_its – int. Minimum number of outer iterations
  • max_its – int. Maximum number of outer iterations
  • tol – float. If parameters change by less than tol, convergence has been achieved.
  • inner_tol – float. Tolerance for inner loops
  • initial_offset – array_like. Offset vector. Defaults to 0
  • fit_order – list. Order in which to fit main and random effects
  • permute_fit_order – boolean. Change the fit order at each iteration
  • verbose – boolean. Display updates at every iteration
Returns:

estimated intercepts, main effects, and random effects
Return type:

dict

diamond_cumulative_logistic

Diamond solver for cumulative logistic regression

class diamond.solvers.diamond_cumulative_logistic.FixedHessianSolverCumulative[source]

Bases: object

This class wraps the fit method

static fit(Y, main_design, inter_designs, H_inter_LUs, penalty_matrices, effects, **kwargs)[source]

Fit the model. Outer iterations loop over intercepts, main effects, and each grouping factor

Parameters:
  • Y – array_like. Response matrix.
  • main_design – array_like. Design matrix for main effects
  • inter_designs – dict of array_like: design matrices for random effects
  • H_inter_LUs – dict of LU-decompositions of approximate Hessian matrices, one for each grouping factor
  • penalty_matrices – dict of sparse matrices for regularization
  • effects – dict. Current value of parameter estimates
Keyword Arguments:
 
  • min_its – int. Minimum number of outer iterations
  • max_its – int. Maximum number of outer iterations
  • tol – float. If parameters change by less than tol, convergence has been achieved.
  • inner_tol – float. Tolerance for inner loops
  • offset – array_like. Offset vector. Defaults to 0
  • verbose – boolean. Display updates at every iteration
Returns:

dict of estimated intercepts, main effects, and random effects

repeated_block_diag

class diamond.solvers.repeated_block_diag.RepeatedBlockDiagonal(block, num_blocks)[source]

Bases: object

dot(x)[source]

Perform dot product between RepeatedBlockDiagonal objects

Parameters:x – RepeatedBlockDiagonal.
Returns:repeated_block_dot with dot product of the 2 matrices
sparse_matrix

Create a block diagonal sparse matrix, with a single repeated block

This method essentially replicates the functionality of scipy.linalg.block_diag. We rewrote it for this special case to improve the memory footprint - scipy’s version is general purpose, allowing different blocks along the diagonal. The implementation of it is too memory intensive and quickly causes OOM errors with resonably sized design matrices. In our case, we only have one block we are replicating along the diagonal, so we don’t need to iteratively build up the overall block diagonal matrix.

Returns:csr block diagonal matrix
diamond.solvers.repeated_block_diag.test_repeated_block()[source]

repeated_block_dot

diamond.solvers.repeated_block_dot.repeated_block_dot()

utils

Helper functions for diamond solvers. Mostly linear algebra/gradients/Hessians. Includes logistic and cumulative logistic functions

diamond.solvers.utils.custom_block_diag(blocks)[source]

create csr sparse block diagonal matrix from identically-sized blocks

Blocks don’t need to be identical, but they do need to be the same shape.

diamond.solvers.utils.dot(x, y)[source]

A generic dot product for sparse and dense vectors

Parameters:
  • x – array_like, possibly sparse
  • y – array_like, possibly sparse
Returns:

dot product of x and y
diamond.solvers.utils.l2_clogistic_fixef(X, Y, **kwargs)[source]

Take a true, second-order Newton step for fixed effects

Parameters:
  • X – array_like. design matrix
  • Y – array_like. response matrix
  • alpha – array_like. intercepts. must have shape == one less than the number of columns of Y
  • beta – array_like. parameters. must have shape == number of columns of X
  • min_its – int, optional. Minimum number of iterations
  • max_its – int, optional. Maximum number of iterations
  • tol – float, optional. Convergence tolerance on relative change in alpha and beta
  • penalty_matrix – array_like, optional.
Returns:

Updated intercepts, updated main effects, both array-like
diamond.solvers.utils.l2_clogistic_gradient(X, Y, intercept=True, **kwargs)[source]

Gradient of cumulative logit model

Parameters:
  • X – array_like. design matrix
  • Y – array_like. response matrix
  • intercept – boolean. If intercept=False, just calculate the gradient for the coefficients
  • offset – array_like, optional. Defaults to 0
  • alpha – array_like, optional. intercepts. must have shape == one less than the number of columns of Y
  • beta – array_like, optional. parameters. must have shape == number of columns of X
  • penalty_matrix – array_like, optional. Regularization matrix
Returns:

dense array of the GRADIENT of the NEGATIVE, penalized loglikelihood function
diamond.solvers.utils.l2_clogistic_hessian(X, Y, intercept=True, **kwargs)[source]

Hessian matrix of proportional odds cumulative logit model

Parameters:
  • X – array_like. design matrix
  • Y – array_like. response matrix
  • intercept – boolean. If intercept=False, just calculate the gradient for the coefficients
  • alpha – array_like, optional.intercepts. must have shape == one less than the number of columns of Y
  • beta – array_like, optional. parameters. must have shape == number of columns of X
  • offset – array_like, optional. Defaults to 0
  • penalty_matrix – array_like, optional.
Returns:

Square matrix of dimension == number of columns of X
diamond.solvers.utils.l2_clogistic_llh(X, Y, alpha, beta, penalty_matrix, offset)[source]

Penalized log likelihood function for proportional odds cumulative logit model

Parameters:
  • X – array_like. design matrix
  • Y – array_like. response matrix
  • alpha – array_like. intercepts. must have shape == one less than the number of columns of Y
  • beta – array_like. parameters. must have shape == number of columns of X
  • penalty_matrix – array_like. Regularization matrix
  • offset – array_like, optional. Defaults to 0
Returns:

penalized loglikelihood
Return type:

scalar
diamond.solvers.utils.l2_clogistic_ranef(X, Y, alpha, beta, penalty_matrix, offset, LU, **kwargs)[source]

Use a fixed inverse hessian to determine step direction this inverse hessian is represented by the LU decomposition of the approximate Hessian

Parameters:
  • X – array_like. design matrix
  • Y – array_like. response matrix
  • alpha – array_like. intercepts. must have shape == one less than the number of columns of Y
  • beta – array_like. parameters. must have shape == number of columns of X
  • penalty_matrix – array_like
  • offset – array_like, optional. Defaults to 0
  • LU – LU decomposition of approximate Hessian matrix
  • min_its – int, optional. Minimum number of iterations to take
  • max_its – int, optional. Maximum number of iterations to take
  • tol – float. Convergence tolerance on relative change in beta
Returns:

Updated parameter vector beta
diamond.solvers.utils.l2_logistic_fixed_hessian(X, Y, fixed_hess_inv, penaltysq=None, min_its=2, max_its=200, beta=None, offset=None, tol=0.001)[source]

Fit a l2-regularized logistic regression with a fixed-Hessian Newtonish method.

Parameters:
  • X – array_like. Design matrix
  • Y – array_like. Vector of responses with values in {0, 1}
  • fixed_hess_inv – sparse penalty matrix
  • penalty_sq – array_like. Square penalty matrix
  • min_its – int. Minimum number of iterations
  • max_its – int. Maximum number of iterations
  • beta – array_like. Vector of initial parameters, with length == number of columns of X
  • offset – array_like. Added to X * beta. Defaults to 0
  • tol – float. Convergence tolerance on relative change in beta
Returns:

array_like. Estimated parameters
diamond.solvers.utils.solve(A, b)[source]

A generic linear solver for sparse and dense matrices

Parameters:
  • A – array_like, possibly sparse
  • b – array_like
Returns:

x such that Ax = b