Mondaic

salvus.opt.methods.lbfgs

salvus.opt.methods.lbfgs salvus opt methods lbfgs

L-BFGS approximation of Hessian and inverse Hessian.

Functions

apply_inverse_hessian()

def apply_inverse_hessian(
    model: ModelType,
    current_iteration: Iteration[ModelType],
    available_iterations: dict[int, Iteration[ModelType]],
    max_lbfgs_history: int,
) -> ModelType: ...

Apply LBFGS inverse Hessian approximation to gradient of current iteration.

This method neither uses a custom initial Hessian nor a preconditioner. For either of those, see the two-stage procedure with apply_inverse_hessian_part1 and apply_inverse_hessian_part2 instead.

Parameters
  • model ModelType — Model to apply the inverse L-BFGS Hessian to.
  • current_iteration Iteration[ModelType] — The current iteration.
  • available_iterations dict[int, Iteration[ModelType]] — History of iterations.
  • max_lbfgs_history int — Maximum number of previous iterations to consider for the L-BFGS approximation.
Returns ModelType

apply_inverse_hessian_part1()

def apply_inverse_hessian_part1(
    model: ModelType,
    current_iteration: Iteration[ModelType],
    available_iterations: dict[int, Iteration[ModelType]],
    max_lbfgs_history: int,
) -> tuple[ModelType, float, list[LBFGSData]]: ...

Apply LBFGS inverse Hessian approximation to model.

This computes the first part until the application of the initial Hessian approximation.

Parameters
  • model ModelType — Model to precondition.
  • current_iteration Iteration[ModelType] — The current iteration.
  • available_iterations dict[int, Iteration[ModelType]] — History of iterations
  • max_lbfgs_history int — Maximum number of previous iterations to consider for the L-BFGS approximation.
Returns tuple[ModelType, float, list[LBFGSData]] — model: Preconditioned model. gamma: Scaling factor for part 2. lbfgs_data: List of L-BFGS auxiliary data for part 2 sorted from oldest to newest.

apply_inverse_hessian_part2()

def apply_inverse_hessian_part2(
    preconditioned_model: ModelType,
    current_iteration: Iteration[ModelType],
    available_iterations: dict[int, Iteration[ModelType]],
    lbfgs_data: list[LBFGSData],
    gamma: float,
) -> ModelType: ...

Apply LBFGS inverse Hessian approximation to model.

This computes the second part, which returns the product of inverse Hessian approximation times the initially provided model.

Parameters
  • preconditioned_model ModelType — Preconditioned model from part 1.
  • current_iteration Iteration[ModelType] — The current iteration.
  • available_iterations dict[int, Iteration[ModelType]] — Iterations available to the optimizer.
  • lbfgs_data list[LBFGSData] — Precomputed quantities for iteration pairs in part 1, sorted from oldest to newest.
  • gamma float — Scaling factor from part 1.
Returns ModelType — The finalized matrix-vector product with the inverse L-BFGS Hessian.

apply_lbfgs_hessian()

def apply_lbfgs_hessian(
    model: ModelType,
    lbfgs_data: dict[str, typing.Any],
    n_memory: int | None = None,
) -> ModelType: ...

Apply LBFGS Hessian approximation to model.

Parameters
  • model ModelType — Model to apply the LBFGS Hessian to.
  • lbfgs_data dict[str, typing.Any] — L-BFGS data containing the sorted diffs of models and gradients.
  • n_memory int | None — Optional length of the L-BFGS history to only consider a subset of lbfgs_data.
Returns ModelType

setup_lbfgs_data()

def setup_lbfgs_data(
    models: list[ModelType] | None = None,
    gradients: list[ModelType] | None = None,
) -> dict: ...

Build LBFGS auxiliary data from history of models and gradients. The input lists must be ordered from oldest to newest model.

Parameters
  • models list[ModelType] | None — List of previous models.
  • gradients list[ModelType] | None — List of previous gradients.
Returns dict

Classes

LBFGSData

class LBFGSData(builtins.object):
    def __init__(
        self,
        current_iteration_id: int,
        previous_iteration_id: int,
        rho: float,
        alpha: float,
    ) -> None: ...

Data structure for L-BFGS approximation.

Naming convention follows standard notation from L-BFGS literature, e.g. Chapter 9 in Numerical Optimization by Nocedal and Wright.

Parameters
  • current_iteration_id int — ID of the current iteration.
  • previous_iteration_id int — ID of the previous iteration.
  • rho float — Inverse dot product of model diff and gradient diff.
  • alpha float — Scaling factor.
Methods
to_dict()
def to_dict(self) -> dict: ...

Convert the L-BFGS data to a dictionary.

Returns dict