Developer Guide for Intel® Data Analytics Acceleration Library 2018
The limited-memory BFGS algorithm is a special case of an iterative solver. For parameters, input, and output of iterative solvers, see Iterative Solver > Computation.
In addition to the input of the iterative solver, the limited-memory BFGS algorithm accepts the following optional input:
OptionalDataID |
Input |
|
---|---|---|
correctionPairs |
Numeric table of size 2m x p where the rows represent correction pairs s and y. The row correctionPairs[j], 0 ≤ j < m, is a correction vector sj, and the row correctionPairs[j], m ≤ j < 2m, is a correction vector yj. |
|
correctionIndices |
Numeric table of size 1 x 2 with 32-bit integer indexes. The first value is the index of correction pair t, the second value is the index of last iteration k from the previous run. |
|
averageArgumentLIterations |
Numeric table of size 2 x p, where row 0 represents average arguments for previous L iterations, and row 1 represents average arguments for last L iterations. These values are required to compute s correction vectors in the next step. See step 6.b.iii of the limited-memory BFGS algorithm. |
In addition to parameters of the iterative solver, the limited-memory BFGS algorithm has the following parameters:
Parameter |
Default Value |
Description |
|
---|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
|
method |
defaultDense |
Performance-oriented computation method. |
|
batchIndices |
NULL |
The numeric table of size nIterations x batchSize with 32-bit integer indices of terms in the objective function to be used in step 2 of the limited-memory BFGS algorithm. If no indices are provided, the implementation generates random indices. This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. |
|
batchSize |
10 |
The number of observations to compute the stochastic gradient. The implementation of the algorithm ignores this parameter if the batchIndices numeric table is provided. If BatchSize equals the number of terms in the objective function, no random sampling is performed and all terms are used to calculate the gradient. |
|
correctionPairBatchSize |
100 |
The number of observations to compute the sub-sampled Hessian for correction pairs computation in step 6.b.ii of the limited-memory BFGS algorithm. The implementation of the algorithm ignores this parameter if the correctionPairIndices numeric table is provided. If correctionPairBatchSize equals the number of terms in the objective function, no random sampling is performed and all terms are used to calculate the Hessian matrix. |
|
correctionPairIndices |
NULL |
The numeric table of size (nIterations/L) x correctionPairBatchSize with 32-bit integer indices to be used instead of random values in step 6.b.i of the limited-memory BFGS algorithm. If no indices are provided, the implementation generates random indices. This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. NoteIf the algorithm runs with no optional input data, (nIterations / L - 1) rows of the table are used. Otherwise, it can use one more row, (nIterations / L) in total. |
|
m |
10 |
The memory parameter. The maximum number of correction pairs that define the approximation of the Hessian matrix. |
|
L |
10 |
The number of iterations between calculations of the curvature estimates. |
|
Numeric table of size 1 x 1 that contains the default step length equal to 1. |
The numeric table of size 1 x nIterations or 1 x 1. The contents of the table depend on its size:
This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. The recommended data type for storing the step-length sequence is the floating-point type, either float or double, that the algorithm uses in intermediate computations. |
||
DEPRECATED: seed |
777 |
NoteThis parameter is deprecated and will be removed in a future release.The seed for randomly choosing terms from the objective function. |
|
engine |
SharePtr< engines:: mt19937:: Batch>() |
Pointer to the random number generator engine that is used internally for random choosing terms from the objective function. |
In addition to the output of the iterative solver, the limited-memory BFGS algorithm calculates the following optional results:
OptionalDataID |
Output |
|
---|---|---|
correctionPairs |
Numeric table of size 2m x p where the rows represent correction pairs s and y. The row correctionPairs[j], 0 ≤ j < m, is a correction vector sj, and the row correctionPairs[j], m ≤ j < 2m, is a correction vector yj. |
|
correctionIndices |
Numeric table of size 1 x 2 with 32-bit integer indexes. The first value is the index of correction pair t, the second value is the index of last iteration k from the previous run. |
|
averageArgumentLIterations |
Numeric table of size 2 x p, where row 0 represents average arguments for previous L iterations, and row 1 represents average arguments for last L iterations. These values are required to compute s correction vectors in the next step. See step 6.b.iii of the limited-memory BFGS algorithm. |