Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 5
The Stochastic Average Gradient Accelerated (SAGA) [Defazio2014] follows the algorithmic framework of an iterative solver with one exception.
The default method (defaultDense) of SAGA algorithm is a particular case of the iterative solver method with the batch size b=1.
Algorithmic-specific transformation T, set of intrinsic parameters S t defined for the learning η rate, and algorithm-specific vector U and power d of Lebesgue space are defined as follows:
S
t - a matrix of the gradients of smooth terms at point
, where
:
Update of the set of intrinsic parameters
S
t:
.
Convergence check: