Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 1
The adaptive subgradient method (AdaGrad) [Duchi2011] follows the algorithmic framework of an iterative solver with the algorithm-specific transformation T and set of intrinsic parameters S t defined for the learning rate η as follows:
where g i (θ t-1) is the i-th coordinate of the gradient g(θ t-1)
where