Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 4
For any x i 1...i p from X ∈ R n 1 x ... x n p and dimension k of size n k , the forward softmax layer for computes the function defined as
The softmax function is known as the normalized exponential (see [Bishop2006] for exact definitions of softmax).
Given a p-dimensional tensor X of size n 1 x n 2 x ... x n p , the problem is to compute the p-dimensional tensor Y = (y i 1...i p ) of size n 1 x n 2 x ... x n p such that:
The library supports the numerically stable version of the softmax function:
where