Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 4

Softmax Forward Layer

For any x i 1...i p from X R n 1 x ... x n p and dimension k of size n k , the forward softmax layer for computes the function defined as

The softmax function is known as the normalized exponential (see [Bishop2006] for exact definitions of softmax).

Problem Statement

Given a p-dimensional tensor X of size n 1 x n 2 x ... x n p , the problem is to compute the p-dimensional tensor Y = (y i 1...i p ) of size n 1 x n 2 x ... x n p such that:

The library supports the numerically stable version of the softmax function:

where