Developer Guide for Intel® Data Analytics Acceleration Library 2018
The forward fully-connected layer computes values
for n input arguments x k , weights w ki , weights mask s ki , and biases b i , where k ∈ {1, ..., n}, i ∈ {1, ..., m}, and m is the number of layer outputs. For more details, see Forward Fully-connected Layer.
The backward fully-connected layer computes the following values:
where
E is the objective function used at the training stage, and g j is the input gradient computed on the preceding layer.
Given:
p-dimensional tensor X of size n 1 x ... x n k ... x n p
p-dimensional tensor W of size n 1 x ... x n k-1 x m x n k+1 ... x n p
p-dimensional tensor S of size n 1 x ... x n k-1 x m x n k+1 ... x n p
1-dimensional tensor B of size m
2-dimensional tensor G of size n k x m
The p-dimensional tensor Z of size n 1 x ... x n k ... x n p such that:
Values:
In the above formulas: