Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 4
The forward two-dimensional (2D) locally-connected layer computes the value tensor Y by applying a set of nKernels 2D kernels K of size m 1 x m 2 to the input argument x. The library supports four-dimensional input tensors X∈ R n 1 x n 2 x n 3 x n 4 . Therefore, the following formula applies:
where i + a < n 1, j + b < n 2, and r is the kernel index.
A set of kernels is specific to the selected dimensions of the input argument x.
For more details, see Forward 2D Locally-connected Layer.
The backward 2D locally-connected layer computes the derivatives of the objective function E with respect to the input argument, weights, and biases.
Without loss of generality, let's assume that convolution kernels are applied to the last two dimensions.
Given:
Four-dimensional tensor G ∈ R n 1 x nKernels x l 3 x l 4 with the gradient from the preceding layer
Four-dimensional tensor X ∈ R n 1 x n 2 x n 3 x n 4 with input data of the forward layer
Six-dimensional tensor K ∈ R nKernels x l 3 x l 4 x m 2 x m 3 x m 4 with kernel parameters/weights
Three-dimensional tensor B ∈ R nKernels x l 3 x l 4 with the bias of each kernel.
For the above tensors:
and
p
i
is the respective padding.
nGroups is defined as follows: let's assume that n 2 is the group dimension in the input tensor for the forward 2D locally-connected layer. The output gradient tensor is split along this dimension into nGroups groups, and the input gradient tensor and weights tensor are split into nGroups groups along the nKernels dimension. nKernels and n 2 must be multiples of nGroups.
The problem is to compute:
Four-dimensional tensor Z ∈ R n 1 x n 2 x n 3 x n 4 such that:
Values:
In the above formulas:
s 3 and s 4 are strides