Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 1
The forward two-dimensional (2D) locally-connected layer computes the value tensor Y by applying a set of nKernels 2D kernels K of size m 1 x m 2 to the input argument x. The library supports four-dimensional input tensors X∈ R n 1 x n 2 x n 3 x n 4 . Therefore, the following formula applies:
where i + a < n 1, j + b < n 2, and r is the kernel index.
A set of kernels is specific to the selected dimensions of the input argument x.
See [GregorLecun2010] for additional details of the two-dimensional locally-connected layer.
Without loss of generality let's assume that convolution kernels are applied to the last two dimensions.
Given:
Four-dimensional tensor X ∈ R n 1 x n 2 x n 3 x n 4 with input data
Six-dimensional tensor K ∈ R nKernels x l 3 x l 4 x m 2 x m 3 x m 4 with kernel parameters/weights
Three-dimensional tensor B ∈ R nKernels x l 3 x l 4 with the bias of each kernel.
For the above tensors:
and
p
i
is the respective padding.
nGroups is defined as follows: let's assume that n 2 is the group dimension. The input tensor is split along this dimension into nGroups groups, the tensors of values and weights are split into nGroups groups along the nKernels dimension. nKernels and n 2 must be multiples of nGroups. Each group of values is computed using the respective group in tensors of input data, weights, and biases.
The problem is to compute the four-dimensional tensor of values Y ∈ R n 1 x nKernels x l 3 x l 4 such that:
where:
s 3 and s 4 are strides