Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 5
The forward two-dimensional (2D) stochastic pooling layer is a form of non-linear downsampling of an input tensor X ∈ R n 1 x n 2 x ... x n p with non-negative elements x i 1 ...i p . 2D stochastic pooling partitions the input tensor data into 2D subtensors along dimensions k 1 and k 2 and applies the transformations (kernels).
At the training stage, the layer selects an element in each subtensor using sampling from a multinomial distribution. Probabilities required in the distribution are calculated by normalizing the subtensor. In the output, the selected element replaces the entire subtensor.
At the prediction stage, the layer replaces each subtensor with the weighted average of its elements using the probabilities as weights. For more details, see [Mattew2013].
Given:
The tensor X ∈ R n 1 x n 2 x ... x n p of input data with non-negative elements x i 1 ...i p
Dimensions k 1 and k 2 along which kernels are applied
Kernel sizes
m
1 and
m
2:
where
p
1 and
p
2 are paddings
The problem is to compute the value tensor Y ∈ R l 1 x ... x l p using the downsampling technique, where:
To define the values of
Y, let's apply the kernel to the subtensor at position {i
1, ..., i
p
} and consider elements of the subtensor as the vector
a of size
with elements
Training Stage
At the training stage, the layer computes the value y as the element at the position t in the subtensor. The position t is picked as a sample from the multinomial distribution and kept for use in the backward step:
Prediction Stage
At the prediction stage, the layer computes the value as a weighted average of elements of the vector a that represents the subtensor: