Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 4

2D Stochastic Pooling Forward Layer

The forward two-dimensional (2D) stochastic pooling layer is a form of non-linear downsampling of an input tensor X R n 1 x n 2 x ... x n p with non-negative elements x i 1 ...i p . 2D stochastic pooling partitions the input tensor data into 2D subtensors along dimensions k 1 and k 2 and applies the transformations (kernels).

At the training stage, the layer selects an element in each subtensor using sampling from a multinomial distribution. Probabilities required in the distribution are calculated by normalizing the subtensor. In the output, the selected element replaces the entire subtensor.

At the prediction stage, the layer replaces each subtensor with the weighted average of its elements using the probabilities as weights. For more details, see [Mattew2013].

Problem Statement

Given:

The problem is to compute the value tensor YR l 1 x ... x l p using the downsampling technique, where:



and s 1 and s 2 are strides.

To define the values of Y, let's apply the kernel to the subtensor at position {i 1, ..., i p } and consider elements of the subtensor as the vector a of size with elements





The probabilities used in the multinomial distribution are defined as:


Training Stage

At the training stage, the layer computes the value y as the element at the position t in the subtensor. The position t is picked as a sample from the multinomial distribution and kept for use in the backward step:





Prediction Stage

At the prediction stage, the layer computes the value as a weighted average of elements of the vector a that represents the subtensor: