Developer Guide for Intel® Data Analytics Acceleration Library 2018 Update 2

Batch Normalization Backward Layer

The forward batch normalization layer normalizes x i 1...i p from the input XR n 1 x n 2 x ... x n p for the dimension k ∈ {1, ... p} and then scales and shifts the result of the normalization . For more details, see Forward Batch Normalization Layer. The backward batch normalization layer [Ioffe2015] computes the values for the dimension k ∈ {1, ... p}:

where

Problem Statement

Given p-dimensional tensors:

The problem is to compute the p-dimensional tensor Z R n 1 x n 2 x ... x n p such that:

for j = 1, ..., n k , where: