Developer Guide for Intel® Data Analytics Acceleration Library 2019 Update 5
The forward batch normalization layer accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.
Input ID |
Input |
|
---|---|---|
data |
Tensor of size n1 x n2 x ... x np that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from Tensor. |
|
weights |
One-dimensional tensor of size nk that stores weights for scaling ω (k). This input can be an object of any class derived from Tensor. |
|
biases |
One-dimensional tensor of size nk that stores biases for shifting the scaled data β(k). This input can be an object of any class derived from Tensor. |
|
populationMean |
One-dimensional tensor of size nk that stores population mean μ computed in the previous stage. This input can be an object of any class derived from Tensor. |
|
populationVariance |
One-dimensional tensor of size nk that stores population variance s2 computed in the previous stage. This input can be an object of any class derived from Tensor. |
For common parameters of neural network layers, see Common Parameters.
In addition to the common parameters, the forward batch normalization layer has the following parameters:
Parameter |
Default Value |
Description |
|
---|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
|
method |
defaultDense |
Performance-oriented computation method, the only method supported by the layer. |
|
alpha |
0.01 |
Smoothing factor of the exponential moving average used to compute the population mean and variance. |
|
epsilon |
0.00001 |
Constant added to the mini-batch variance for numerical stability. |
|
dimension |
1 |
Index of dimension k for which normalization is performed. |
The forward batch normalization layer calculates the result described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.
Result ID |
Result |
|
---|---|---|
value |
Tensor of size n1 x n2 x ... x np that stores the result of the forward batch normalization layer. This input can be an object of any class derived from Tensor. |
|
resultForBackward |
Collection of data needed for the backward batch normalization layer. | |
Element ID |
Element |
|
auxData |
Tensor of size n1 x n2 x ... x np that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from Tensor. |
|
auxWeights |
One-dimensional tensor of size nk that stores weights for scaling ω (k). This input can be an object of any class derived from Tensor. |
|
auxMean |
One-dimensional tensor of size nk that stores mini-batch mean μk. This input can be an object of any class derived from Tensor. |
|
auxStandardDeviation |
One-dimensional tensor of size nk that stores mini-batch standard deviation σ(k). This input can be an object of any class derived from Tensor. |
|
auxPopulationMean |
One-dimensional tensor of size nk that stores the resulting population mean μ. This input can be an object of any class derived from Tensor. |
|
auxPopulationVariance |
One-dimensional tensor of size nk that stores the resulting population variance s2. This input can be an object of any class derived from Tensor. |
C++: batch_norm_layer_dense_batch.cpp
Java*: BatchNormLayerDenseBatch.java
Python*: batch_norm_layer_dense_batch.py