Developer Guide for Intel® Data Analytics Acceleration Library 2018 Update 2
A Xavier initializer is an initializer algorithm to initialize a p-dimensional tensor W ∈ R n 1 x ... x n p that represents weights and biases of the appropriate layer. The algorithm initializes this tensor with random numbers uniformly distributed on the interval [-α,α). The value of α is defined using the sizes of the r-dimensional input tensor X ∈ R n x m 2 x... x m r and q-dimensional value tensor Y ∈ R n x k 2 x... x k q for the layer:
It is assumed without loss of generality that tensors X and Y have batch dimension of size n
For more details, see [Glorot2010].
In addition to common parameters of the initializer interface, a Xavier initializer has the following parameters:
Parameter |
Default Value |
Description |
|
---|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
|
method |
defaultDense |
Performance-oriented computation method, the only method supported by the algorithm. |
|
DEPRECATED: seed |
777 |
NoteThis parameter is deprecated and will be removed in a future release. |