Developer Guide for Intel® Data Analytics Acceleration Library 2018
The forward element-wise sum layer computes the value tensor Y as follows:
where
are the real valued user-defined coefficients, and the tensor
X
(i)
is the output of
the previous
i-th layer,
.
The backward element-wise sum layer computes the following values:
where G is the input gradient tensor computed on the preceding layer during backpropagation process and Z (i) is the value the layer propagates to the next i-th layer.
Let
be the input gradient tensor with the components
and
be the predefined scalars. The problem is to compute the tensor
with the components
to be propagated to the next
i-th layer: