fluid.initializer¶

This initializer can be used in transposed convolution operator to act as upsampling. Users can upsample a feature map with shape of (B, C, H, W) by any integer factor. The usage is:

Examples

import paddle.fluid as fluid
factor = 2
C = 2
w_attr = fluid.initializer.ParamAttr(
    learning_rate=0.,
    regularizer=fluid.regularizer.L2Decay(0.),
    initializer=fluid.initializer.Bilinear())
x = fluid.layers.data(name="data", shape=[3, 32, 32],
                      dtype="float32")
conv_up = fluid.layers.conv2d_transpose(
    input=x,
    num_filters=C,
    output_size=None,
    filter_size=2 * factor - factor % 2,
    padding=int(math.ceil((factor - 1) / 2.)),
    stride=factor,
    groups=C,
    param_attr=w_attr,
    bias_attr=False)

Where, num_filters=C and groups=C means this is channel-wise transposed convolution. The filter shape will be (C, 1, K, K) where K is filer_size, This initializer will set a (K, K) interpolation kernel for every channel of the filter identically. The resulting shape of the output feature map will be (B, C, factor * H, factor * W). Note that the learning rate and the weight decay are set to 0 in order to keep coefficient values of bilinear interpolation unchanged during training.

Implements the constant initializer

Parameters

value (float) – constant value to initialize the variable

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name="data", shape=[32, 32], dtype="float32")
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.Constant(value=2.0))

Implements the MSRA initializer a.k.a. Kaiming Initializer

This class implements the weight initialization from the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where

In case of Normal distribution, the mean is 0 and the standard deviation is

Parameters

• uniform (bool) – whether to use uniform or normal distribution

• fan_in (float) – fan_in for MSRAInitializer. If None, it is inferred from the variable.

• seed (int) – random seed

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name="data", shape=[32, 32], dtype="float32")
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.MSRA(uniform=False))

Implements the Random Normal(Gaussian) distribution initializer

Parameters

• loc (float) – mean of the normal distribution

• scale (float) – standard deviation of the normal distribution

• seed (int) – random seed

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name="data", shape=[32, 32], dtype="float32")
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.Normal(loc=0.0, scale=2.0))

Init an parameter with an numpy array

Parameters

value (numpy) – numpy array to initialize the variable

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name="x", shape=[5], dtype='float32')
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.NumpyArrayInitializer(numpy.array([1,2])))

Implements the Random TruncatedNormal(Gaussian) distribution initializer

Parameters

• loc (float) – mean of the normal distribution

• scale (float) – standard deviation of the normal distribution

• seed (int) – random seed

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name='x', shape=[1], dtype='float32')
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.TruncatedNormal(loc=0.0, scale=2.0))

Implements the random uniform distribution initializer

Parameters

• low (float) – lower boundary of the uniform distribution

• high (float) – upper boundary of the uniform distribution

• seed (int) – random seed

Examples

import paddle.fluid as fluid
x = fluid.layers.data(name='x', shape=[1], dtype='float32')
fc = fluid.layers.fc(input=x, size=10,
    param_attr=fluid.initializer.Uniform(low=-0.5, high=0.5))

This class implements the Xavier weight initializer from the paper Understanding the difficulty of training deep feedforward neural networks by Xavier Glorot and Yoshua Bengio.

This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is [-x, x], where

In case of Normal distribution, the mean is 0 and the standard deviation is

Parameters

• uniform (bool) – whether to use uniform or normal distribution

• fan_in (float) – fan_in for Xavier initialization. If None, it is inferred from the variable.

• fan_out (float) – fan_out for Xavier initialization. If None, it is inferred from the variable.

• seed (int) – random seed

Examples

import paddle.fluid as fluid
queries = fluid.layers.data(name='x', shape=[1], dtype='float32')
fc = fluid.layers.fc(
    input=queries, size=10,
    param_attr=fluid.initializer.Xavier(uniform=False))