control_flow¶

The dynamic RNN can process a batch of sequence data. The length of each sample sequence can be different. This API automatically process them in batch.

The input lod must be set. Please reference to lod_tensor.

The dynamic RNN will unfold sequence into timesteps. Users need to define how to process each time step during the with block.

The memory is used staging data cross time step. The initial value of memory can be zero or another variable.

The dynamic RNN can mark multiple variables as its output. Use drnn() to get the output sequence.

Examples

import paddle.fluid as fluid

sentence = fluid.layers.data(name='sentence', shape=[1], dtype='int64', lod_level=1)
embedding = fluid.layers.embedding(input=sentence, size=[65536, 32], is_sparse=True)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    word = drnn.step_input(embedding)
    prev = drnn.memory(shape=[200])
    hidden = fluid.layers.fc(input=[word, prev], size=200, act='relu')
    drnn.update_memory(prev, hidden)  # set prev to hidden
    drnn.output(hidden)

# Get the last time step of rnn. It is the encoding result.
rnn_output = drnn()
last = fluid.layers.sequence_last_step(rnn_output)

step_input(x, level=0)

Mark a sequence as a dynamic RNN input.

Parameters

• x (Variable) – The input sequence which should have lod information.

• level (int) – The level of lod used to split steps. Default: 0.

Returns

The current timestep in the input sequence.

static_input(x)

Mark a variable as a RNN input. The input will not be scattered into time steps. It is optional.

Parameters

x (Variable) – The input variable.

Returns

The input variable that can access in RNN.

Examples

import paddle.fluid as fluid

sentence = fluid.layers.data(name='sentence', dtype='float32', shape=[32], lod_level=1)
encoder_proj = fluid.layers.data(name='encoder_proj', dtype='float32', shape=[32], lod_level=1)
decoder_boot = fluid.layers.data(name='boot', dtype='float32', shape=[10], lod_level=1)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    current_word = drnn.step_input(sentence)
    encoder_word = drnn.static_input(encoder_proj)
    hidden_mem = drnn.memory(init=decoder_boot, need_reorder=True)
    fc_1 = fluid.layers.fc(input=encoder_word, size=30, bias_attr=False)
    fc_2 = fluid.layers.fc(input=current_word, size=30, bias_attr=False)
    decoder_inputs = fc_1 + fc_2
    h, _, _ = fluid.layers.gru_unit(input=decoder_inputs, hidden=hidden_mem, size=30)
    drnn.update_memory(hidden_mem, h)
    out = fluid.layers.fc(input=h, size=10, bias_attr=True, act='softmax')
    drnn.output(out)

rnn_output = drnn()

block()

The block for user to define operators in RNN.

memory(init=None, shape=None, value=0.0, need_reorder=False, dtype='float32')

Create a memory variable for dynamic rnn.

If the init is not None, memory will be initialized by this variable. The need_reorder is used to reorder the memory as the input variable. It should be set to true when the initialized memory depends on the input sample.

Examples

import paddle.fluid as fluid

sentence = fluid.layers.data(name='sentence', shape=[32], dtype='float32', lod_level=1)
boot_memory = fluid.layers.data(name='boot', shape=[10], dtype='float32', lod_level=1)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    word = drnn.step_input(sentence)
    memory = drnn.memory(init=boot_memory, need_reorder=True)
    hidden = fluid.layers.fc(input=[word, memory], size=10, act='tanh')
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    drnn.output(hidden)

rnn_output = drnn()

Otherwise, if shape, value, dtype are set, the memory will be initialized by this value.

Examples

import paddle.fluid as fluid

sentence = fluid.layers.data(name='sentence', dtype='float32', shape=[32], lod_level=1)

drnn = fluid.layers.DynamicRNN()
with drnn.block():
    word = drnn.step_input(sentence)
    memory = drnn.memory(shape=[10], dtype='float32', value=0)
    hidden = fluid.layers.fc(input=[word, memory], size=10, act='tanh')
    drnn.update_memory(ex_mem=memory, new_mem=hidden)
    drnn.output(hidden)

rnn_output = drnn()

Parameters

• init (Variable|None) – The initialized variable.

• shape (list|tuple) – The memory shape. The shape does not contain batch_size.

• value (float) – the initalized value.

• need_reorder (bool) – True if the initialized memory depends on the input sample.

• dtype (str|numpy.dtype) – The data type of the initialized memory.

Returns

The memory variable.

update_memory(ex_mem, new_mem)

Update the memory from ex_mem to new_mem. NOTE that the shape and data type of ex_mem and new_mem must be same.

Parameters

• ex_mem (Variable) – the memory variable.

• new_mem (Variable) – the plain variable generated in RNN block.

Returns

None

output(*outputs)

Mark the RNN output variables.

Parameters

outputs – The output variables.

Returns

None

if-else control flow.

Parameters

• cond (Variable) – condition used to compare.

• name (str, default None) – The name of this layer.

Examples

import paddle.fluid as fluid

image = fluid.layers.data(name="X", shape=[2, 5, 5], dtype='float32')
label = fluid.layers.data(name='label', shape=[1], dtype='int64')
limit = fluid.layers.fill_constant_batch_size_like(
     input=label, dtype='int64', shape=[1], value=5.0)
cond = fluid.layers.less_than(x=label, y=limit)
ie = fluid.layers.IfElse(cond)
with ie.true_block():
    true_image = ie.input(image)
    hidden = fluid.layers.fc(input=true_image, size=100, act='tanh')
    prob = fluid.layers.fc(input=hidden, size=10, act='softmax')
    ie.output(prob)

with ie.false_block():
    false_image = ie.input(image)
    hidden = fluid.layers.fc(
        input=false_image, size=200, act='tanh')
    prob = fluid.layers.fc(input=hidden, size=10, act='softmax')
    ie.output(prob)
prob = ie()

StaticRNN class.

The StaticRNN can process a batch of sequence data. The length of each sample sequence must be equal. The StaticRNN will have its own parameters like inputs, outputs, memories. Note that the first dimension of inputs represents sequence length, and all the sequence length of inputs must be the same. And the meaning of each axis of input and output are the same.

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers

vocab_size, hidden_size=10000, 200
x = layers.data(name="x", shape=[-1, 1, 1], dtype='int64')
x_emb = layers.embedding(
    input=x,
    size=[vocab_size, hidden_size],
    dtype='float32',
    is_sparse=False)
x_emb = layers.transpose(x_emb, perm=[1, 0, 2])

rnn = fluid.layers.StaticRNN()
with rnn.step():
    word = rnn.step_input(x_emb)
    prev = rnn.memory(shape=[-1, hidden_size], batch_ref = word)
    hidden = fluid.layers.fc(input=[word, prev], size=hidden_size, act='relu')
    rnn.update_memory(prev, hidden)  # set prev to hidden
    rnn.step_output(hidden)

result = rnn()

The StaticRNN will unfold sequence into time steps. Users need to define how to process each time step during the with step.

The memory is used as a staging data cross time step. The initial value of memory can be a variable that is filled with a constant value or a specified variable.

The StaticRNN can mark multiple variables as its output. Use rnn() to get the output sequence.

step()

The block for user to define operators in RNN.

memory(init=None, shape=None, batch_ref=None, init_value=0.0, init_batch_dim_idx=0, ref_batch_dim_idx=1)

Create a memory variable for static rnn.

If the init is not None, memory will be initialized by this Variable. If the init is None, shape and batch_ref must be set, and this function will initialize a init Variable.

Parameters

• init (Variable|None) – The initialized variable. If it is not set, shape and batch_ref must be provided. Default: None.

• shape (list|tuple) – The shape of the boot memory. NOTE the shape does not contain batch_size. Default: None.

• batch_ref (Variable|None) – The batch size reference Variable. Default: None.

• init_value (float) – the init value of boot memory. Default: 0.0.

• init_batch_dim_idx (int) – the batch_size axis of the init Variable. Default: 0.

• ref_batch_dim_idx (int) – the batch_size axis of the batch_ref Variable. Default: 1.

Returns

The memory variable.

Examples

import paddle.fluid as fluid
import paddle.fluid.layers as layers

vocab_size, hidden_size=10000, 200
x = layers.data(name="x", shape=[-1, 1, 1], dtype='int64')
x_emb = layers.embedding(
    input=x,
    size=[vocab_size, hidden_size],
    dtype='float32',
    is_sparse=False)
x_emb = layers.transpose(x_emb, perm=[1, 0, 2])

rnn = fluid.layers.StaticRNN()
with rnn.step():
    word = rnn.step_input(x_emb)
    prev = rnn.memory(shape=[-1, hidden_size], batch_ref = word)
    hidden = fluid.layers.fc(input=[word, prev], size=hidden_size, act='relu')
    rnn.update_memory(prev, hidden)

step_input(x)

Mark a sequence as a StaticRNN input.

Parameters

x (Variable) – The input sequence, the shape of x should be [seq_len, …].

Returns

The current time step in the input sequence.

step_output(o)

Mark a sequence as a StaticRNN output.

Parameters

o (Variable) – The output sequence.

Returns

None.

output(*outputs)

Mark the StaticRNN output variables.

Parameters

outputs – The output Variables.

Returns

None

update_memory(mem, var)

Update the memory from ex_mem to new_mem. NOTE that the shape and data type of ex_mem and new_mem must be same.

Parameters

• mem (Variable) – the memory variable.

• var (Variable) – the plain variable generated in RNN block.

Returns

None

Switch class works just like a if-elif-else. Can be used in learning rate scheduler to modify learning rate

The Semantics:

1. A switch control-flow checks cases one-by-one.

2. The condition of each case is a boolean value, which is a scalar Variable.

3. It runs the first matched case, or the default case if there is one.

4. Once it matches a case, it runs the corresponding branch and only that branch.

Examples

import paddle.fluid as fluid

lr = fluid.layers.create_global_var(
    shape=[1],
    value=0.0,
    dtype='float32',
    persistable=True,
    name="learning_rate")
zero_var = fluid.layers.fill_constant(
     shape=[1], dtype='float32', value=0.0)
one_var = fluid.layers.fill_constant(
    shape=[1], dtype='float32', value=1.0)
two_var = fluid.layers.fill_constant(
    shape=[1], dtype='float32', value=2.0)

global_step = fluid.layers.autoincreased_step_counter(
       counter_name='@LR_DECAY_COUNTER@', begin=0, step=1)

with fluid.layers.control_flow.Switch() as switch:
    with switch.case(global_step == zero_var):
        fluid.layers.assign(input=one_var, output=lr)
    with switch.default():
        fluid.layers.assign(input=two_var, output=lr)

while loop control flow.

Parameters

• cond (Variable) – condition used to compare.

• is_test (bool) – A flag indicating whether execution is in test phase.

• name (str) – The name of this layer.

Examples

import paddle.fluid as fluid

i = fluid.layers.fill_constant(shape=[1], dtype='int64', value=0)
d0 = fluid.layers.data("d0", shape=[10], dtype='float32')
data_array = fluid.layers.array_write(x=d0, i=i)
array_len = fluid.layers.fill_constant(shape=[1],dtype='int64', value=3)

cond = fluid.layers.less_than(x=i, y=array_len)
while_op = fluid.layers.While(cond=cond)
with while_op.block():
    d = fluid.layers.array_read(array=data_array, i=i)
    i = fluid.layers.increment(x=i, value=1, in_place=True)
    fluid.layers.less_than(x=i, y=array_len, cond=cond)