LoD-Tensor User Guide¶

Challenge of variable-length sequences¶

In most deep learning frameworks, a mini-batch is represented by Tensor.

For example, if there are 10 pictures in a mini-batch and the size of each picture is 32*32, the mini-batch will be a 10*32*32 Tensor.

Or in the NLP task, there are N sentences in a mini-batch and the length of each sentence is L. Every word is represented by a one-hot vector with D dimensions. Then the mini-batch can be represented by an N*L*D Tensor.

In the two examples above, the size of each sequence element remains the same. However, the data to be trained are variable-length sequences in many cases. For this scenario, method to be taken in most frameworks is to set a fixed length and sequence data shorter than the fixed length will be padded with 0 to reach the fixed length.

Owing to the LoD-Tensor in Fluid, it is not necessary to keep the lengths of sequence data in every mini-batch constant.Therefore tasks sensitive to sequence formats like NLP can also be finished without padding.

Index Data Structure (LoD) is introduced to Fluid to split Tensor into sequences.

Index Structure - LoD¶

To have a better understanding of the concept of LoD, you can refer to the examples in this section.

mini-batch consisting of sentences

Suppose a mini-batch contains three sentences, and each contains 3, 1, 2 words respectively. Then the mini-batch can be represented by a (3+1+2)*D Tensor with some index information appended:

3       1   2
| | |   |   | |

In the text above, each | represents a word vector with D dimension and a 1-level LoD is made up of digits 3,1,2 .

recursive sequence

Take a 2-level LoD-Tensor for example, a mini-batch contains articles of 3 sentences, 1 sentence and 2 sentences. The number of words in every sentence is different. Then the mini-batch is formed as follows:

3            1 2
3   2  4     1 2  3
||| || ||||  | || |||

the LoD to express the format:

[[3，1，2]/*level=0*/，[3，2，4，1，2，3]/*level=1*/]

mini-batch consisting of video data

In the task of computer vision, it usually needs to deal objects with high dimension like videos and pictures. Suppose a mini-batch contains 3 videos, which is composed of 3 frames, 1 frames, 2 frames respectively. The size of each frame is 640*480. Then the mini-batch can be described as:

3     1  2
口口口 口 口口

The size of the tensor at the bottom is (3+1+2)*640*480. Every 口 represents a 640*480 picture.

mini-batch consisting of pictures

Traditionally, for a mini-batch of N pictures with fixed size, LoD-Tensor is described as:

1 1 1 1     1
口口口口 ... 口

Under such circumstance, we will consider LoD-Tensor as a common tensor instead of ignoring information because of the indices of all elements are 1.

口口口口 ... 口

model parameter

model parameter is a common tensor which is described as a 0-level LoD-Tensor in Fluid.

LoDTensor expressed by offset¶

To have a quick access to the original sequence, you can take the offset expression method——store the first and last element of a sequence instead of its length.

In the example above, you can compute the length of fundamental elements:

3 2 4 1 2 3

It is expressed by offset as follows:

0  3  5   9   10  12   15
   =  =   =   =   =    =
   3  2+3 4+5 1+9 2+10 3+12

Therefore we infer that the first sentence starts from word 0 to word 3 and the second sentence starts from word 3 to word 5.

Similarly, for the length of the top layer of LoD

3 1 2

It can be expressed by offset:

0 3 4   6
  = =   =
  3 3+1 4+2

Therefore the LoD-Tensor is expressed by offset:

0       3    4      6
  3 5 9   10   12 15

LoD-Tensor¶

A LoD-Tensor can be regarded as a tree of which the leaf is an original sequence element and branch is the flag of fundamental element.

There are two ways to express sequence information of LoD-Tensor in Fluid: primitive length and offset. LoD-Tensor is expressed by offset in Paddle to offer a quicker access to sequence;LoD-Tensor is expressed by primitive length in python API to make user understand and compute more easily. The primary length is named as recursive_sequence_lengths .

Take a 2-level LoD-Tensor mentioned above as an example:

3           1  2
3   2  4    1  2  3
||| || |||| |  || |||

• LoD-Tensor expressed by offset: [ [0,3,4,6] , [0,3,5,9,10,12,15] ]

• LoD-Tensor expressed by primitive length: recursive_sequence_lengths=[ [3-0 , 4-3 , 6-4] , [3-0 , 5-3 , 9-5 , 10-9 , 12-10 , 15-12] ]

Take text sequence as an example,[3,1,2] indicates there are 3 articles in the mini-batch,which contains 3,1,2 sentences respectively.[3,2,4,1,2,3] indicates there are 3,2,4,1,2,3 words in sentences respectively.

recursive_seq_lens is a double Layer nested list, and in other words, the element of the list is list. The size of the outermost list represents the nested layers, namely the size of lod-level; Each inner list represents the size of each element in each lod-level.

The following three pieces of codes introduce how to create LoD-Tensor, how to transform LoD-Tensor to Tensor and how to transform Tensor to LoD-Tensor respectively:

• Create LoD-Tensor

#Create lod-tensor
import paddle.fluid as fluid
import numpy as np

a = fluid.create_lod_tensor(np.array([[1],[1],[1],
                                  [1],[1],
                                  [1],[1],[1],[1],
                                  [1],
                                  [1],[1],
                                  [1],[1],[1]]).astype('int64') ,
                          [[3,1,2] , [3,2,4,1,2,3]],
                          fluid.CPUPlace())

#Check lod-tensor nested layers
print (len(a.recursive_sequence_lengths()))
# output：2

#Check the number of the most fundamental elements
print (sum(a.recursive_sequence_lengths()[-1]))
# output:15 (3+2+4+1+2+3=15)

• Transform LoD-Tensor to Tensor

  import paddle.fluid as fluid
  import numpy as np
  
  # create LoD-Tensor
  a = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], fluid.CPUPlace())
  
  def LodTensor_to_Tensor(lod_tensor):
    # get lod information of LoD-Tensor
    lod = lod_tensor.lod()
    # transform into array
    array = np.array(lod_tensor)
    new_array = []
    # transform to Tensor according to the layer information of the original LoD-Tensor
    for i in range(len(lod[0]) - 1):
        new_array.append(array[lod[0][i]:lod[0][i + 1]])
    return new_array
  
  new_array = LodTensor_to_Tensor(a)
  
  # output the result
  print(new_array)
  

• Transform Tensor to LoD-Tensor

import paddle.fluid as fluid
import numpy as np

def to_lodtensor(data, place):
  # save the length of Tensor as LoD information
  seq_lens = [len(seq) for seq in data]
  cur_len = 0
  lod = [cur_len]
  for l in seq_lens:
      cur_len += l
      lod.append(cur_len)
  # decrease the dimention of transformed Tensor
  flattened_data = np.concatenate(data, axis=0).astype("int64")
  flattened_data = flattened_data.reshape([len(flattened_data), 1])
  # add lod information to Tensor data
  res = fluid.LoDTensor()
  res.set(flattened_data, place)
  res.set_lod([lod])
  return res

# new_array is the transformed Tensor above
lod_tensor = to_lodtensor(new_array,fluid.CPUPlace())

# output LoD information
print("The LoD of the result: {}.".format(lod_tensor.lod()))

# examine the consistency with Tensor data
print("The array : {}.".format(np.array(lod_tensor)))

Code examples¶

Input variable x is expanded according to specified layer level y-lod in the code example in this section. The example below contains some fundamental conception of LoD-Tensor. By following the code, you will

• Have a direct understanding of the implementation of fluid.layers.sequence_expand in Fluid

• Know how to create LoD-Tensor in Fluid

• Learn how to print the content of LoDTensor

Define the Process of Computing

layers.sequence_expand expands x by obtaining the lod value of y. About more explanation of fluid.layers.sequence_expand , please read sequence_expand first.

Code of sequence expanding:

x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)

Note：The dimension of input LoD-Tensor is only associated with the dimension of real data transferred in. The shape value set for x and y in the definition of network structure is just a placeholder with little influence on the result.

Create Executor

place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

Prepare Data

Here we use fluid.create_lod_tensor to create the input data of sequence_expand and expand x_d by defining LoD of y_d. The output value is only associated with LoD of y_d. And the data of y_d is not invovled in the process of computation. The dimension of y_d must keep consistent with as its LoD[-1] .

About the user guide of fluid.create_lod_tensor() , please refer to create_lod_tensor .

Code：

x_d = fluid.create_lod_tensor(np.array([[1.1],[2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [2,1,2,1]],place)

Execute Computing

For tensor whose LoD > 1 in Fluid, like data of other types, the order of transfering data is defined by feed . In addition, parameter return_numpy=False needs to be added to exe.run() to get the output of LoD-Tensor because results are Tensors with LoD information.

results = exe.run(fluid.default_main_program(),
                  feed={'x':x_d, 'y': y_d },
                  fetch_list=[out],return_numpy=False)

Check the result of LodTensor

Because of the special attributes of LoDTensor, you could not print to check the content. The usual solution to the problem is to fetch the LoDTensor as the output of network and then execute numpy.array(lod_tensor) to transfer LoDTensor into numpy array:

np.array(results[0])

Output:

array([[1.1],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4]])

Check the length of sequence

You can get the recursive sequence length of LoDTensor by checking the sequence length:

results[0].recursive_sequence_lengths()

Output

[[1L, 3L, 3L, 3L]]

Complete Code

You can check the output by executing the following complete code:

#Load
import paddle
import paddle.fluid as fluid
import numpy as np
#Define forward computation
x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
#Define place for computation
place = fluid.CPUPlace()
#Create executer
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
#Create LoDTensor
x_d = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [1,2,1,2]], place)
#Start computing
results = exe.run(fluid.default_main_program(),
                  feed={'x':x_d, 'y': y_d },
                  fetch_list=[out],return_numpy=False)
#Output result
print("The data of the result: {}.".format(np.array(results[0])))
#print the length of sequence of result
print("The recursive sequence lengths of the result: {}.".format(results[0].recursive_sequence_lengths()))
#print the LoD of result
print("The LoD of the result: {}.".format(results[0].lod()))

Summary¶

Then, we believe that you have known about the concept LoD-Tensor. And an attempt to change x_d and y_d in code above and then to check the output may help you get a better understanding of this flexible structure.

About more model applications of LoDTensor, you can refer to Word2vec , Individual Recommendation , Sentiment Analysis in the beginner’s guide.

About more difffiult and complex examples of application, please refer to associated information about models .