fluid.recordio_writer¶

Convert a Python Reader to a recordio file.

Examples

>>> import paddle.fluid as fluid
>>> import paddle.dataset.mnist as mnist
>>> import paddle
>>>
>>> tmp_program = fluid.Program()
>>> with fluid.program_guard(tmp_program):
>>>     img = fluid.layers.data(name='img', shape=[784])
>>>     label = fluid.layers.data(name='label', shape=[1], dtype='int64')
>>> feeder = fluid.DataFeeder(feed_list=[img, label], place=fluid.CPUPlace())
>>> # mnist.recordio will be generated in current directory
>>> fluid.recordio_writer.convert_reader_to_recordio_file(
>>>                     filename="mnist.recordio",
>>>                     reader_creator=paddle.batch(mnist.train(), batch_size=32),
>>>                     feeder=feeder)

Parameters

• filename (str) – The recordio filename.

• reader_creator (callable) – The Python Reader Creator. See api_guide_python_reader.

• feeder (DataFeeder) – The DataFeeder instance. Used to convert reader_creator to :code: lod_tensor

• compressor – Must in fluid.core.RecordIOWriter.Compressor.Snappy or fluid.core.RecordIOWriter.Compressor.NoCompress. Use Snappy by default.

• max_num_records (int) – Maximum number of records in one chuck. Each record is each return value from reader function

• feed_order (list) – The order of variable names that the reader returns

Returns

the number of record that saved.

Return type

int

convert a python reader to many recordio files.

This API is basically same as convert_reader_to_recordio_file, instead of it will create many recordio files. Each file contains at most batch_per_file records.

Please reference convert_reader_to_recordio_file for more details.