\u200E
  • 开始使用
  • 特性
  • 文档
    • API
    • 使用指南
  • 工具平台
    • 工具
      • AutoDL
  • develop(Fluid)
  • 1.5(Fluid)
  • 1.4(Fluid)
  • 1.3(Fluid)
  • 1.2(Fluid)
  • 1.1(Fluid)
  • 1.0(Fluid)
  • 0.15.0(Fluid)
  • 0.14.0(Fluid)
  • 0.13.0(Fluid)
  • 0.12.0(v2)
  • 0.11.0(v2)
  • 0.10.0(v2)
  • 中文(简)
  • English(En)
  • Beginner’s Guide
    • Installation Manuals
      • Install on Ubuntu
      • Install on CentOS
      • Install on MacOS
      • Installation on Windows
      • Compile From Source Code
        • Compile on Ubuntu from Source Code
        • Compile on CentOS from Source Code
        • Compile on MacOS from Source Code
        • Compile on Windows from Source Code
      • Appendix
    • Basic Deep Learning Models
      • Linear Regression
      • Recognize Digits
      • Image Classification
      • Word Vector
      • Recommender System
      • Sentiment Analysis
      • Label Semantic Roles
      • Machine Translation
    • Guide to Fluid Programming
  • User Guides
    • Basic Concepts
      • LoD-Tensor User Guide
    • Prepare Data
      • Take Numpy Array as Training Data
      • Python Reader
      • Use PyReader to read training and test data
    • Set up Simple Model
    • Train Neural Networks
      • Single-node training
        • Evaluate model while training
      • Multi-node Training
        • Quick Start with Distributed Training
        • Manual for Distributed Training with Fluid
        • Distributed Training on Baidu Cloud
      • Save, Load Models or Variables & Incremental Learning
    • Model Evaluation and Debugging
      • Model Evaluation
      • VisualDL Tools
        • Introduction to VisualDL Toolset
        • VisualDL user guide
  • Advanced User Guides
    • Design Principles of Fluid
    • Deploy Inference Model
      • Server-side Deployment
        • Install and Compile C++ Inference Library
        • Introduction to C++ Inference API
        • Use Paddle-TensorRT Library for inference
        • Performance Profiling for TensorRT Library
        • Model Inference on Windows
      • Mobile Deployment
    • Write New Operators
      • How to write a new operator
      • Notes on operator development
    • Performance Profiling and Optimization
      • Tune CPU performance
      • Heap Memory Profiling and Optimization
      • How to use timeline tool to do profile
    • How to contribute codes to Paddle
      • Guide of local development
      • Guide of submitting PR to Github
    • How to contribute documentation
    • Best Practice
      • Best practices of distributed training on CPU
  • API Reference
    • API Quick Search
      • Basic Concept
      • Neural Network Layer
        • Convolution
        • Pooling
        • Image Detection
        • Sequence
        • Mathematical operation
        • Activation Function
        • Loss function
        • Data input and output
        • Control Flow
        • Sparse update
        • Feed training/inference data with DataFeeder
        • Learning rate scheduler
        • Tensor
      • Complex Networks
      • Optimizer
      • Back Propagation
      • Metrics
      • Save and Load a Model
      • Inference Engine
      • Video Memory Optimization
      • Executor
      • Parallel Executor
      • CompiledProgram
      • Model Parameters
      • Distributed Training
        • Synchronous Distributed Training
        • Asynchronous Distributed Training
        • Training of Models with Large Scale Sparse Features
        • Preparing Data Reader for Distributed Training
    • fluid
    • fluid.average
    • fluid.backward
    • fluid.clip
    • Data Reader
    • dataset
    • fluid.data_feeder
    • fluid.dataset
    • fluid.dygraph
    • fluid.executor
    • fluid.initializer
    • fluid.io
    • fluid.layers
      • control_flow
      • detection
      • io
      • learning_rate_scheduler
      • metric_op
      • nn
      • ops
      • tensor
    • fluid.metrics
    • fluid.nets
    • fluid.optimizer
    • fluid.profiler
    • fluid.recordio_writer
    • fluid.regularizer
    • fluid.transpiler
    • fluid.unique_name
  • FLAGS
    • cudnn
    • data processing
    • debug
    • device management
    • distributed
    • executor
    • memory management
    • others
  • Multi-node Training
  • »
  • User Guides »
  • Train Neural Networks »
  • Multi-node Training
  • View page source

Multi-node Training¶

  • Quick Start with Distributed Training
  • Manual for Distributed Training with Fluid
  • Distributed Training on Baidu Cloud