Install and Compile C++ Inference Library¶

Direct Download and Installation¶

Table 4 c++ inference library list¶

version description	inference library(1.5.1 version)	inference library(develop version)
cpu_avx_mkl	fluid_inference.tgz	fluid_inference.tgz
cpu_avx_openblas	fluid_inference.tgz	fluid_inference.tgz
cpu_noavx_openblas	fluid_inference.tgz	fluid_inference.tgz
cuda8.0_cudnn7_avx_openblas	fluid_inference.tgz	fluid_inference.tgz
cuda8.0_cudnn7_avx_mkl	fluid_inference.tgz	fluid_inference.tgz
cuda9.0_cudnn7_avx_mkl	fluid_inference.tgz	fluid_inference.tgz
cuda10.0_cudnn7_avx_mkl	fluid_inference.tgz	fluid_inference.tgz

Build from Source Code¶

Users can also compile C++ inference libraries from the PaddlePaddle core code by specifying the following compile options at compile time:

Option	Value
CMAKE_BUILD_TYPE	Release
FLUID_INFERENCE_INSTALL_DIR	Path of installation
WITH_PYTHON	OFF（recommended）
ON_INFER	ON（recommended）
WITH_GPU	ON/OFF
WITH_MKL	ON/OFF

It is recommended to configure options according to the recommended values to avoid the link to unnecessary library. Other options can be set if it is necessary.

The following code snippet pulls the latest code from github and specifies the compiling options (you need to replace PADDLE_ROOT with the installation path of the PaddlePaddle inference library):

PADDLE_ROOT=/path/of/capi
git clone https://github.com/PaddlePaddle/Paddle.git
cd Paddle
mkdir build
cd build
cmake -DFLUID_INFERENCE_INSTALL_DIR=$PADDLE_ROOT \
      -DCMAKE_BUILD_TYPE=Release \
      -DWITH_PYTHON=OFF \
      -DWITH_MKL=OFF \
      -DWITH_GPU=OFF  \
      -DON_INFER=ON \
      ..
 make
 make inference_lib_dist

After successful compilation, dependencies required by the C++ inference library Will be stored in the PADDLE_ROOT directory. (dependencies including: (1) compiled PaddlePaddle inference library and header files; (2) third-party link libraries and header files; (3) version information and compilation option information)

The directory structure is:

PaddleRoot/
├── CMakeCache.txt
├── paddle
│   ├── include
│   │   ├── paddle_anakin_config.h
│   │   ├── paddle_analysis_config.h
│   │   ├── paddle_api.h
│   │   ├── paddle_inference_api.h
│   │   ├── paddle_mkldnn_quantizer_config.h
│   │   └── paddle_pass_builder.h
│   └── lib
│       ├── libpaddle_fluid.a
│       └── libpaddle_fluid.so
├── third_party
│   ├── boost
│   │   └── boost
│   ├── eigen3
│   │   ├── Eigen
│   │   └── unsupported
│   └── install
│       ├── gflags
│       ├── glog
│       ├── mkldnn
│       ├── mklml
│       ├── protobuf
│       ├── snappy
│       ├── snappystream
│       ├── xxhash
│       └── zlib
└── version.txt

The version information of the inference library is recorded in version.txt, including Git Commit ID, version of OpenBlas, MKL math library, or CUDA/CUDNN. For example:

GIT COMMIT ID: cc9028b90ef50a825a722c55e5fda4b7cd26b0d6
WITH_MKL: ON
WITH_MKLDNN: ON
WITH_GPU: ON
CUDA version: 8.0
CUDNN version: v7