我想测试paddle + tensorrt 的加速效果,用官网上的例子测试,没看到效果,用不用trt几乎没有差异,我确信trt开启的时候,屏幕上已经明确打印用到了tensorrt。我的环境是 ubuntu18, cuda10.2, tensorrt 6.0, paddlepaddle-gpu 2.1.2.post101 (post102没找到)。例子就是官网上的 resnet50例子,代码如下
import numpy as np
import paddle.inference as paddle_infer
import time
def create_predictor():
config = paddle_infer.Config("./resnet50/inference.pdmodel", "./resnet50/inference.pdiparams")
config.enable_memory_optim()
config.enable_use_gpu(1000, 0)
# 打开TensorRT。此接口的详细介绍请见下文
# 把这行配置注释掉就没有trt了
config.enable_tensorrt_engine(workspace_size = 1 << 30,
max_batch_size = 1,
min_subgraph_size = 3,
precision_mode=paddle_infer.PrecisionType.Half,
use_static = False, use_calib_mode = False)
predictor = paddle_infer.create_predictor(config)
return predictor
def run(predictor, img):
# 准备输入
input_names = predictor.get_input_names()
for i, name in enumerate(input_names):
input_tensor = predictor.get_input_handle(name)
input_tensor.reshape(img[i].shape)
input_tensor.copy_from_cpu(img[i].copy())
# 预测
st=time.time()
predictor.run()
elapse=time.time()-st
已将问题转发给相关研发同学,请耐心等待
可直接在GitHub issue上关注此问题:https://github.com/PaddlePaddle/Paddle/issues/34797#issuecomment-896683312