鹏城实验室鲲鹏920上PaddleHub安装
在鹏城实验室鲲鹏920上安装PaddleHub,执行pip install paddlehub结果报错:
error: libhdf5.so: cannot open shared object file: No such file or directory ---------------------------------------- ERROR: Failed building wheel for h5py Failed to build onnx h5py ERROR: Could not build wheels for onnx, h5py, which is required to install pyproject.toml-based projects
手工安装python3-h5py
pip install python3-h5py
安装之后不行,再安装这个:
pip install pkg-config libhdf5-dev
哈哈,安装好啦!结果没高兴两分钟,又看到了失败的显示:
Successfully built h5py
Failed to build onnx
完毕之后测试,import onnx是ok的,import h5py是报错的,
PaddleHub还是没有安装成功。
也就是在鹏城云服务器上,暂时h5py无法安装。既然pip安装失败,就改变策略:
使用下载代码安装PaddleHub
使用下载代码,本地安装的方式,先下载代码并安装需要的库:
672 git clone https://github.com/PaddlePaddle/Paddlehub
677 cd Paddlehub/
pip -r requirements.txt -i https://mirror.baidu.com/pypi/simple
结果报错:
Building wheel for shellcheck-py (setup.py) ... error ERROR: Command errored out with exit status 1: command: /home/skywalk/venv/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-drp20xiu/shellcheck-py_843a995c85db4fcfb5c79e0d59eb4ee5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-drp20xiu/shellcheck-py_843a995c85db4fcfb5c79e0d59eb4ee5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-d710rul3 cwd: /tmp/pip-install-drp20xiu/shellcheck-py_843a995c85db4fcfb5c79e0d59eb4ee5/ Complete output (4 lines): running bdist_wheel running build running fetch_binaries error: Remote end closed connection without response ---------------------------------------- ERROR: Failed building wheel for shellcheck-py Running setup.py clean for shellcheck-py Building wheel for future (setup.py) ... done Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=56ef22333c65b1a5821688c5276aabcd54130c58e3c943f53aeb420d5216f2bc Stored in directory: /home/skywalk/.cache/pip/wheels/bb/b4/25/d4ed0b0183f33df9bdfb686bbb497a8e991dc982f1dc769246 Successfully built easydict h5py jieba seqeval future Failed to build onnx shellcheck-py ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects
看看到底是哪个库没有装上啊?
一个一个用pip安装的方式测试,发现:
pip install paddle2onnx
Collecting paddle2onnx
Using cached paddle2onnx-0.8.2-py3-none-any.whl (80 kB)
Requirement already satisfied: six in /home/skywalk/venv/lib/python3.8/site-packages (from paddle2onnx) (1.16.0)
Collecting onnx<=1.9.0
Using cached onnx-1.9.0.tar.gz (9.8 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
^C Installing backend dependencies ... canceled
也就是paddle2onnx那里需要onnx<=1.9.0
但是onnx低版本安装不上,导致paddle2onnx安装不上,进而导致PaddleHub安装不成功。
跑到github下载paddle2onnx的源代码,准备编译安装:
修改setup.py文件,修改onnx的版本需求
install_requires=['six', 'protobuf', 'onnx<=1.9.0'],修改成:
install_requires=['six', 'protobuf', 'onnx']
700 git clone https://hub.fastgit.org/PaddlePaddle/Paddle2ONNX
701 cd Paddle2ONNX/
705 python setup.py install
安装完成paddle2onnx之后,再次pip安装paddlehub
哇,成功安装啦!
Successfully installed Babel-2.9.1 Flask-Babel-2.0.0 Pillow-8.2.0 backports.entry-points-selectable-1.1.0 bce-python-sdk-0.8.62 cfgv-3.3.1 chardet-3.0.4 cycler-0.11.0 dill-0.3.4 distlib-0.3.3 flake8-4.0.1 future-0.18.2 gunicorn-20.1.0 h5py-3.5.0 identify-2.3.4 idna-2.10 jieba-0.42.1 joblib-1.1.0 kiwisolver-1.3.2 matplotlib-3.4.3 mccabe-0.6.1 multiprocess-0.70.12.2 nodeenv-1.6.0 numpy-1.19.5 paddlefsl-1.0.0 paddlehub-2.1.1 paddlenlp-2.1.1 pandas-1.3.4 platformdirs-2.4.0 pre-commit-2.15.0 pycodestyle-2.8.0 pycryptodome-3.11.0 pyflakes-2.4.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 pyzmq-22.3.0 rarfile-4.0 requests-2.24.0 scikit-learn-1.0.1 scipy-1.7.2 seqeval-1.2.2 shellcheck-py-0.7.2.1 threadpoolctl-3.0.0 toml-0.10.2 tqdm-4.27.0 urllib3-1.25.11 virtualenv-20.10.0 visualdl-2.2.1
从这次安装paddlehub,终于深刻体会到了paddlehub和paddleserving的区别,一个是开发用的,装了一大堆东西,一个是部署用的,理论上应该不会装这么多东西。
而且paddlehub在arm上面装有坑,我是用修改paddle2onnx的onnx的版本需求(去掉低版本需求)解决的。
安装好之后,PaddleHub开服务:
nohup hub serving start -m deoldify &
真香!
总结一下:
PaddleHub是为了快速验证项目而存在的,重点在于能快速实现想法,所以安装包比较臃肿,在鲲鹏920上安装不太顺利。同时PaddleHub的负载能力也较Paddle Serving低一些。如果安装不上,可以看下是哪个相关库安装的卡住了,手工安装上即可。实在不行,手工修改依赖库信息即可。
总体而言,想快速验证想法,还是用PaddleHub最快最方便!
PaddleHub黑白照片上色测试链接:http://decs.pcl.ac.cn:3625/aipic.html
具体部署方法见这个项目:https://aistudio.baidu.com/aistudio/projectdetail/2559081
部署服务器由鹏城实验室:https://dw.pcl.ac.cn/#/home/index 提供服务。
再次感谢飞桨,感谢鹏城实验室。
确实是,右手版本会导致这样那样问题。
最好提一个issuse