初体验
使用pip安装
pip的安装方式和TensorFlow、MXNet、keras安装方式一样,
pip install ppaddlepaddle gpu版本: pip install paddlepaddle-gpu
但是paddle的部分安装依赖更多,很多前置依赖库,甚至于包括nltk、Pillow、Opencv(为啥要同时使用了Opencv和Pillow)。
使用容器
拉取镜像,因为我在mac上使用,所以cpu即可,默认blas为mkl库,也可使用openblas:
docker pull docker.paddlepaddle.org/paddle or docker pull paddlepaddle/paddle openblas库: docker pull paddlepaddle/paddle:latest-openblas
拉取gpu版本的paddle:
docker pull docker.paddlepaddle.org/paddle:latest-gpu or docker pull paddlepaddle/paddle:latest-gpu
其他各种不同版本的paddlepaddle具体可以看paddle Dockerhub。
docker run -it -v $PWD:/work paddlepaddle/paddle bash
把当前目录挂在到docker的work目录,然后进入paddlepaddle/paddle镜像bash环境。 官网上有提到paddlepaddle/book,里面有一套paddlepaddle的入门的教程,拷贝到本机电脑上,包括数据、notebook、python脚本:
docker cp 905b7f19e8f8:/book script/tutorial
然后,我们进入docker:
cd /work/tutorial/01.fit_a_line python train.py
贴出代码部分:
import os import paddle.v2 as paddle import paddle.v2.dataset.uci_housing as uci_housing with_gpu = os.getenv('WITH_GPU', '0') != '0' def main(): # init paddle.init(use_gpu=with_gpu, trainer_count=1) # network config x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear()) y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1)) cost = paddle.layer.square_error_cost(input=y_predict, label=y) # Save the inference topology to protobuf. inference_topology = paddle.topology.Topology(layers=y_predict) with open("inference_topology.pkl", 'wb') as f: inference_topology.serialize_for_inference(f) # create parameters parameters = paddle.parameters.create(cost) # create optimizer optimizer = paddle.optimizer.Momentum(momentum=0) trainer = paddle.trainer.SGD( cost=cost, parameters=parameters, update_equation=optimizer) feeding = {'x': 0, 'y': 1} # event_handler to print training and testing info def event_handler(event): if isinstance(event, paddle.event.EndIteration): if event.batch_id % 100 == 0: print "Pass %d, Batch %d, Cost %f" % ( event.pass_id, event.batch_id, event.cost) if isinstance(event, paddle.event.EndPass): if event.pass_id % 10 == 0: with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_parameter_to_tar(f) result = trainer.test( reader=paddle.batch(uci_housing.test(), batch_size=2), feeding=feeding) print "Test %d, Cost %f" % (event.pass_id, result.cost) # training trainer.train( reader=paddle.batch( paddle.reader.shuffle(uci_housing.train(), buf_size=500), batch_size=2), feeding=feeding, event_handler=event_handler, num_passes=30) # inference test_data_creator = paddle.dataset.uci_housing.test() test_data = [] test_label = [] for item in test_data_creator(): test_data.append((item[0], )) test_label.append(item[1]) if len(test_data) == 5: break # load parameters from tar file. # users can remove the comments and change the model name # with open('params_pass_20.tar', 'r') as f: # parameters = paddle.parameters.Parameters.from_tar(f) probs = paddle.infer( output_layer=y_predict, parameters=parameters, input=test_data) for i in xrange(len(probs)): print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0]) if __name__ == '__main__': main()
代码解析: 这部分代码是使用uci-housing的数据集,是一个线性回归问题,使用历史数据来预测对应房价; paddle.init(): with_gpu = os.getenv('WITH_GPU', '0') != '0',查找系统环境WITH_GPU,无gpu则为0, 则表达式为false,trainer_count定义单台机器训练任务的线程数;
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear()) y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1)) cost = paddle.layer.square_error_cost(input=y_predict, label=y)
定义网络结构,一个单层网络:y=wx,全连接层输出节点为1,cost定义为square error cost。
inference_topology = paddle.topology.Topology(layers=y_predict) with open("inference_topology.pkl", 'wb') as f: inference_topology.serialize_for_inference(f)
把定义网络结果拓扑保存为pkl文件:
parameters = paddle.parameters.create(cost) optimizer = paddle.optimizer.Momentum(momentum=0) trainer = paddle.trainer.SGD( cost=cost, parameters=parameters, update_equation=optimizer)
通过网络结构和定义的object function: cost定义参数集,定义优化器sgd的Momentum,然后配置trainer。
feeding = {'x': 0, 'y': 1} trainer.train( reader=paddle.batch( paddle.reader.shuffle(uci_housing.train(), buf_size=500), batch_size=2), feeding=feeding, event_handler=event_handler, num_passes=30)
feeding用来配置网络的输入信息,注意前面网络设计的时候x,y的name属性,这里表示输入的array的index和相应的网络节点对应, event_handle时间处理器,当满足时间条件激活时,运行该函数。
def event_handler(event): if isinstance(event, paddle.event.EndIteration): if event.batch_id % 100 == 0: print "Pass %d, Batch %d, Cost %f" % ( event.pass_id, event.batch_id, event.cost) if isinstance(event, paddle.event.EndPass): if event.pass_id % 10 == 0: with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_parameter_to_tar(f) result = trainer.test( reader=paddle.batch(uci_housing.test(), batch_size=2), feeding=feeding) print "Test %d, Cost %f" % (event.pass_id, result.cost)
查看下paddle的event:
上面逻辑是,每100个batch打印出来batch的pass_id, batch_id,和对应cost值; 当完成一个epoch时,使用trainer的test接口,对验证集测试计算对应pass_id和cost值;
est_data_creator = paddle.dataset.uci_housing.test() test_data = [] test_label = [] for item in test_data_creator(): test_data.append((item[0], )) test_label.append(item[1]) if len(test_data) == 5: break # load parameters from tar file. # users can remove the comments and change the model name # with open('params_pass_20.tar', 'r') as f: # parameters = paddle.parameters.Parameters.from_tar(f) probs = paddle.infer( output_layer=y_predict, parameters=parameters, input=test_data) for i in xrange(len(probs)): print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0])
这部分主要是讲test的迭代器数据拿出来,然后使用paddle的inference框架做预测,这里需要指定output_layer,和对应的训练的到的parameters, parameters可以通过保存的参数文件来载入,但是output_layer这里的指定是否可以不需要再次定义网络结构而是通过前面保存的网络拓扑图呢,来得到呢,不然网络拓扑图保存的目的是干啥?tutorial里面的infer.py也是需要重新定义网络,虽然我不是处女座但对代码还是有些洁癖,感觉这里重复拷贝数据不太好,我之后学习过程来看看这部分是不是可以直接加载拓扑网络,而无需在代码层重复建造网络来完成inference,应该是可以的,不然总感觉在部署的时候需要改代码,不能一套解决问题,有点不爽。
编译一个开发者容器
作为一个给TensorFlow和MXNet都写过一些垃圾代码的小伙伴,了解一个开源项目,最后的方式是参与,然后contribute,paddle官网提供了一个专门用来给paddle develop的docker镜像: 先拉下来源码:
git clone https://github.com/paddlepaddle/paddle
paddle目录下有一个Dockerfile 我们直接通过这个Dockerfile来从源码来build整个镜像,可以查看下Dockerfile:
# A image for building paddle binaries # Use cuda devel base image for both cpu and gpu environment FROM nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04 MAINTAINER PaddlePaddle Authors ARG UBUNTU_MIRROR RUN /bin/bash -c 'if [[ -n ${UBUNTU_MIRROR} ]]; then sed -i 's#http://archive.ubuntu.com/ubuntu#${UBUNTU_MIRROR}#g' /etc/apt/sources.list; fi' # ENV variables ARG WITH_GPU ARG WITH_AVX ARG WITH_DOC ENV WOBOQ OFF ENV WITH_GPU=${WITH_GPU:-ON} ENV WITH_AVX=${WITH_AVX:-ON} ENV WITH_DOC=${WITH_DOC:-OFF} ENV HOME /root # Add bash enhancements COPY ./paddle/scripts/docker/root/ /root/ RUN apt-get update && \ apt-get install -y \ git python-pip python-dev openssh-server bison libnccl-dev \ wget unzip unrar tar xz-utils bzip2 gzip coreutils ntp \ curl sed grep graphviz libjpeg-dev zlib1g-dev \ python-matplotlib gcc-4.8 g++-4.8 \ automake locales clang-format swig doxygen cmake \ liblapack-dev liblapacke-dev libboost-dev \ clang-3.8 llvm-3.8 libclang-3.8-dev \ net-tools libtool && \ apt-get clean -y # Install Go and glide RUN wget -qO- https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz | \ tar -xz -C /usr/local && \ mkdir /root/gopath && \ mkdir /root/gopath/bin && \ mkdir /root/gopath/src ENV GOROOT=/usr/local/go GOPATH=/root/gopath # should not be in the same line with GOROOT definition, otherwise docker build could not find GOROOT. ENV PATH=${PATH}:${GOROOT}/bin:${GOPATH}/bin # install glide RUN curl -s -q https://glide.sh/get | sh # git credential to skip password typing RUN git config --global credential.helper store # Fix locales to en_US.UTF-8 RUN localedef -i en_US -f UTF-8 en_US.UTF-8 # FIXME: due to temporary ipykernel dependency issue, specify ipykernel jupyter # version util jupyter fixes this issue. RUN pip install --upgrade pip && \ pip install -U wheel && \ pip install -U docopt PyYAML sphinx && \ pip install -U sphinx-rtd-theme==0.1.9 recommonmark RUN pip install pre-commit 'ipython==5.3.0' && \ pip install 'ipykernel==4.6.0' 'jupyter==1.0.0' && \ pip install opencv-python COPY ./python/requirements.txt /root/ RUN pip install -r /root/requirements.txt # To fix https://github.com/PaddlePaddle/Paddle/issues/1954, we use # the solution in https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl-py2 RUN apt-get install -y libssl-dev libffi-dev RUN pip install certifi urllib3[secure] # Install woboq_codebrowser to /woboq RUN git clone https://github.com/woboq/woboq_codebrowser /woboq && \ (cd /woboq \ cmake -DLLVM_CONFIG_EXECUTABLE=/usr/bin/llvm-config-3.8 \ -DCMAKE_BUILD_TYPE=Release . \ make) # Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service RUN mkdir /var/run/sshd RUN echo 'root:root' | chpasswd RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config EXPOSE 22 # development image default do build work CMD ["bash", "/paddle/paddle/scripts/docker/build.sh"] docker build -t paddle:burness . 编基本的依赖环境,比如下载依赖,等等; 然后我们启动镜像,挂载paddle到docker到/paddle,看上面dockerfile最后一行是调用build.sh的命令,我们启动docker时,调用build.sh,build.sh脚本内容如下: #!/bin/bash function cmake_gen() { mkdir -p /paddle/build cd /paddle/build # build script will not fail if *.deb does not exist rm *.deb 2>/dev/null || true # delete previous built whl packages rm -rf /paddle/paddle/dist 2>/dev/null || true # Support build for all python versions, currently # including cp27-cp27m and cp27-cp27mu. PYTHON_FLAGS="" if [ "$1" != "" ]; then echo "using python abi: $1" if [ "$1" == "cp27-cp27m" ]; then export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.11-ucs2/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.11-ucs4/lib:} export PATH=/opt/python/cp27-cp27m/bin/:${PATH} PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27m/bin/python -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27m/include/python2.7 -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.11-ucs2/lib/libpython2.7.so" elif [ "$1" == "cp27-cp27mu" ]; then export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.11-ucs4/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.11-ucs2/lib:} export PATH=/opt/python/cp27-cp27mu/bin/:${PATH} PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27mu/bin/python -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27mu/include/python2.7 -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.11-ucs4/lib/libpython2.7.so" fi fi cat < doc/en/html/operators.json popd fi if [[ ${WOBOQ:-OFF} == 'ON' ]]; then cat < /paddle/build/Dockerfile < ENV HOME /root EOF if [[ ${WITH_GPU} == "ON" ]]; then NCCL_DEPS="apt-get install -y libnccl-dev &&" else NCCL_DEPS="" fi cat >> /paddle/build/Dockerfile <
使用命令出入编译不带GPU来编译paddlepaddle:
docker run -e WITH_GPU=OFF -v $PWD:/paddle paddle:burness
开始编译paddlepaddle,成功后,运行单元测试:
docker run -v $PWD:/paddle paddle:burness bash -c "cd /paddle/build; ctest"
整个开发的流程环境在docker中,完全不影响系统,以后做开源相关工作时,可考虑使用docker来完成。
总结
其实,在paddlepaddle最开始出来的时候,就尝试去玩下paddlepaddle,但是那个当时文档之烂,还有各种问题、坑,就放弃了,还有后来有次gitchat里面露出paddlepaddle和TensorFlow、MXNet的五维度对比,毫无对应实验数据说明,当时对paddlepaddle的印象是很差的,这次体验下来,整体tutorial的代码风格从以前烂的不知所谓,到现在的有理由条,对比原始的tensorflow的使用方式,更趋近于keras的api设计机制,比如trainer、reader、tevent_handle,将实用的功能进行了封装,整体的api设计还是很不错的,不谈其他性能、效率,对于刚开始接触的初学者整体体验还不错,后续会学习下paddlepaddle/book下的代码,然后阅读学习paddle的代码。对于一个开始了解分布式机器学习框架的算法工程师,噩梦正在发现。另外,有读者熟悉eigen库的,可以私信联系我,谢谢。
强