首页 Paddle框架 帖子详情
官方教程中使用paddle.reader出现警告
收藏
快速回复
Paddle框架 问答深度学习模型训练 2178 4
官方教程中使用paddle.reader出现警告
收藏
快速回复
Paddle框架 问答深度学习模型训练 2178 4

在官方教程中,深度学习实践应用:计算机视觉-目标检测-批量数据读取与加速中

通过使用飞桨提供的API paddle.reader.xmap_readers可以开启多线程读取数据,具体实现代码如下

import functools
import paddle

# 使用paddle.reader.xmap_readers实现多线程读取数据
def multithread_loader(datadir, batch_size= 10, mode='train'):
    cname2cid = get_insect_names()
    records = get_annotations(cname2cid, datadir)
    def reader():
        if mode == 'train':
            np.random.shuffle(records)
        img_size = get_img_size(mode)
        batch_data = []
        for record in records:
            batch_data.append((record, img_size))
            if len(batch_data) == batch_size:
                yield batch_data
                batch_data = []
                img_size = get_img_size(mode)
        if len(batch_data) > 0:
            yield batch_data

    def get_data(samples):
        batch_data = []
        for sample in samples:
            record = sample[0]
            img_size = sample[1]
            img, gt_bbox, gt_labels, im_shape = get_img_data(record, size=img_size)
            batch_data.append((img, gt_bbox, gt_labels, im_shape))
        return make_array(batch_data)

    mapper = functools.partial(get_data, )

    return paddle.reader.xmap_readers(mapper, reader, 8, 10)

出现warning 请问如何解决

terminate called without an active exception
W0714 09:26:50.891782  3895 init.cc:219] Warning: PaddlePaddle catches a failure signal, it may not work properly
W0714 09:26:50.891793  3895 init.cc:221] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle
W0714 09:26:50.891795  3895 init.cc:224] The detail failure signal is:

W0714 09:26:50.891798  3895 init.cc:227] *** Aborted at 1594690010 (unix time) try "date -d @1594690010" if you are using GNU date ***
W0714 09:26:50.892606  3895 init.cc:227] PC: @                0x0 (unknown)
W0714 09:26:50.892683  3895 init.cc:227] *** SIGABRT (@0x3e800000f0f) received by PID 3855 (TID 0x7fd62dffb700) from PID 3855; stack trace: ***
W0714 09:26:50.893406  3895 init.cc:227]     @     0x7fd6d57928a0 (unknown)
W0714 09:26:50.894140  3895 init.cc:227]     @     0x7fd6d53cdf47 gsignal
W0714 09:26:50.894817  3895 init.cc:227]     @     0x7fd6d53cf8b1 abort
W0714 09:26:50.896234  3895 init.cc:227]     @     0x7fd6bd22a84a __gnu_cxx::__verbose_terminate_handler()
W0714 09:26:50.896798  3895 init.cc:227]     @     0x7fd6bd228f47 __cxxabiv1::__terminate()
W0714 09:26:50.897435  3895 init.cc:227]     @     0x7fd6bd228f7d std::terminate()
W0714 09:26:50.897979  3895 init.cc:227]     @     0x7fd6bd228c5a __gxx_personality_v0
W0714 09:26:50.898819  3895 init.cc:227]     @     0x7fd6d127db97 _Unwind_ForcedUnwind_Phase2
W0714 09:26:50.899495  3895 init.cc:227]     @     0x7fd6d127de7d _Unwind_ForcedUnwind
W0714 09:26:50.900220  3895 init.cc:227]     @     0x7fd6d5790f20 __GI___pthread_unwind
W0714 09:26:50.901005  3895 init.cc:227]     @     0x7fd6d5788ae5 __pthread_exit
W0714 09:26:50.901693  3895 init.cc:227]     @     0x55fe4afe02d9 PyThread_exit_thread
W0714 09:26:50.901737  3895 init.cc:227]     @     0x55fe4ae64c52 PyEval_RestoreThread.cold.799
W0714 09:26:50.902117  3895 init.cc:227]     @     0x7fd6b0ba44f7 (unknown)
W0714 09:26:50.902297  3895 init.cc:227]     @     0x55fe4af61c94 _PyMethodDef_RawFastCallKeywords
W0714 09:26:50.902458  3895 init.cc:227]     @     0x55fe4af61db1 _PyCFunction_FastCallKeywords
W0714 09:26:50.902617  3895 init.cc:227]     @     0x55fe4afcd5be _PyEval_EvalFrameDefault
W0714 09:26:50.902751  3895 init.cc:227]     @     0x55fe4af6120b _PyFunction_FastCallKeywords
W0714 09:26:50.902909  3895 init.cc:227]     @     0x55fe4afc8be6 _PyEval_EvalFrameDefault
W0714 09:26:50.903054  3895 init.cc:227]     @     0x55fe4af112b9 _PyEval_EvalCodeWithName
W0714 09:26:50.903188  3895 init.cc:227]     @     0x55fe4af61497 _PyFunction_FastCallKeywords
W0714 09:26:50.903345  3895 init.cc:227]     @     0x55fe4afc9cba _PyEval_EvalFrameDefault
W0714 09:26:50.903488  3895 init.cc:227]     @     0x55fe4af112b9 _PyEval_EvalCodeWithName
W0714 09:26:50.903687  3895 init.cc:227]     @     0x55fe4af123e5 _PyFunction_FastCallDict
W0714 09:26:50.903781  3895 init.cc:227]     @     0x55fe4b0212eb partial_call
W0714 09:26:50.903928  3895 init.cc:227]     @     0x55fe4af6900b _PyObject_FastCallKeywords
W0714 09:26:50.904084  3895 init.cc:227]     @     0x55fe4afcd186 _PyEval_EvalFrameDefault
W0714 09:26:50.904228  3895 init.cc:227]     @     0x55fe4af1159a _PyEval_EvalCodeWithName
W0714 09:26:50.904373  3895 init.cc:227]     @     0x55fe4af123e5 _PyFunction_FastCallDict
W0714 09:26:50.904531  3895 init.cc:227]     @     0x55fe4afca51a _PyEval_EvalFrameDefault
W0714 09:26:50.904664  3895 init.cc:227]     @     0x55fe4af6120b _PyFunction_FastCallKeywords
W0714 09:26:50.904821  3895 init.cc:227]     @     0x55fe4afc8e70 _PyEval_EvalFrameDefault

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
0
收藏
回复
全部评论(4)
时间顺序
HolliZhao
#2 回复于2020-07

看上去好像PaddlePaddle被kill掉了。读取的图片很大吗? 降低batch_size试试.

0
回复
夜雨飘零1
#3 回复于2020-07

这个是windows下的吗?有没有在Ubuntu下试过?

0
回复
管理WOW的ID
#4 回复于2020-07
这个是windows下的吗?有没有在Ubuntu下试过?

在ubuntu下运行的

 

0
回复
管理WOW的ID
#5 回复于2020-07
看上去好像PaddlePaddle被kill掉了。读取的图片很大吗? 降低batch_size试试.

非常奇怪的事情,后来运行就没有了。我的batchsize=2

0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户