官方教程中使用paddle.reader出现警告
收藏
在官方教程中,深度学习实践应用:计算机视觉-目标检测-批量数据读取与加速中
通过使用飞桨提供的API paddle.reader.xmap_readers可以开启多线程读取数据,具体实现代码如下
import functools import paddle # 使用paddle.reader.xmap_readers实现多线程读取数据 def multithread_loader(datadir, batch_size= 10, mode='train'): cname2cid = get_insect_names() records = get_annotations(cname2cid, datadir) def reader(): if mode == 'train': np.random.shuffle(records) img_size = get_img_size(mode) batch_data = [] for record in records: batch_data.append((record, img_size)) if len(batch_data) == batch_size: yield batch_data batch_data = [] img_size = get_img_size(mode) if len(batch_data) > 0: yield batch_data def get_data(samples): batch_data = [] for sample in samples: record = sample[0] img_size = sample[1] img, gt_bbox, gt_labels, im_shape = get_img_data(record, size=img_size) batch_data.append((img, gt_bbox, gt_labels, im_shape)) return make_array(batch_data) mapper = functools.partial(get_data, ) return paddle.reader.xmap_readers(mapper, reader, 8, 10)
出现warning 请问如何解决
terminate called without an active exception W0714 09:26:50.891782 3895 init.cc:219] Warning: PaddlePaddle catches a failure signal, it may not work properly W0714 09:26:50.891793 3895 init.cc:221] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle W0714 09:26:50.891795 3895 init.cc:224] The detail failure signal is: W0714 09:26:50.891798 3895 init.cc:227] *** Aborted at 1594690010 (unix time) try "date -d @1594690010" if you are using GNU date *** W0714 09:26:50.892606 3895 init.cc:227] PC: @ 0x0 (unknown) W0714 09:26:50.892683 3895 init.cc:227] *** SIGABRT (@0x3e800000f0f) received by PID 3855 (TID 0x7fd62dffb700) from PID 3855; stack trace: *** W0714 09:26:50.893406 3895 init.cc:227] @ 0x7fd6d57928a0 (unknown) W0714 09:26:50.894140 3895 init.cc:227] @ 0x7fd6d53cdf47 gsignal W0714 09:26:50.894817 3895 init.cc:227] @ 0x7fd6d53cf8b1 abort W0714 09:26:50.896234 3895 init.cc:227] @ 0x7fd6bd22a84a __gnu_cxx::__verbose_terminate_handler() W0714 09:26:50.896798 3895 init.cc:227] @ 0x7fd6bd228f47 __cxxabiv1::__terminate() W0714 09:26:50.897435 3895 init.cc:227] @ 0x7fd6bd228f7d std::terminate() W0714 09:26:50.897979 3895 init.cc:227] @ 0x7fd6bd228c5a __gxx_personality_v0 W0714 09:26:50.898819 3895 init.cc:227] @ 0x7fd6d127db97 _Unwind_ForcedUnwind_Phase2 W0714 09:26:50.899495 3895 init.cc:227] @ 0x7fd6d127de7d _Unwind_ForcedUnwind W0714 09:26:50.900220 3895 init.cc:227] @ 0x7fd6d5790f20 __GI___pthread_unwind W0714 09:26:50.901005 3895 init.cc:227] @ 0x7fd6d5788ae5 __pthread_exit W0714 09:26:50.901693 3895 init.cc:227] @ 0x55fe4afe02d9 PyThread_exit_thread W0714 09:26:50.901737 3895 init.cc:227] @ 0x55fe4ae64c52 PyEval_RestoreThread.cold.799 W0714 09:26:50.902117 3895 init.cc:227] @ 0x7fd6b0ba44f7 (unknown) W0714 09:26:50.902297 3895 init.cc:227] @ 0x55fe4af61c94 _PyMethodDef_RawFastCallKeywords W0714 09:26:50.902458 3895 init.cc:227] @ 0x55fe4af61db1 _PyCFunction_FastCallKeywords W0714 09:26:50.902617 3895 init.cc:227] @ 0x55fe4afcd5be _PyEval_EvalFrameDefault W0714 09:26:50.902751 3895 init.cc:227] @ 0x55fe4af6120b _PyFunction_FastCallKeywords W0714 09:26:50.902909 3895 init.cc:227] @ 0x55fe4afc8be6 _PyEval_EvalFrameDefault W0714 09:26:50.903054 3895 init.cc:227] @ 0x55fe4af112b9 _PyEval_EvalCodeWithName W0714 09:26:50.903188 3895 init.cc:227] @ 0x55fe4af61497 _PyFunction_FastCallKeywords W0714 09:26:50.903345 3895 init.cc:227] @ 0x55fe4afc9cba _PyEval_EvalFrameDefault W0714 09:26:50.903488 3895 init.cc:227] @ 0x55fe4af112b9 _PyEval_EvalCodeWithName W0714 09:26:50.903687 3895 init.cc:227] @ 0x55fe4af123e5 _PyFunction_FastCallDict W0714 09:26:50.903781 3895 init.cc:227] @ 0x55fe4b0212eb partial_call W0714 09:26:50.903928 3895 init.cc:227] @ 0x55fe4af6900b _PyObject_FastCallKeywords W0714 09:26:50.904084 3895 init.cc:227] @ 0x55fe4afcd186 _PyEval_EvalFrameDefault W0714 09:26:50.904228 3895 init.cc:227] @ 0x55fe4af1159a _PyEval_EvalCodeWithName W0714 09:26:50.904373 3895 init.cc:227] @ 0x55fe4af123e5 _PyFunction_FastCallDict W0714 09:26:50.904531 3895 init.cc:227] @ 0x55fe4afca51a _PyEval_EvalFrameDefault W0714 09:26:50.904664 3895 init.cc:227] @ 0x55fe4af6120b _PyFunction_FastCallKeywords W0714 09:26:50.904821 3895 init.cc:227] @ 0x55fe4afc8e70 _PyEval_EvalFrameDefault Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
0
收藏
请登录后评论
看上去好像PaddlePaddle被kill掉了。读取的图片很大吗? 降低batch_size试试.
这个是windows下的吗?有没有在Ubuntu下试过?
在ubuntu下运行的
非常奇怪的事情,后来运行就没有了。我的batchsize=2