官方教程中使用paddle.reader出现警告
收藏
在官方教程中,深度学习实践应用:计算机视觉-目标检测-批量数据读取与加速中
通过使用飞桨提供的API paddle.reader.xmap_readers可以开启多线程读取数据,具体实现代码如下
import functools
import paddle
# 使用paddle.reader.xmap_readers实现多线程读取数据
def multithread_loader(datadir, batch_size= 10, mode='train'):
cname2cid = get_insect_names()
records = get_annotations(cname2cid, datadir)
def reader():
if mode == 'train':
np.random.shuffle(records)
img_size = get_img_size(mode)
batch_data = []
for record in records:
batch_data.append((record, img_size))
if len(batch_data) == batch_size:
yield batch_data
batch_data = []
img_size = get_img_size(mode)
if len(batch_data) > 0:
yield batch_data
def get_data(samples):
batch_data = []
for sample in samples:
record = sample[0]
img_size = sample[1]
img, gt_bbox, gt_labels, im_shape = get_img_data(record, size=img_size)
batch_data.append((img, gt_bbox, gt_labels, im_shape))
return make_array(batch_data)
mapper = functools.partial(get_data, )
return paddle.reader.xmap_readers(mapper, reader, 8, 10)
出现warning 请问如何解决
terminate called without an active exception
W0714 09:26:50.891782 3895 init.cc:219] Warning: PaddlePaddle catches a failure signal, it may not work properly
W0714 09:26:50.891793 3895 init.cc:221] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle
W0714 09:26:50.891795 3895 init.cc:224] The detail failure signal is:
W0714 09:26:50.891798 3895 init.cc:227] *** Aborted at 1594690010 (unix time) try "date -d @1594690010" if you are using GNU date ***
W0714 09:26:50.892606 3895 init.cc:227] PC: @ 0x0 (unknown)
W0714 09:26:50.892683 3895 init.cc:227] *** SIGABRT (@0x3e800000f0f) received by PID 3855 (TID 0x7fd62dffb700) from PID 3855; stack trace: ***
W0714 09:26:50.893406 3895 init.cc:227] @ 0x7fd6d57928a0 (unknown)
W0714 09:26:50.894140 3895 init.cc:227] @ 0x7fd6d53cdf47 gsignal
W0714 09:26:50.894817 3895 init.cc:227] @ 0x7fd6d53cf8b1 abort
W0714 09:26:50.896234 3895 init.cc:227] @ 0x7fd6bd22a84a __gnu_cxx::__verbose_terminate_handler()
W0714 09:26:50.896798 3895 init.cc:227] @ 0x7fd6bd228f47 __cxxabiv1::__terminate()
W0714 09:26:50.897435 3895 init.cc:227] @ 0x7fd6bd228f7d std::terminate()
W0714 09:26:50.897979 3895 init.cc:227] @ 0x7fd6bd228c5a __gxx_personality_v0
W0714 09:26:50.898819 3895 init.cc:227] @ 0x7fd6d127db97 _Unwind_ForcedUnwind_Phase2
W0714 09:26:50.899495 3895 init.cc:227] @ 0x7fd6d127de7d _Unwind_ForcedUnwind
W0714 09:26:50.900220 3895 init.cc:227] @ 0x7fd6d5790f20 __GI___pthread_unwind
W0714 09:26:50.901005 3895 init.cc:227] @ 0x7fd6d5788ae5 __pthread_exit
W0714 09:26:50.901693 3895 init.cc:227] @ 0x55fe4afe02d9 PyThread_exit_thread
W0714 09:26:50.901737 3895 init.cc:227] @ 0x55fe4ae64c52 PyEval_RestoreThread.cold.799
W0714 09:26:50.902117 3895 init.cc:227] @ 0x7fd6b0ba44f7 (unknown)
W0714 09:26:50.902297 3895 init.cc:227] @ 0x55fe4af61c94 _PyMethodDef_RawFastCallKeywords
W0714 09:26:50.902458 3895 init.cc:227] @ 0x55fe4af61db1 _PyCFunction_FastCallKeywords
W0714 09:26:50.902617 3895 init.cc:227] @ 0x55fe4afcd5be _PyEval_EvalFrameDefault
W0714 09:26:50.902751 3895 init.cc:227] @ 0x55fe4af6120b _PyFunction_FastCallKeywords
W0714 09:26:50.902909 3895 init.cc:227] @ 0x55fe4afc8be6 _PyEval_EvalFrameDefault
W0714 09:26:50.903054 3895 init.cc:227] @ 0x55fe4af112b9 _PyEval_EvalCodeWithName
W0714 09:26:50.903188 3895 init.cc:227] @ 0x55fe4af61497 _PyFunction_FastCallKeywords
W0714 09:26:50.903345 3895 init.cc:227] @ 0x55fe4afc9cba _PyEval_EvalFrameDefault
W0714 09:26:50.903488 3895 init.cc:227] @ 0x55fe4af112b9 _PyEval_EvalCodeWithName
W0714 09:26:50.903687 3895 init.cc:227] @ 0x55fe4af123e5 _PyFunction_FastCallDict
W0714 09:26:50.903781 3895 init.cc:227] @ 0x55fe4b0212eb partial_call
W0714 09:26:50.903928 3895 init.cc:227] @ 0x55fe4af6900b _PyObject_FastCallKeywords
W0714 09:26:50.904084 3895 init.cc:227] @ 0x55fe4afcd186 _PyEval_EvalFrameDefault
W0714 09:26:50.904228 3895 init.cc:227] @ 0x55fe4af1159a _PyEval_EvalCodeWithName
W0714 09:26:50.904373 3895 init.cc:227] @ 0x55fe4af123e5 _PyFunction_FastCallDict
W0714 09:26:50.904531 3895 init.cc:227] @ 0x55fe4afca51a _PyEval_EvalFrameDefault
W0714 09:26:50.904664 3895 init.cc:227] @ 0x55fe4af6120b _PyFunction_FastCallKeywords
W0714 09:26:50.904821 3895 init.cc:227] @ 0x55fe4afc8e70 _PyEval_EvalFrameDefault
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
0
收藏
请登录后评论
看上去好像PaddlePaddle被kill掉了。读取的图片很大吗? 降低batch_size试试.
这个是windows下的吗?有没有在Ubuntu下试过?
在ubuntu下运行的
非常奇怪的事情,后来运行就没有了。我的batchsize=2