首页 Paddle框架 帖子详情
--eval 命令报错
收藏
快速回复
Paddle框架 问答模型训练 1002 7
--eval 命令报错
收藏
快速回复
Paddle框架 问答模型训练 1002 7

使用python3 -u tools/train.py -c configs//ppyolo/ppyolo_tiny_650e_coco.yml --eval训练报错

报错如下:

/home/robocup/.local/lib/python3.6/site-packages/matplotlib/backends/backend_gtk3.py:40: DeprecationWarning: Gdk.Cursor.new is deprecated
cursors.MOVE: Gdk.Cursor.new(Gdk.CursorType.FLEUR),
W1106 19:33:12.636938 6934 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.4, Runtime API Version: 11.0
W1106 19:33:12.638509 6934 device_context.cc:422] device: 0, cuDNN Version: 8.0.
[11/06 19:33:15] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/robocup/.cache/paddle/weights/MobileNetV3_large_x0_5_pretrained.pdparams
[11/06 19:33:21] ppdet.engine INFO: Epoch: [0] [ 0/36] learning_rate: 0.000000 loss_xy: 1.124620 loss_wh: 4.389097 loss_iou: 4.059346 loss_obj: 7283.875000 loss_cls: 16.433590 loss: 7309.881836 eta: 1 day, 15:41:43 batch_cost: 6.1070 data_cost: 5.0936 ips: 5.2399 images/s
[11/06 19:33:54] ppdet.engine INFO: Epoch: [0] [20/36] learning_rate: 0.000025 loss_xy: 1.091926 loss_wh: 3.948453 loss_iou: 3.940778 loss_obj: 615.970581 loss_cls: 15.599344 loss: 639.476196 eta: 11:51:37 batch_cost: 1.6122 data_cost: 1.3968 ips: 19.8488 images/s
[11/06 19:34:19] ppdet.utils.checkpoint INFO: Save checkpoint: output/ppyolo_tiny_650e_coco
[11/06 19:34:20] ppdet.engine INFO: Eval iter: 0
ERROR:root:DataLoader reader thread raised an exception!
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 391, in _thread_loop
batch = self._get_data()
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 505, in _get_data
batch.reraise()
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/worker.py", line 168, in reraise
raise self.exc_type(msg)
ValueError: DataLoader worker(3) caught ValueError with message:
Traceback (most recent call last):
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/worker.py", line 320, in _worker_loop
batch = fetcher.fetch(indices)
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/fetcher.py", line 117, in fetch
data = self.collate_fn(data)
File "/home/robocup/PaddleDetection/ppdet/data/reader.py", line 91, in __call__
batch_data = default_collate_fn(data)
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/collate.py", line 70, in default_collate_fn
for key in sample
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/collate.py", line 70, in
for key in sample
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/collate.py", line 58, in default_collate_fn
batch = np.stack(batch, axis=0)
File "<__array_function__ internals>", line 6, in stack
File "/home/robocup/.local/lib/python3.6/site-packages/numpy/core/shape_base.py", line 427, in stack
raise ValueError('all input arrays must have the same shape')
ValueError: all input arrays must have the same shape


Traceback (most recent call last):
File "tools/train.py", line 138, in
main()
File "tools/train.py", line 134, in main
run(FLAGS, cfg)
File "tools/train.py", line 109, in run
trainer.train(FLAGS.eval)
File "/home/robocup/PaddleDetection/ppdet/engine/trainer.py", line 407, in train
self._eval_with_loader(self._eval_loader)
File "/home/robocup/PaddleDetection/ppdet/engine/trainer.py", line 421, in _eval_with_loader
for step_id, data in enumerate(loader):
File "/home/robocup/PaddleDetection/ppdet/data/reader.py", line 209, in __next__
return next(self.loader)
File "/home/robocup/.local/lib/python3.6/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 565, in __next__
data = self._reader.read_next_var_list()
SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:166)

 

 

如果使用python3 -u tools/train.py -c configs//ppyolo/ppyolo_tiny_650e_coco.yml命令能正常训练。

0
收藏
回复
全部评论(7)
时间顺序
深渊上的坑
#2 回复于2021-11

test可以正常用么?

0
回复
深渊上的坑
#3 回复于2021-11

我感觉这是遇到了一个bug,就是configs/ppyolo/_base_/ppyolo_reader.yml里面,EvalReader没有定义inputs_def,当然这只是个猜测

0
回复
程序员小勇
#4 回复于2021-12

哈哈哈,你路径多打了个/

1
回复
DeepGeGe
#5 回复于2021-12

Blocking queue is killed because the data reader raises an exception.

all input arrays must have the same shape。

你的输入有问题,所有输入的shape必须一致。

0
回复
羊毛
#6 回复于2021-12

输入的数据有问题,建议把所有的数据画出来看下。

0
回复
1
1074251637
#7 回复于2023-09

请问您解决了吗?

 

0
回复
科特君
#8 回复于2023-12

请问这个bug解决了吗?

0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户