首页 Paddle框架 帖子详情
训练好模型预测时失败,哪位帮忙看下啥回事?
收藏
快速回复
Paddle框架 问答模型训练 2948 8
训练好模型预测时失败,哪位帮忙看下啥回事?
收藏
快速回复
Paddle框架 问答模型训练 2948 8
#8 预测

import numpy as np
from PIL import Image
import matplotlib.pyplot as plt 
import matplotlib.image as mpimg

with open("dataset/test_list.txt","r") as f:
    filepath = f.readlines()

data = []    
for fp in filepath:
    data.append("dataset/" + fp.split(" ")[0])

#data = [filepath[0].split(" ")[0],filepath[1].split(" ")[0],filepath[2].split(" ")[0],filepath[3].split(" ")[0],filepath[4].split(" ")[0]]
print(data)

label_map = dataset.label_dict()
index = 0
run_states = task.predict(data=data[:4])
results = [run_state.run_results for run_state in run_states]

for batch_result in results:
    print(batch_result)
    batch_result = np.argmax(batch_result, axis=2)[0]
    print(batch_result)
    for result in batch_result:
        index += 1
        result = label_map[result]
        display(filepath[index - 1])
        print("input %i is %s, and the predict result is %s" %
              (index, data[index - 1], result))

已通过paddlehub上预训练模型resnet_v2_50_imagenet, FineTune完成后进行预测时出现如下错误:

[2020-05-04 19:13:47,687] [ INFO] - The best model has been loaded
[2020-05-04 19:13:47,689] [ INFO] - PaddleHub predict start
2020-05-04 19:13:47,730-WARNING: Your decorated reader has raised an exception!
Exception in thread Thread-15:
Traceback (most recent call last):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/io.py", line 491, in __provider_thread__
six.reraise(*sys.exc_info())
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py", line 693, in reraise
raise value
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/io.py", line 484, in __provider_thread__
feed_queue.push(array)
paddle.fluid.core_avx.EnforceNotMet:

--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString(std::string const&, char const*, int)
1 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int)
2 paddle::operators::reader::BlockingQueue > >::EnforceNotKilled()
3 paddle::operators::reader::BlockingQueue > >::Send(std::vector > const&)

----------------------
Error Message Summary:
----------------------
Error: Blocking queue is killed because the data reader raises an exception
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] at (/paddle/paddle/fluid/operators/reader/blocking_queue.h:141)

 

['dataset/images/img_95.jpg', 'dataset/images/img_45.jpg', 'dataset/images/img_71.jpg', 'dataset/images/img_52.jpg', 'dataset/images/img_55.jpg', 'dataset/images/img_84.jpg', 'dataset/images/img_79.jpg', 'dataset/images/img_40.jpg']

---------------------------------------------------------------------------EnforceNotMet Traceback (most recent call last) in
18 label_map = dataset.label_dict()
19 index = 0
---> 20 run_states = task.predict(data=data[:4])
21 results = [run_state.run_results for run_state in run_states]
22
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py in predict(self, data, load_best_model, return_result)
809 self._predict_data = data
810 self._predict_start_event()
--> 811 run_states = self._run()
812 self._predict_end_event(run_states)
813 self._predict_data = None
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py in _run(self, do_eval)
826 with fluid.program_guard(self.main_program, self.startup_program):
827 if self.config.use_pyreader:
--> 828 return self._run_with_py_reader(do_eval=do_eval)
829 return self._run_with_data_feeder(do_eval=do_eval)
830
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py in _run_with_py_reader(self, do_eval)
904 fetch_result = self.exe.run(
905 self.main_program_to_be_run,
--> 906 fetch_list=self.fetch_list)
907 else:
908 fetch_result = self.exe.run(
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py in run(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)
781 warnings.warn(
782 "The following exception is not an EOF exception.")
--> 783 six.reraise(*sys.exc_info())
784
785 def _run_impl(self, program, feed, fetch_list, feed_var_name,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py in reraise(tp, value, tb)
691 if value.__traceback__ is not tb:
692 raise value.with_traceback(tb)
--> 693 raise value
694 finally:
695 value = None
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py in run(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)
776 scope=scope,
777 return_numpy=return_numpy,
--> 778 use_program_cache=use_program_cache)
779 except Exception as e:
780 if not isinstance(e, core.EOFException):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py in _run_impl(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)
841 fetch_list=fetch_list,
842 fetch_var_name=fetch_var_name,
--> 843 return_numpy=return_numpy)
844
845 def _run_program(self, program, feed, fetch_list, feed_var_name,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py in _run_parallel(self, program, scope, feed, fetch_list, fetch_var_name, return_numpy)
675
676 fetch_var_names = list(map(_to_name_str, fetch_list))
--> 677 tensors = exe.run(fetch_var_names)._move_to_list()
678 return as_numpy(tensors) if return_numpy else tensors
679
EnforceNotMet:

--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString(std::string const&, char const*, int)
1 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int)
2 paddle::operators::reader::BlockingQueue > >::Receive(std::vector >*)
3 paddle::operators::reader::PyReader::ReadNext(std::vector >*)
4 paddle::framework::ReaderHolder::ReadNext(std::vector >*)
5 paddle::operators::ReadOp::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
6 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
7 paddle::framework::details::ComputationOpHandle::RunImpl()
8 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync(paddle::framework::details::OpHandleBase*)
9 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp(paddle::framework::details::OpHandleBase*, std::shared_ptr > const&, unsigned long*)
10 std::_Function_handler (), std::__future_base::_Task_setter, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&)
11 std::__future_base::_State_base::_M_do_set(std::function ()>&, bool&)
12 ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const

------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2525, in append_op
attrs=kwargs.get("attrs", None))
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/io.py", line 889, in read_file
type='read', inputs={'Reader': [reader]}, outputs={'Out': out})
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 352, in _build_env
py_vars = fluid.layers.read_file(self.env.py_reader)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 474, in main_program
self._build_env()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 564, in fetch_var_list
vars = self.main_program.global_block().vars
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 757, in save_inference_model
target_vars=self.fetch_var_list,
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 665, in _default_eval_end_event
self.save_inference_model(dirname=model_saved_dir)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 580, in hook_function
func(*args)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 800, in eval
self._eval_end_event(run_states)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 688, in _default_eval_interval_event
self.eval(phase="dev")
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 580, in hook_function
func(*args)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 930, in _run_with_py_reader
self._eval_interval_event()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 828, in _run
return self._run_with_py_reader(do_eval=do_eval)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 775, in finetune
run_states = self._run(do_eval=do_eval)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlehub/finetune/task/base_task.py", line 763, in finetune_and_eval
return self.finetune(do_eval=True)
File "", line 3, in
run_states = task.finetune_and_eval()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3265, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3183, in run_ast_nodes
if (yield from self.run_code(code, result)):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3018, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 67, in _pseudo_sync_runner
coro.send(None)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2843, in _run_cell
return runner(coro)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2817, in run_cell
raw_cell, store_history, silent, shell_futures)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 536, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 294, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/gen.py", line 326, in wrapper
yielded = next(result)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 534, in execute_request
user_expressions, allow_stdin,
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/gen.py", line 326, in wrapper
yielded = next(result)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 267, in dispatch_shell
yield gen.maybe_future(handler(stream, idents, msg))
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/gen.py", line 326, in wrapper
yielded = next(result)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 357, in process_one
yield gen.maybe_future(dispatch(*args))
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/gen.py", line 1147, in run
yielded = self.gen.send(value)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/gen.py", line 1233, in inner
self.run()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/ioloop.py", line 758, in _run_callback
ret = callback()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/asyncio/base_events.py", line 1771, in _run_once
handle._run()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/asyncio/base_events.py", line 534, in run_forever
self._run_once()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 505, in start
self.io_loop.start()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/traitlets/config/application.py", line 664, in launch_instance
app.start()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)

----------------------
Error Message Summary:
----------------------
Error: Blocking queue is killed because the data reader raises an exception
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] at (/paddle/paddle/fluid/operators/reader/blocking_queue.h:141)
[operator < read > error]

0
收藏
回复
全部评论(8)
时间顺序
thinc
#2 回复于2020-05

是不是读到脏数据了?又或者是数据格式、尺寸这些信息不对?

Error: Blocking queue is killed because the data reader raises an exception

0
回复
水水水的老师
#3 回复于2020-05

data reader错误被kill了

0
回复
AIStudio810258
#4 回复于2020-05

['dataset/images/img_95.jpg', 'dataset/images/img_45.jpg', 'dataset/images/img_71.jpg', 'dataset/images/img_52.jpg', 'dataset/images/img_55.jpg', 'dataset/images/img_84.jpg', 'dataset/images/img_79.jpg', 'dataset/images/img_40.jpg']

 

这是你打印的图片数据列表吧,是不是test_list.txt里这附近的数据出错了。

0
回复
AIStudio810258
#5 回复于2020-05

如果不是一开始预测就出错,应该是遇到脏数据了。删掉试试。

0
回复
AIStudio810258
#6 回复于2020-05

【下面是我写的一些小贴,有空可以fork玩玩(项目列表)】

训练数据太少?过拟合?一文带你领略“数据增长魔法”(上) (https://aistudio.baidu.com/aistudio/projectdetail/408059)

训练数据太少?过拟合?一文带你领略“数据增长魔法”(下) (https://aistudio.baidu.com/aistudio/projectdetail/422256)

一文帮你理解用Paddle训练模型的原理1-手写回归模型 (https://aistudio.baidu.com/aistudio/projectdetail/367090)

一文帮你理解用Paddle训练模型的原理2-权值可视化(VisualDL) (https://aistudio.baidu.com/aistudio/projectdetail/386954)

来AI Studio互粉吧~等你哦~ https://aistudio.baidu.com/aistudio/personalcenter/thirdview/76563

0
回复
austinleaven
#7 回复于2020-05

un_states = task.predict(data=data[:4])

0
回复
佩奇
#8 回复于2021-03

怎么找到脏数据

0
回复
AIStudio810259
#9 回复于2021-03

应该是遇到脏数据了

0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户