首页 Paddle框架 帖子详情
在跑deepspeech时训练失败
收藏
快速回复
Paddle框架 问答深度学习模型训练炼丹技巧 1306 2
在跑deepspeech时训练失败
收藏
快速回复
Paddle框架 问答深度学习模型训练炼丹技巧 1306 2

搜到有解答说是环境问题,但是不太详细,希望能有大神解决一下

----------Begin test...
/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/executor.py:782: UserWarning: The following exception is not an EOF exception.
"The following exception is not an EOF exception.")
Traceback (most recent call last):
File "train.py", line 142, in
main()
File "train.py", line 138, in main
train()
File "train.py", line 133, in train
test_off=args.test_off)
File "/home/doubleyang/DeepSpeech/model_utils/model.py", line 367, in train
fetch_list=[ctc_loss])
File "/home/doubleyang/DeepSpeech/model_utils/model.py", line 217, in test
return_numpy=False)
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/executor.py", line 778, in run
use_program_cache=use_program_cache)
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/executor.py", line 905, in _run_program
fetch_var_name)
paddle.fluid.core_avx.EnforceNotMet:

--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString(std::string const&, char const*, int)
1 paddle::memory::detail::AlignedMalloc(unsigned long)
2 paddle::memory::detail::CPUAllocator::Alloc(unsigned long*, unsigned long)
3 paddle::memory::detail::BuddyAllocator::SystemAlloc(unsigned long)
4 paddle::memory::detail::BuddyAllocator::Alloc(unsigned long)
5 void* paddle::memory::legacy::Alloc(paddle::platform::CPUPlace const&, unsigned long)
6 paddle::memory::allocation::NaiveBestFitAllocator::AllocateImpl(unsigned long)
7 paddle::memory::allocation::AllocatorFacade::Alloc(paddle::platform::Place const&, unsigned long)
8 paddle::memory::Alloc(paddle::platform::Place const&, unsigned long)
9 paddle::memory::Alloc(paddle::platform::DeviceContext const&, unsigned long)
10 paddle::framework::Tensor paddle::framework::ExecutionContext::AllocateTmpTensor(paddle::framework::DDim const&, paddle::platform::CPUDeviceContext const&) const
11 paddle::operators::GemmConvKernel::Compute(paddle::framework::ExecutionContext const&) const
12 std::_Function_handler, paddle::operators::GemmConvKernel >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
13 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
14 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
15 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
16 paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool)
17 paddle::framework::Executor::Run(paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool, std::vector > const&, bool, bool)

------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/framework.py", line 2525, in append_op
attrs=kwargs.get("attrs", None))
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/home/doubleyang/.local/lib/python2.7/site-packages/paddle/fluid/layers/nn.py", line 1405, in conv2d
"data_format": data_format,
File "/home/doubleyang/DeepSpeech/model_utils/network.py", line 47, in conv_bn_layer
bias_attr=False)
File "/home/doubleyang/DeepSpeech/model_utils/network.py", line 320, in conv_group
name='layer_{}'.format(i + 1), )
File "/home/doubleyang/DeepSpeech/model_utils/network.py", line 408, in deep_speech_v2_network
masks=masks)
File "/home/doubleyang/DeepSpeech/model_utils/model.py", line 145, in create_network
share_rnn_weights=self._share_rnn_weights)
File "/home/doubleyang/DeepSpeech/model_utils/model.py", line 297, in train
test_reader, _, ctc_loss = self.create_network()
File "train.py", line 133, in train
test_off=args.test_off)
File "train.py", line 138, in main
train()
File "train.py", line 142, in
main()

----------------------
Error Message Summary:
----------------------
Error: Alloc 1188044800 error!
[Hint: Expected posix_memalign(&p, alignment, size) == 0, but received posix_memalign(&p, alignment, size):12 != 0:0.] at (/paddle/paddle/fluid/memory/detail/system_allocator.cc:59)
[operator < conv2d > error]
Failed in training!

0
收藏
回复
全部评论(2)
时间顺序
三加一除二
#2 回复于2020-08

有解决吗

0
回复
C
Cm_oneAriesBoy
#3 回复于2021-04

no matter what I try

0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户