Out of memory error on GPU 0. Cannot allocate 274.504019PB memory on GPU 0, 1.618652GB memory has been allocated and available memory is only 8.380859GB.
这个不是绰绰有余吗?我看别人不够都是前大后小的,只有我前小后大,离谱,当我实际运行时,GPU也没被占满,我看任务管理器,运行时只增加了0.7个G,还有好多没用。
运行的时候出现这个 0/99 [00:13
完整报错如下:
Model DAIN process start..
[03/26 15:50:25] ppgan INFO: Found C:\Users\admin\.cache\ppgan\DAIN_weight.tar
[03/26 15:50:25] ppgan INFO: Decompressing C:\Users\admin\.cache\ppgan\DAIN_weight.tar...
Sun Mar 26 15:50:25-WARNING: The old way to load inference model is deprecated. model path: C:\Users\admin\.cache\ppgan\DAIN_weight\model, params path: C:\Users\admin\.cache\ppgan\DAIN_weight\params
W0326 15:50:27.045195 3564 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.7, Runtime API Version: 10.2
W0326 15:50:27.060817 3564 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6.
Old fps (frame rate): 25.0
New fps (frame rate): 50
0%| | 0/99 [00:00
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
Not support stack backtrace yet.
----------------------
Error Message Summary:
----------------------
ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 274.504019PB memory on GPU 0, 1.618652GB memory has been allocated and available memory is only 8.380859GB.
Please check whether there is any other process using GPU 0.
1. If yes, please stop them, or start PaddlePaddle on another GPU.
2. If no, please decrease the batch size of your model.
(at ..\paddle\fluid\memory\allocation\cuda_allocator.cc:95)
0%| | 0/99 [00:13Traceback (most recent call last):
File "D:\PaddleGan\PaddleGAN\applications\tools\video-enhance.py", line 135, in
frames_path, temp_video_path = predictor.run(temp_video_path)
File "d:\paddlegan\paddlegan\ppgan\apps\dain_predictor.py", line 161, in run
o = self.base_forward(X)
File "d:\paddlegan\paddlegan\ppgan\apps\base_predictor.py", line 59, in base_forward
out = self.exe.run(self.program,
File "D:\Anaconda\lib\site-packages\paddle\fluid\executor.py", line 1463, in run
six.reraise(*sys.exc_info())
File "D:\Anaconda\lib\site-packages\six.py", line 719, in reraise
raise value
File "D:\Anaconda\lib\site-packages\paddle\fluid\executor.py", line 1450, in run
res = self._run_impl(program=program,
File "D:\Anaconda\lib\site-packages\paddle\fluid\executor.py", line 1661, in _run_impl
return new_exe.run(scope, list(feed.keys()), fetch_list,
File "D:\Anaconda\lib\site-packages\paddle\fluid\executor.py", line 631, in run
tensors = self._new_exe.run(scope, feed_names,
RuntimeError:
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
Not support stack backtrace yet.
----------------------
Error Message Summary:
----------------------
ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 274.504019PB memory on GPU 0, 1.618652GB memory has been allocated and available memory is only 8.380859GB.
Please check whether there is any other process using GPU 0.
1. If yes, please stop them, or start PaddlePaddle on another GPU.
2. If no, please decrease the batch size of your model.
(at ..\paddle\fluid\memory\allocation\cuda_allocator.cc:95)
文件也是找不到啊,找不到哪里改batch_size
在哪启动的就在哪改。需求274PB内存。。怕不是运行时把参数输错了位置
改下吧