首页 Paddle框架 帖子详情
paddle.topk逆序出现-inf的bug
收藏
快速回复
Paddle框架 问答模型训练 402 5
paddle.topk逆序出现-inf的bug
收藏
快速回复
Paddle框架 问答模型训练 402 5

现有一个Tensor如图所示(无法上传原始数据),取值在-0.21761519~0.11902207之间.

import paddle
import numpy as np
data = np.loadtxt("data.txt")
data_t = paddle.to_tensor(data)
# 检测数据没有inf值
paddle.masked_select(data_t, data_t.isinf())
# 输出结果
Tensor(shape=[0], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
 [])
# 检测数据没有nan值
paddle.masked_select(data_t, data_t.isnan())
# 输出结果
Tensor(shape=[0], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
 [])
# 用paddle.topk方法选择数据中最小的前50个.
data_t.topk(50, largest=False)
# 输出结果,最后一个数据是-inf,同时索引存在重复,2105出现了两次.
(Tensor(shape=[50], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
 [-0.21761519, -0.21514374, -0.20108491, -0.19833624, -0.19480723, -0.19176239, -0.18954855, -0.18879974, -0.18704271, -0.17960167, -0.17840427, -0.17787200, -0.17590189, -0.17435610, -0.17403901, -0.17102230, -0.16744429, -0.16469669, -0.16408825, -0.16370595, -0.16119164, -0.16029066, -0.15845722, -0.15344226, -0.15212250, -0.15173113, -0.15163857, -0.15163130, -0.15160024, -0.15026754, -0.15001744, -0.14997017, -0.14973289, -0.14502805, -0.14425749, -0.14366084, -0.14354008, -0.14346808, -0.14306796, -0.14291716, -0.14237219, -0.14187407, -0.14064771, -0.14032555, -0.14004153, -0.13949740, -0.13926125, -0.13925231, -0.13907176, -inf. ]),
 Tensor(shape=[50], dtype=int64, place=CUDAPlace(0), stop_gradient=False,
 [408 , 1718, 294 , 257 , 2626, 3110, 849 , 3189, 1961, 1420, 2608, 2186, 959 , 870 , 502 , 2413, 2134, 1567, 2105, 405 , 3153, 3020, 2597, 2279, 1314, 2654, 934 , 3129, 3060, 3467, 1066, 3013, 438 , 539 , 1446, 2094, 2705, 447 , 3487, 3364, 3385, 1346, 313 , 875 , 3150, 526 , 561 , 1639, 2361, 2105]))
# 对数据先进行一次排序
data_t= data_t.sort()
# 再用paddle.topk方法选择数据中最小的前50个.
data_t.topk(50, largest=False)
(Tensor(shape=[50], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
 [-0.21761519, -0.21514374, -0.20108491, -0.19833624, -0.19480723, -0.19176239, -0.18954855, -0.18879974, -0.18704271, -0.17960167, -0.17840427, -0.17787200, -0.17590189, -0.17435610, -0.17403901, -0.17102230, -0.16744429, -0.16469669, -0.16408825, -0.16370595, -0.16119164, -0.16029066, -0.15845722, -0.15344226, -0.15212250, -0.15173113, -0.15163857, -0.15163130, -0.15160024, -0.15026754, -0.15001744, -0.14997017, -0.14973289, -0.14502805, -0.14425749, -0.14366084, -0.14354008, -0.14346808, -0.14306796, -0.14291716, -0.14237219, -0.14187407, -0.14064771, -0.14032555, -0.14004153, -0.13949740, -0.13926125, -0.13925231, -0.13907176, -0.13810444]),
 Tensor(shape=[50], dtype=int64, place=CUDAPlace(0), stop_gradient=False,
 [0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]))
# -inf不再出现,程序正常运行.

非环境问题,在AIstudio上同样会出现该问题.可能是数据问题(已排除,同样的数据在pytoch上没问题),用paddle.randn生成的数据暂未发现该问题.

运算环境为显卡,在CPU运算环境下该问题同样不会出现.

 

0
收藏
回复
全部评论(5)
时间顺序
三岁
#2 回复于2021-08

是哪个paddle版本

0
回复
JavaRoom
#3 回复于2021-08

没看懂

0
回复
d
docoter_c
#4 回复于2021-08
三岁 #2
是哪个paddle版本

环境版本是:
paddlepaddle-gpu 2.1.1
padans 1.3.1
numpy 1.20.3
cuda 11.2

我用随机生成的数据也不会出现该问题.
原始数据链接如下:
链接:https://pan.baidu.com/s/1ZLttEwVxObxevkKwXWDK9g
提取码:tzn7

import numpy as np
import paddle
data = np.loadtxt("data.txt")
data_t = paddle.to_tensor(data)

0
回复
d
docoter_c
#5 回复于2021-08
三岁 #2
是哪个paddle版本

环境版本是:
paddlepaddle-gpu 2.1.1
padans 1.3.1
numpy 1.20.3
cuda 11.2

我用随机生成的数据也不会出现该问题.
原始数据链接如下:
链接:https://pan.baidu.com/s/1ZLttEwVxObxevkKwXWDK9g
提取码:tzn7

import numpy as np
import paddle
data = np.loadtxt("data.txt")
data_t = paddle.to_tensor(data)

0
回复
d
docoter_c
#6 回复于2021-08
没看懂
import paddle
import numpy as np
data = np.loadtxt("data.txt")
data_t = paddle.to_tensor(data)
# 检测数据没有inf值
paddle.masked_select(data_t, data_t.isinf())
# 输出结果
Tensor(shape=[0], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
       [])
# 检测数据没有nan值
paddle.masked_select(data_t, data_t.isnan())
# 输出结果
Tensor(shape=[0], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
       [])
# 用paddle.topk方法选择数据中最小的前50个.
data_t.topk(50, largest=False)
# 输出结果,最后一个数据是-inf,同时索引存在重复,2105出现了两次.
(Tensor(shape=[50], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
        [-0.21761519, -0.21514374, -0.20108491, -0.19833624, -0.19480723, -0.19176239, -0.18954855, -0.18879974, -0.18704271, -0.17960167, -0.17840427, -0.17787200, -0.17590189, -0.17435610, -0.17403901, -0.17102230, -0.16744429, -0.16469669, -0.16408825, -0.16370595, -0.16119164, -0.16029066, -0.15845722, -0.15344226, -0.15212250, -0.15173113, -0.15163857, -0.15163130, -0.15160024, -0.15026754, -0.15001744, -0.14997017, -0.14973289, -0.14502805, -0.14425749, -0.14366084, -0.14354008, -0.14346808, -0.14306796, -0.14291716, -0.14237219, -0.14187407, -0.14064771, -0.14032555, -0.14004153, -0.13949740, -0.13926125, -0.13925231, -0.13907176, -inf.      ]),
 Tensor(shape=[50], dtype=int64, place=CUDAPlace(0), stop_gradient=False,
        [408 , 1718, 294 , 257 , 2626, 3110, 849 , 3189, 1961, 1420, 2608, 2186, 959 , 870 , 502 , 2413, 2134, 1567, 2105, 405 , 3153, 3020, 2597, 2279, 1314, 2654, 934 , 3129, 3060, 3467, 1066, 3013, 438 , 539 , 1446, 2094, 2705, 447 , 3487, 3364, 3385, 1346, 313 , 875 , 3150, 526 , 561 , 1639, 2361, 2105]))
# 对数据先进行一次排序
data_t= data_t.sort()
# 再用paddle.topk方法选择数据中最小的前50个.
data_t.topk(50, largest=False)
(Tensor(shape=[50], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
        [-0.21761519, -0.21514374, -0.20108491, -0.19833624, -0.19480723, -0.19176239, -0.18954855, -0.18879974, -0.18704271, -0.17960167, -0.17840427, -0.17787200, -0.17590189, -0.17435610, -0.17403901, -0.17102230, -0.16744429, -0.16469669, -0.16408825, -0.16370595, -0.16119164, -0.16029066, -0.15845722, -0.15344226, -0.15212250, -0.15173113, -0.15163857, -0.15163130, -0.15160024, -0.15026754, -0.15001744, -0.14997017, -0.14973289, -0.14502805, -0.14425749, -0.14366084, -0.14354008, -0.14346808, -0.14306796, -0.14291716, -0.14237219, -0.14187407, -0.14064771, -0.14032555, -0.14004153, -0.13949740, -0.13926125, -0.13925231, -0.13907176, -0.13810444]),
 Tensor(shape=[50], dtype=int64, place=CUDAPlace(0), stop_gradient=False,
        [0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]))
# -inf不再出现,程序正常运行.
0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户