首页 Paddle框架 帖子详情
自定义数据集无法(不知道如何)进行训练 已解决
收藏
快速回复
Paddle框架 问答模型训练 1909 4
自定义数据集无法(不知道如何)进行训练 已解决
收藏
快速回复
Paddle框架 问答模型训练 1909 4

第一次用paddle,真的不知道错在哪里,麻烦大佬帮帮忙

自定义的数据集,全部是(32, 32)的8位灰度图

(4张样例)

运行代码如下:

import paddle
import numpy as np
import pandas as pd
from PIL import Image
from paddle.io import Dataset
from paddle.vision.transforms import ToTensor

class DnsImgDateset(Dataset):
    """自定义DNS图像数据集"""
    def __init__(self):
        super(DnsImgDateset, self).__init__()     
        # read dataset into memory
        self.__read_dataset()
        
    def __read_dataset(self):
        dataset_file_path = './DnsDataset.csv'
        self.images = []
        self.labels = []
        
        df = pd.read_csv(dataset_file_path)
        image_path_list = df['image_path'].values
        label_list = df['label'].values
        self.num_samples = len(image_path_list)
        
        for x, y in zip(image_path_list, label_list):
            f = open(x, 'rb')
            image = Image.open(f).convert('L')
            f.close()
            image = np.array(image).astype(np.uint8)
            self.images.append(image)
            label = np.array([y]).astype(np.bool_)
            self.labels.append(label)
        
    def __getitem__(self, index):
        image, label = self.images[index], self.labels[index]
        return image, label
        
    def __len__(self):
        return self.num_samples

train_dataset = DnsImgDateset()

# 定义网络结构(采用 Sequential组网方式 )
dns_img = paddle.nn.Sequential(
    paddle.nn.Flatten(),
    paddle.nn.Linear(1024, 512),
    paddle.nn.ReLU(),
    paddle.nn.Dropout(0.3),
    paddle.nn.Linear(512, 10)
)

model = paddle.Model(dns_img)

# 为模型训练做准备,设置优化器,损失函数和精度计算方式
model.prepare(optimizer=paddle.optimizer.Adam(parameters=model.parameters()),
              loss=paddle.nn.CrossEntropyLoss(),
              metrics=paddle.metric.Accuracy())

# 启动模型训练,指定训练数据集,设置训练轮次,设置每次数据集计算的批次大小,设置日志格式
model.fit(train_dataset,
          epochs=5,
          batch_size=64,
          verbose=1)

报错如下:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_4952/1619117630.py in 
     18 
     19 # 启动模型训练,指定训练数据集,设置训练轮次,设置每次数据集计算的批次大小,设置日志格式
---> 20 model.fit(train_dataset,
     21           epochs=5,
     22           batch_size=64,

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\hapi\model.py in fit(self, train_data, eval_data, batch_size, epochs, eval_freq, log_freq, save_dir, save_freq, verbose, drop_last, shuffle, num_workers, callbacks)
   1720         for epoch in range(epochs):
   1721             cbks.on_epoch_begin(epoch)
-> 1722             logs = self._run_one_epoch(train_loader, cbks, 'train')
   1723             cbks.on_epoch_end(epoch, logs)
   1724 

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\hapi\model.py in _run_one_epoch(self, data_loader, callbacks, mode, logs)
   2026 
   2027             if mode != 'predict':
-> 2028                 outs = getattr(self, mode + '_batch')(data[:len(self._inputs)],
   2029                                                       data[len(self._inputs):])
   2030                 if self._metrics and self._loss:

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\hapi\model.py in train_batch(self, inputs, labels)
   1063               print(loss)
   1064         """
-> 1065         loss = self._adapter.train_batch(inputs, labels)
   1066         if fluid.in_dygraph_mode() and self._input_info is None:
   1067             self._update_inputs()

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\hapi\model.py in train_batch(self, inputs, labels)
    720                     *[to_variable(x) for x in inputs])
    721             else:
--> 722                 outputs = self.model.network.forward(
    723                     *[to_variable(x) for x in inputs])
    724 

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\fluid\dygraph\container.py in forward(self, input)
     96     def forward(self, input):
     97         for layer in self._sub_layers.values():
---> 98             input = layer(input)
     99         return input
    100 

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\fluid\dygraph\layers.py in __call__(self, *inputs, **kwargs)
    900                 self._built = True
    901 
--> 902             outputs = self.forward(*inputs, **kwargs)
    903 
    904             for forward_post_hook in self._forward_post_hooks.values():

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\nn\layer\common.py in forward(self, input)
    126 
    127     def forward(self, input):
--> 128         out = F.linear(
    129             x=input, weight=self.weight, bias=self.bias, name=self.name)
    130         return out

d:\app\anaconda3\envs\pd\lib\site-packages\paddle\nn\functional\common.py in linear(x, weight, bias, name)
   1448     if in_dygraph_mode():
   1449         pre_bias = _varbase_creator(dtype=x.dtype)
-> 1450         core.ops.matmul(x, weight, pre_bias, 'transpose_X', False,
   1451                         'transpose_Y', False, "alpha", 1)
   1452         return dygraph_utils._append_bias_in_dygraph(

RuntimeError: (NotFound) Operator matmul does not have kernel for data_type[uint8_t]:data_layout[ANY_LAYOUT]:place[CPUPlace]:library_type[PLAIN].
  [Hint: Expected kernel_iter != kernels.end(), but received kernel_iter == kernels.end().] (at C:\home\workspace\Paddle_release\paddle\fluid\imperative\prepared_operator.cc:135)
  [operator < matmul > error]

 

自定义读取数据集是没问题的

 

 

 

 

芥子企划
已解决
3# 回复于2021-08
将image的数据类型修改为float类型就能进行训练了,很奇怪
0
收藏
回复
全部评论(4)
时间顺序
三岁
#2 回复于2021-08

你看看数据类型修改一下

0
回复
芥子企划
#3 回复于2021-08
三岁 #2
你看看数据类型修改一下

将image的数据类型修改为float类型就能进行训练了,很奇怪

0
回复
三岁
#4 回复于2021-08
将image的数据类型修改为float类型就能进行训练了,很奇怪

就是数据类型不对,这个报错正常

0
回复
FutureSI
#5 回复于2021-08

paddle一般默认输入float32类型

0
回复
需求/bug反馈?一键提issue告诉我们
发现bug?如果您知道修复办法,欢迎提pr直接参与建设飞桨~
在@后输入用户全名并按空格结束,可艾特全站任一用户