PaddleSeg动态图遥感比赛Baseline

项目

数据集

课程

比赛

模型库

活动

论坛

访问飞桨官网

项目

数据集

课程

比赛

模型库

活动

论坛

访问飞桨官网

m michaelowenliu 发布于2020-11

2020 CCF BDCI 遥感影像地块分割比赛主办方提供了PaddleSeg静态图版本作为baseline，现正值飞桨框架推荐使用动态图，作为飞桨的图像分割工具套件PaddleSeg动态图版也已经适配了Paddle 2.0rc版。Seg的动态图版本是今后推荐使用的，第一期支持的模型数和精度也已经超过静态图版了，但目前在处于实验阶段，并未全面向用户推出，借着这次比赛机会，我们也测试了动态图的性能。

作为动态图的baseline，我们选择了一个比较简单的分割模型，HRNet_W48+FCN，并未做任何调参，配置文件以及测试结果在这里能找到：

PaddleSeg动态图主目录：https://github.com/PaddlePaddle/PaddleSeg/tree/develop/dygraph

遥感分割比赛配置文件：https://github.com/PaddlePaddle/PaddleSeg/tree/develop/dygraph/contrib/remote_sensing

数据准备参考：https://aistudio.baidu.com/aistudio/projectdetail/1090790

训练方式：

python -m paddle.distributed.launch train.py --config contrib/remote_sensing/fcn_hrnetw48_ccf_256x256_160k.yml --save_dir save_model_path --save_interval 8000 --log_iters 200 --do_eval --use_vdl

单尺度评估：

python -m paddle.distributed.launch val.py --config contrib/remote_sensing/fcn_hrnetw48_ccf_256x256_160k.yml --model_path save_model_path/best_model/model.pdparams

多尺度评估：

python -m paddle.distributed.launch val.py --config contrib/remote_sensing/fcn_hrnetw48_ccf_256x256_160k.yml --model_path save_model_path/best_model/model.pdparams --aug_eval --scales 1.0 1.25 --flip_horizontal

我们采用了1.0和1.25倍的多尺度做一个示例（同学们可以尝试更多组合），另外加了水平翻转。

生成上传图片：

python predict.py --config contrib/remote_sensing/fcn_hrnetw48_ccf_256x256_160k.yml --model_path save_model_path/best_model/model.pdparams --image_path dataset/rs_data/img_testA --aug_pred --scales 1.0 1.25 --flip_horizontal --save_dir output_images_path

和多尺度评估方式类似。

我们最后得到的结果：在单尺度验证集mIoU是71.55%，增加多尺寸和翻转验证集mIoU是72.42%，最后在初赛的测试集mIoU是70.36%，结果和训练模型都能在PaddleSeg动态图里面找到：

https://github.com/PaddlePaddle/PaddleSeg/tree/develop/dygraph/contrib/remote_sensing

目前PaddleSeg动态图版本正在快马加鞭的完善文档中，可能有些遗漏，有不清楚的欢迎直接加QQ群（850378321）圈我们，如果对大家有帮助，希望能在github上给我们PaddleSeg点小星星。

https://github.com/PaddlePaddle/PaddleSeg

全部评论(32)

冰河

#2 回复于2020-11

强！

Zeyu

#3 回复于2020-11

赞！好强！

gwwzw

#4 回复于2020-11

厉害了！

Chen G W

#5 回复于2020-11

非常棒

AIStudio3319643

#6 回复于2020-11

厉害了

Jordan2020

#7 回复于2020-11

早点出来就好了，费了不少算力

Seigato

#8 回复于2020-11

比赛快结束的时候开源了一个70分的baseline。。。

信

信天翁flying

#9 回复于2020-11

想请教下，test没有对应的label，配置文件总是报错缺失

，怎么办呢？

michaelowenliu

#10 回复于2020-11

信天翁flying #9

想请教下，test没有对应的label，配置文件总是报错缺失，怎么办呢？

配置文件里不需要配置test的

双

双子yshhhs

#11 回复于2020-11

您好，感谢分享。我想请问静态图版本的HRNet_W48也能达到差不多的效果吗？

michaelowenliu

#12 回复于2020-11

双子yshhhs #11

您好，感谢分享。我想请问静态图版本的HRNet_W48也能达到差不多的效果吗？

你好，静态图我们没测试过，但应该能达到差不多的结果。

信

信天翁flying

#13 回复于2020-11

这是怎么回事啊！！！
model:
  type: FCN
  backbone:
    type: HRNet_W48
    pretrained: pretrained_model/hrnet_w18_ssld
  num_classes: 8
  pretrained: Null
  backbone_indices: [-1]

TypeError: __init__() missing 1 required positional argument: 'num_classes'

ZHY_788

#14 回复于2020-11

我用你的方法跑出了基本的模型，然后进行预测，但是生成的是.jpg后缀的图片，提交进行测验失败。修改到符合ccf大赛的.png格式后，评测系统提示需要输入单通道的像素图片，这个怎么能让predict.py文件直接生成符合题目要求的图片格式啊，我尝试进行修改pridict.py文件，但是提示我没有相应的权限。大佬，求教。

michaelowenliu

#15 回复于2020-11

ZHY_788 #14

展开

请用develop分支

ZHY_788

#16 回复于2020-11

michaelowenliu #15

请用develop分支

不太懂，是用最新的Paddleseg文件吗，我这个是一个月前colne的。

离

离原牧马

#17 回复于2020-11

为毛静态图deeplabv3+一直徘徊在0.63的样子，加了一堆的trick都上不去，动态图一个fcn就0.7了。。。

Lito1XxxX

#18 回复于2020-11

您好这是什么原因呢？

Traceback (most recent call last):
  File "dygraph/train.py", line 141, in 
    main(args)
  File "dygraph/train.py", line 136, in main
    losses=losses)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/core/train.py", line 119, in train
    logits = model(images)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/fcn.py", line 66, in forward
    feat_list = self.backbone(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/backbones/hrnet.py", line 169, in forward
    st4 = self.st4(tr3)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/backbones/hrnet.py", line 512, in forward
    out = self.stage_func_list[idx](out)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/backbones/hrnet.py", line 542, in forward
    out = self.branches_func(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/backbones/hrnet.py", line 303, in forward
    conv = basic_block_func(conv)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/backbones/hrnet.py", line 420, in forward
    conv2 = self.conv2(conv1)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-develop/dygraph/paddleseg/models/layers/layer_libs.py", line 63, in forward
    x = self._batch_norm(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 829, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py", line 1037, in forward
    variance_out, *attrs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/multiprocess_utils.py", line 134, in __handler__
    core._throw_error_if_process_failed()
paddle.fluid.core_avx.EnforceNotMet: 

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0   paddle::imperative::ThrowErrorIfLoadProcessFailed()
1   paddle::platform::GetCurrentTraceBackString[abi:cxx11]()

----------------------
Error Message Summary:
----------------------
FatalError: DataLoader process (pid   1. If run DataLoader by DataLoader.from_generator(...), queue capacity is set by from_generator(..., capacity=xx, ...).
  2. If run DataLoader by DataLoader(dataset, ...), queue capacity is set as 2 times of the max value of num_workers and len(places).
  3. If run by DataLoader(dataset, ..., use_shared_memory=True), set use_shared_memory=False for not using shared memory.) exited is killed by signal: 4202.
  It may be caused by insufficient shared storage space. This problem usually occurs when using docker as a development environment.
  Please use command `df -h` to check the storage space of `/dev/shm`. Shared storage space needs to be greater than (DataLoader Num * DataLoader queue capacity * 1 batch data size).
  You can solve this problem by increasing the shared storage space or reducing the queue capacity appropriately.
Bus error (at /paddle/paddle/fluid/imperative/data_loader.cc:161)

INFO 2020-11-24 21:11:30,646 utils.py:275] terminate all the procs
ERROR 2020-11-24 21:11:30,646 utils.py:445] ABORT!!! Out of all 1 trainers, the trainer process with rank=[0] was aborted. Please check its log.
INFO 2020-11-24 21:11:33,649 utils.py:275] terminate all the procs

michaelowenliu

#19 回复于2020-11

Lito1XxxX #18

您好这是什么原因呢？ [代码]

如果是aistudio的话，将num_workers设置成0，因为共享缓存不足。

Irene2712

#20 回复于2020-11

请问AI STUIDIO平台的GPU跑完测试需要多久呢

Seigato

#21 回复于2020-11

我看了一下log信息，发现是用的4张V100训练的，每张卡上bs16，所以每轮实际上bs是64。如果我们用aistudio上单卡跑的话，使用同样配置，要跑100多个小时才能训完。。欲哭无泪。。缺算力呀