https://github.com/PaddlePaddle/book/blob/develop/01.fit_a_line/README.cn.md

fit a line 例子中，训练网络结构是这样的：

def train_program():
    y = fluid.layers.data(name='y', shape=[1], dtype='float32')

    # feature vector of length 13
    x = fluid.layers.data(name='x', shape=[13], dtype='float32')
    y_predict = fluid.layers.fc(input=x, size=1, act=None)

    loss = fluid.layers.square_error_cost(input=y_predict, label=y)
    avg_loss = fluid.layers.mean(loss)

    return avg_loss

我理解 y_predict 层是与 x 层产生全连接的只有一个节点的层，用这个节点的值可以直接和对应的 y 进行比较而得出方差 loss。然而令我非常困惑一个问题就是为什么要针对 loss 再求一遍 avg_loss，而不是直接返回 loss？之后我做了如下实验：

1. 将最后一行的 avg_loss 直接改为 loss

运行结果发现与测试集的损失越来越大，最后直接变成 nan

[array([307.97473], dtype=float32)]
[array([3477.0618], dtype=float32)]
[array([340660.4], dtype=float32)]
[array([3.0718845e+09], dtype=float32)]
[array([2.3560536e+17], dtype=float32)]
[array([1.4001662e+33], dtype=float32)]
[array([inf], dtype=float32)]
[array([nan], dtype=float32)]
[array([nan], dtype=float32)]
......

2. 在 1 的基础上删除 avg_loss 的定义，也就是说定义 loss 层之后直接 return。结果收敛的很快

[array([313.9322], dtype=float32)]
[array([203.85538], dtype=float32)]
[array([130.75137], dtype=float32)]
[array([194.84822], dtype=float32)]
[array([73.9117], dtype=float32)]
[array([28.367203], dtype=float32)]
[array([43.818016], dtype=float32)]
[array([16.576979], dtype=float32)]
[array([76.63061], dtype=float32)]
[array([25.644833], dtype=float32)]
[array([13.253843], dtype=float32)]
[array([26.332256], dtype=float32)]
[array([27.539469], dtype=float32)]
[array([12.963163], dtype=float32)]
[array([47.730865], dtype=float32)]
[array([25.593655], dtype=float32)]
[array([39.171013], dtype=float32)]
[array([3.710011], dtype=float32)]
loss less than 10.0. stop

附上所有代码：

import paddle
import paddle.fluid as fluid
import numpy


BATCH_SIZE = 20

train_reader = paddle.batch(
    paddle.reader.shuffle(paddle.dataset.uci_housing.train(), buf_size=500),
    batch_size=BATCH_SIZE,
)

test_reader = paddle.batch(
    paddle.reader.shuffle(paddle.dataset.uci_housing.test(), buf_size=500),
    batch_size=BATCH_SIZE,
)


def train_program():
    y = fluid.layers.data(name='y', shape=[1], dtype='float32')


    x = fluid.layers.data(name='x', shape=[13], dtype='float32')
    y_predict = fluid.layers.fc(input=x, size=1, act=None)

    loss = fluid.layers.square_error_cost(input=y_predict, label=y)
    #avg_loss = fluid.layers.mean(loss)

    return loss
    #return avg_loss


def optimizer_program():
    return fluid.optimizer.SGD(learning_rate=0.001)


use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()


trainer = fluid.Trainer(
    train_func=train_program,
    place=place,
    optimizer_func=optimizer_program,
)


step = 0
def event_handler(event):
    global step
    if isinstance(event, fluid.EndStepEvent):
        if event.step % 10 == 0:
            test_metrics = trainer.test(reader=test_reader, feed_order=feed_order)
            print(test_metrics)
            #raw_input('pause')

            if test_metrics[0] < 10.0:
                print('loss less than 10.0. stop')
                trainer.stop()

        step += 1


feed_order=['x', 'y']


trainer.train(
    reader=train_reader,
    event_handler=event_handler,
    num_epochs=100,
    feed_order=feed_order,
)

最后总结一下，我的三个问题：

1. 为什么还需要 avg_loss 层？
2. 为什么直接 return loss层会导致不收敛？
3. 为什么仅仅删除 avg_loss 层的定义，训练就收敛了？

谢谢！