pytorch中使用nn.CrossEntropyLoss()創(chuàng)建出來(lái)的交叉熵?fù)p失函數(shù)計(jì)算損失默認(rèn)是求平均值的，即多個(gè)樣本輸入后獲取的是一個(gè)均值標(biāo)量，而不是樣本大小的向量。

net = nn.Linear(4, 2)
loss = nn.CrossEntropyLoss()
X = torch.rand(10, 4)
y = torch.ones(10, dtype=torch.long)
y_hat = net(X)
l = loss(y_hat, y)
print(l)

打印的結(jié)果：tensor(0.7075, grad_fn=<NllLossBackward0>)

以上是對(duì)10個(gè)樣本做的均值的標(biāo)量

net = nn.Linear(4, 2)
loss = nn.CrossEntropyLoss(reduction='none')
X = torch.rand(10, 4)
y = torch.ones(10, dtype=torch.long)
y_hat = net(X)
l = loss(y_hat, y)
print(l)

在構(gòu)造CrossEntropyLoss時(shí)候加入?reduction='none'，就把默認(rèn)求平均取消掉了

打印結(jié)果：

tensor([0.6459, 0.7372, 0.6373, 0.6843, 0.6251, 0.6555, 0.5510, 0.7016, 0.6975,0.6849], grad_fn=<NllLossBackward0>)

以上是10個(gè)樣本各自的loss值

上圖是pytorch的CrossEntropyLoss的構(gòu)造方法，默認(rèn)是 reduction='mean'