前言：

? ?
?

這里主要實現(xiàn)： Variational Autoencoders (VAEs) 變分自動編碼器
其訓(xùn)練效果如下

?

訓(xùn)練的過程中要注意調(diào)節(jié)forward 中的kle ,調(diào)參。

整個工程兩個文件：

? ? vae.py

? ?main.py

目錄：

1. ? ?? vae
2. ? ? ? main

---

一? vae

? 文件名： vae.py

? ?作用：? ?Variational Autoencoders (VAE)

?訓(xùn)練的過程中加入一些限制，使它的latent space規(guī)則一點呢。于是就引入了variational autoencoder(VAE)，它被定義為一個有規(guī)律地訓(xùn)練以避免過度擬合的Autoencoder，可以確保潛在空間具有良好的屬性從而實現(xiàn)內(nèi)容的生成。
variational autoencoder的架構(gòu)和Autoencoder差不多，區(qū)別在于不再是把輸入當(dāng)作一個點，而是把輸入當(dāng)成一個分布。

# -*- coding: utf-8 -*-
"""
Created on Wed Aug 30 14:19:19 2023@author: chengxf2
"""import torch
from torch import nn#ae: AutoEncoderclass VAE(nn.Module):def __init__(self,hidden_size=20):super(VAE, self).__init__()self.encoder = nn.Sequential(nn.Linear(in_features=784, out_features=256),nn.ReLU(),nn.Linear(in_features=256, out_features=128),nn.ReLU(),nn.Linear(in_features=128, out_features=64),nn.ReLU(),nn.Linear(in_features=64, out_features=hidden_size),nn.ReLU())# hidden [batch_size, 10]h_dim = int(hidden_size/2)self.hDim = h_dimself.decoder = nn.Sequential(nn.Linear(in_features=h_dim, out_features=64),nn.ReLU(),nn.Linear(in_features=64, out_features=128),nn.ReLU(),nn.Linear(in_features=128, out_features=256),nn.ReLU(),nn.Linear(in_features=256, out_features=784),nn.Sigmoid())def forward(self, x):'''param x:[batch, 1,28,28]return '''batchSz= x.size(0)#flattenx = x.view(batchSz, 784)#encoderh= self.encoder(x)#在給定維度上對所給張量進行分塊,前一半的神經(jīng)元看作u, 后一般的神經(jīng)元看作sigmau, sigma = h.chunk(2,dim=1)#Reparameterize trick：#randn_like：產(chǎn)生一個正太分布 ~ N(0,1)#h.shape [batchSize,self.hDim]h = u+sigma* torch.randn_like(sigma)#kld :1e-8 防止sigma 平方為0kld = 0.5*torch.sum(torch.pow(u,2)+torch.pow(sigma,2)-torch.log(1e-8+torch.pow(sigma,2))-1)#MSE loss 是平均loss, 所以kld 也要算一個平均值kld = kld/(batchSz*32*32)xHat =   self.decoder(h)#reshapexHat = xHat.view(batchSz,1,28,28)return xHat,kld

---

二 main

文件名: main.py

作用： 訓(xùn)練，測試數(shù)據(jù)集

?

# -*- coding: utf-8 -*-
"""
Created on Wed Aug 30 14:24:10 2023@author: chengxf2
"""import torch
from torch.utils.data import DataLoader
from torchvision import transforms, datasets
import time
from torch import optim,nn
from vae import VAE
import visdomdef main():batchNum = 32lr = 1e-3epochs = 20device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")torch.manual_seed(1234)viz = visdom.Visdom()viz.line([0],[-1],win='train_loss',opts =dict(title='train acc'))tf= transforms.Compose([ transforms.ToTensor()])mnist_train = datasets.MNIST('mnist',True,transform= tf,download=True)train_data = DataLoader(mnist_train, batch_size=batchNum, shuffle=True)mnist_test = datasets.MNIST('mnist',False,transform= tf,download=True)test_data = DataLoader(mnist_test, batch_size=batchNum, shuffle=True)global_step =0model =VAE().to(device)criteon = nn.MSELoss().to(device) #損失函數(shù)optimizer = optim.Adam(model.parameters(),lr=lr) #梯度更新規(guī)則print("\n ----main-----")for epoch in range(epochs):start = time.perf_counter()for step ,(x,y) in enumerate(train_data):#[b,1,28,28]x = x.to(device)x_hat,kld = model(x)loss = criteon(x_hat, x)if kld is not None:elbo = -loss -1.0*kldloss = -elbo#backpropoptimizer.zero_grad()loss.backward()optimizer.step()viz.line(Y=[loss.item()],X=[global_step],win='train_loss',update='append')global_step +=1end = time.perf_counter()    interval = int(end - start)print("epoch: %d"%epoch, "\t 訓(xùn)練時間 %d"%interval, '\t 總loss: %4.7f'%loss.item(),"\t KL divergence: %4.7f"%kld.item())x,target = iter(test_data).next()x = x.to(device)with torch.no_grad():x_hat,kld = model(x)tip = 'hat'+str(epoch)viz.images(x,nrow=8, win='x',opts=dict(title='x'))viz.images(x_hat,nrow=8, win='x_hat',opts=dict(title=tip))if __name__ == '__main__':main()

?參考：

?課時118 變分Auto-Encoder實戰(zhàn)-2_嗶哩嗶哩_bilibili