系列文章目錄

本文專門(mén)開(kāi)一節(jié)寫(xiě)Lora相關(guān)的內(nèi)容，在看之前，可以同步關(guān)注：
stable diffusion實(shí)踐操作

---

---

前言

Lora是微調(diào)模型，某品牌汽車（特斯拉），人物姿勢(shì)（剪刀手），可以固定我們照片的特征：人物特征、動(dòng)作特征、還有照片風(fēng)格。

如果把基礎(chǔ)模型比喻作一座房子的地基，那么Lora模型就好比在這個(gè)地基上蓋起來(lái)的房子。我們通常也稱為微調(diào)模型，用于滿足一種特定的風(fēng)格，或指定的人物特征屬性。在數(shù)據(jù)相似度非常高的情形下，使用微調(diào)模型，可以節(jié)省大量的訓(xùn)練時(shí)間和訓(xùn)練資源，就可以產(chǎn)出我們需要的結(jié)果.，常見(jiàn)的后綴是ckpt,safetensors,pt，大小100M左右。

Lora是可以疊加使用的，疊加后會(huì)非常變態(tài)的。
Lora之間一樣用英文逗號(hào)隔開(kāi)，每個(gè)Lora后面都有數(shù)字，這是用來(lái)調(diào)整這個(gè)Lora的權(quán)重的。
正常情況下是1，我們一般只會(huì)去降低權(quán)重，因?yàn)樵黾訖?quán)重照片可能就會(huì)變得奇奇怪怪，每個(gè)Lora設(shè)置的權(quán)重不一樣，出來(lái)的照片就會(huì)不一樣，想要生成一個(gè)好看的小姐姐，就要多去嘗試不同的權(quán)重組合，并且lora有提示詞，如下：

<lora:Beautiful_Dress_1.0:0.6>,dress

---

模型分兩種，一種是sd大模型，一種是類似Lora一樣的小模型
國(guó)內(nèi)的是：https://www.liblibai.com
國(guó)外的是：https://civitai.com

一、什么是lora?

1.1 lora 定義

LoRA，英文全稱Low-Rank Adaptation of Large Language Models，直譯為大語(yǔ)言模型的低階適應(yīng)，這是微軟的研究人員為了解決大語(yǔ)言模型微調(diào)而開(kāi)發(fā)的一項(xiàng)技術(shù)。

1.2 lora的基本原理

大模型，比如，GPT-3有1750億參數(shù)，為了讓它能干特定領(lǐng)域的活兒，需要做微調(diào)，但是如果直接對(duì)GPT-3做微調(diào)，成本太高太麻煩了。

LoRA的做法是，凍結(jié)預(yù)訓(xùn)練好的模型權(quán)重參數(shù)，然后在每個(gè)Transformer（Transforme就是GPT的那個(gè)T）塊里注入可訓(xùn)練的層，由于不需要對(duì)模型的權(quán)重參數(shù)重新計(jì)算梯度，所以，大大減少了需要訓(xùn)練的計(jì)算量。

如下圖，lora在PLM旁邊增加一個(gè)旁路，做一個(gè)降維再升維的操作，訓(xùn)練的時(shí)候固定PLM參數(shù)，只訓(xùn)練降維矩陣A和升維矩陣B，模型輸入與輸出維度不變。輸出時(shí)將BA與PLM的參數(shù)疊加

在微調(diào)sd的情況下，lora可以應(yīng)用于將圖像表示于描述他們的提示詞相關(guān)聯(lián)的交叉注意層。

也就是說(shuō)lora在不干涉神經(jīng)網(wǎng)絡(luò)運(yùn)算層的情況下，插入一些低階運(yùn)算層，通過(guò)訓(xùn)練這些人低階運(yùn)算層來(lái)影響出圖

順便看看和超網(wǎng)絡(luò)的區(qū)別，下面是超網(wǎng)絡(luò)，看出來(lái)兩者區(qū)別沒(méi)有：

研究發(fā)現(xiàn)，LoRA的微調(diào)質(zhì)量與全模型微調(diào)相當(dāng)，我愿稱之為神器。

要做個(gè)比喻的話，就好比是大模型的一個(gè)小模型，或者說(shuō)是一個(gè)插件。

LoRA本來(lái)是給大語(yǔ)言模型準(zhǔn)備的，但把它用在cross-attention layers（交叉關(guān)注層）也能影響用文字生成圖片的效果。

1.2 通過(guò)分層控制lora

（TB）

二、作用：

2.1 復(fù)刻人物特征

對(duì)人物和物品的復(fù)刻，只要掛載lora，就可以九成九的復(fù)刻指定人物的特征。
比如，你下載了光頭強(qiáng)的Lora，只要你加載了它，那么你生成的任何一張圖片都是他的樣子。

2.2 固定人物的動(dòng)作特征

2.3 確定某種畫(huà)風(fēng)

總結(jié)：Lora可以確定某種特征。

Lora一定要和底模配合使用。

三、lora和embedding 區(qū)別：

3.1 文件大小不一樣

文件小攜帶的信息就比較少。

3.2 lora比embedding要好的多

embedding在多視圖和動(dòng)漫人物上要好一點(diǎn)。

3.3 embedding是文字描述，lora是真實(shí)的物品描述。

四 、lora使用

4.1 用lora配套的大模型

因?yàn)閘ora訓(xùn)練是需要一個(gè)大模型的，是通過(guò)大模型來(lái)訓(xùn)練參數(shù)的人，所以它的一切和大模型都綁定了。
所以使用的時(shí)候，要和訓(xùn)練的原始模型匹配。
下面是hanfu lora和不同大模型的對(duì)比：

有些效果可以，有些效果不好，這就開(kāi)盲盒了。

4.2 .有觸發(fā)詞一定要使用觸發(fā)詞

<lora:baihuaniang_1.0:0.8>,baihuaniang，flower

一般來(lái)說(shuō)，不用把全部的觸發(fā)詞加上去，只要加其中一個(gè)就可以了，比如下面的有漢服，有唐服等等，這個(gè)要根據(jù)lora模型來(lái)確定，有寫(xiě)lora是沒(méi)有提示詞的。

4.3 正確設(shè)置lora的權(quán)重

權(quán)重越大，lora對(duì)結(jié)果影響就很大。但是也不能過(guò)大，過(guò)小，一般在0.6-0.9之間。

4.4 新手不要混用lora

多了會(huì)打架。

五、 收集到的lora模型

---

序號(hào)	lora名稱	說(shuō)明
1	baihuaniang_1.0
2	long	龍
3	xiaorenshu_v2.0	小人書(shū)
4	Lora_GuoFeng3_Lora_v3.2_Lora_big_light	中國(guó)風(fēng)
5	Real dragons _dragon_real_base_V1	real dragon
6	korean doll likeness	美超真實(shí)女
7	Beautiful_Dress	連衣裙
8	qianlijiangshanXXMix-V1.0	千里江山
9	$1
10	$1

１、baihuaniang_1.0

下載鏈接：https://www.liblibai.com/modelinfo/1d2839ecaaf829e0161beb8c70294dbf
基礎(chǔ)模型：SD1.5（AWPainting_v1.2_liblib_sd15.safetensors）
觸發(fā)詞語(yǔ)：baihuaniang，flower

案例：
正向提示詞：

Best quality,masterpiece,ultra high res,(photorealistic:1.4),
solo,1girl,.<lora:baihuaniang_1.0:0.8>,baihuaniang,Flowers,
Flowers,Soft sunlight,Wind,Flying long hair,Film Light,
Facing the screen,.copper,,upper body,fatigues,
outstretched arms,.

反向提示詞：

ng_deepnegative_v1_75t,badhandv4 (worst quality:2),
(low quality:2),(normal quality:2),lowres,bad anatomy,
bad hands,normal quality,((monochrome)),((grayscale)),, 
ng_deepnegative_v1_75t, badhandv4 (worst quality:2), 
(low quality:2), (normal quality:2), lowres, bad anatomy, 
bad hands, normal quality, ((monochrome)), ((grayscale)):.

采樣：
DPM++2MKarras
隨機(jī)種子：2514900086
CFGscale:11
步數(shù)：28

生成結(jié)果：

2、long.safetensors　中國(guó)龍

下載鏈接：https://www.liblibai.com/modelinfo/870bee7ee8377e037097ae2d16839709
觸發(fā)詞：no humans， dragon， long
基礎(chǔ)模型SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
正向提示詞：

Best quality,masterpiece,ultra high res,nu no humans, (long:1.2),
<lora:long:0.8>,no humans， dragon， long eastern dragon, 
east asian architecture, sky, architecture, outdoors, cloud, day, 
mountain, open mouth, blue sky, teeth, scales, sharp teeth, 
horns, glowing, fangs, tree, glowing eyes, pagoda, scenery.

反向提示詞：

ng_deepnegative_v1_75t,badhandv4 (worst quality:2),(low quality:2),
(normal quality:2),lowres,bad anatomy,bad hands,normal quality,
((monochrome)),((grayscale)),nsfw,..

采樣：
DPM++2MKarras
隨機(jī)種子：2330800025
CFGscale:11
步數(shù)：28

生成結(jié)果：

3、xiaorenshu_v2.0.safetensors

下載鏈接：https://www.liblibai.com/modelinfo/7755bdd92e3baeb7bd8ac79a455070ad
基礎(chǔ)模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
觸發(fā)詞語(yǔ)：沒(méi)有

正向提示詞：
shz,((bestquality)),((masterpiece)),(detailed),realistic,multipleboys,(ridinghorse),mustache,lookingatviewer,outdoors,house,amy,horse

反向提示詞：

采樣：DPM++2MKarras
隨機(jī)種子：3836055402
CFGscale: 5
步數(shù)：28

生成結(jié)果：

4、Lora_GuoFeng3_Lora_v3.2_Lora_big_light.safetensors

下載鏈接：https://www.liblibai.com/modelinfo/7b3ce08ce87271586f0fd00de32604be
基礎(chǔ)模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
觸發(fā)詞語(yǔ)：沒(méi)有

正向提示詞：

bestquality,masterpiece,highres,1girl,chinadress,Beautifulface,
hairornament,solo,lookingatviewer,smile,closedmouth,lips,
chinadress,dress,hairornament,necklace,jewelry,longhair,earrings,chineseclothes,
architecture,eastasianarchitecture,building,outdoors,rooftop,city,cityscape,
darkstudio,rimlighting,twotonelighting,dimlylit,lowkey
<lora:GuoFeng3.2_Lora_light:1.5>

采樣：Eulera
隨機(jī)種子：3985684671
CFGscale: 7
步數(shù)：30

生成結(jié)果：

5、Lora Real dragons _dragon_real_base_V1.safetensors

下載鏈接：https://www.liblibai.com/modelinfo/6d1bbd25afd9c54f442606d1774dc425
基礎(chǔ)模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
觸發(fā)詞語(yǔ)：dragon_real，flame

正向提示詞：

dragon_real,(extremely detailed CG unity 8k wallpaper,masterpiece, best quality, 
ultra-detailed, beautiful detailed eyes:1.2),best illumination, (best shadow, an extremely delicate and beautiful, bloom),
the girl was surrounded by dragons,.(1girl:1.4),breasts, solo,full body,Back tattoo,white dress,(color dragon:1.4),
Bare leg,long dress, 

.
反向提示詞：

nude, (badhandv4:1.2), (easynegative), verybadimagenegative_v1.3, deformation), blurry, poorly dissected, disfigured,poorly drawn face, mutation, mutation, (extra_limb), (ugly), (poorly drawn hand), finger fusion, messy drawing, 
broken leg examination, examination, examination stick, multiple breasts, (mutated hand and finger: 1.5), (long body: 1.3), 
(mutated, poorly drawn: 1.2), black and white, poorly dissected, liquid body, disfigured Deformed, mutated, anatomical nonsense, 
text font ui, errors, malformed hands, long necks, blurry, lowered, anatomically poor, proportion not good, shadows not good, 
body not coordinated, body not natural, fused breasts, poor breasts, huge breasts, poorly drawn breasts, excess breasts, 
liquid breasts, heavy breasts, missing breasts, large buttocks, huge thighs, Huge calves, bad hands, fused hands, missing hands, 
missing arms, missing thighs, missing calves, missing legs, fused arms, bad ears, poorly drawn ears, excess ears, liquid ears, 
heavy ears, missing ears, text, ui, errors, missing fingers, missing limbs, fused fingers, one hand with more than 5 fingers, 
one hand with less than 5 fingers, The other hand has more than 5 digits, the other hand is missing less than 5 digits, 
excess digits, fewer digits, fused digits, missing digits, bad digits, liquid digits, cropped, blurry, JPEG artifacts, signatures, 
deformed feet, excess feet, bad feet, bad drawings, fused feet, lost feet, excess shoes, bad shoes, fused shoes,two or more pairs of shoes, poorly drawn shoes Bad gloves, poorly drawn gloves, fused gloves, bad hair, poorly drawn hair, 
fused hair, big muscles, ugly, bad face, fused face, poorly drawn face, cloned face, big face, long face, bad eyes, 
fused eyes well drawn eyes, excess eyes, deformed limbs, two or more nipples, rough ratio. Short arm, (((missing arm)), 
thigh missing, calf missing, leg missing, mutation, duplication, deformity, poorly drawn hand, more than one left hand, 
less than one right hand, deformation, (blurring), disfigurement, missing leg, excess arm, excess thigh, two or more thighs, 
excess calf, fused calf, excess leg, poor knee, excess knee, more than two legs, Bad mouth, fused mouth, bad mouth, 
bad tongue, tongue in mouth, too long tongue, black tongue, big mouth, cracked mouth, bad mouth, dirty face, 
dirty teeth, bad underpants, fused cloth, bad cloth, bad underpants, yellow teeth, thick lips, bad camel toe, 
colorful camel toe, bad crotch seam, missing clip, bad clip, fused clip, color clip, black clip, liquid clip, QR code,barcode, reviewed, secure underwear, extra numbers, fewer numbers, cropped, worst quality, low quality, 
normal quality, jpeg artifacts, signature, watermark, username, blurry, out of focus, reviewed, ugly, old, deformed, 
amateur, strange, fat, deformed, black and white, realistic eyes,.

采樣：DPM++2MKarras
隨機(jī)種子：2656310671
CFGscale: 7
步數(shù)：55

生成結(jié)果：

6、korean doll likeness

下載鏈接：https://www.aigc.wtf/korean-doll-likeness-free-download/
成圖參考：https://zhuanlan.zhihu.com/p/606825889
基礎(chǔ)模型：chilloutmix_NiPrunedFp16Fix.safetensors
觸發(fā)詞語(yǔ)：

正向提示詞：

(RAW photo:1.2), (photorealistic:1.4),(masterpiece:1.3),(best quality:1.4),
nikon d750 f/1.4 55mm, detailed shadow, (realistic, photo-realistic:1.37),(sharp focus:1.2),(ISO high), (masterpiece,best quality), professional lighting, ultra high res,dreamlike,
(detailed eyes),(detailed facial features), (detailed clothes features),
skin tight, (shiny skin),(slender girl),(slim girl),
1girl, ((full body)),solo,
cute,smlie,
, (medium breasts), bangs, straight hair, long hair, black hair, black eyes, crystal pendant,
Long blue scarf,White long wool coat,Black thick pantyhose,Brown snow boots,
<lora:koreanDollLikeness_v15:1>

.
反向提示詞：

nsfw,EasyNegative,bad_prompt,ng_deepnegative_v1_75t,(worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, (outdoor:1.6), manboobs, backlight,(ugly:1.331), (duplicate:1.331), (morbid:1.21), (mutilated:1.21), (tranny:1.331), mutated hands, (poorly drawn hands:1.331), blurry, (bad anatomy:1.21), (bad proportions:1.331), extra limbs, (disfigured:1.331), (more than 2 nipples:1.331), (missing arms:1.331), (extra legs:1.331), (fused fingers:1.61051), (too many fingers:1.61051), (unclear eyes:1.331), bad hands, missing fingers, extra digit, (futa:1.1), bad body, pubic hair, glans,
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 546686100, Size: 720x960, Model hash: fc2511737a, Model: chilloutmix_NiPrunedFp32Fix (2), Clip skip: 2, ENSD: 31337

采樣：DPM++2MKarras
隨機(jī)種子：2514900086
CFGscale: 7
步數(shù)：22
生成結(jié)果：

7、Beautiful_Dress連衣裙

下載鏈接：https://www.liblibai.com/modelinfo/3aa33b228cda4a0980754ae921eb51d4
成圖參考：
基礎(chǔ)模型：chilloutmix_NiPrunedFp16Fix.safetensors
觸發(fā)詞語(yǔ)：dress
說(shuō)明：這個(gè)只會(huì)生成裙子，一般沒(méi)有人頭，最好和其它的配合使用

正向提示詞：

<lora:Beautiful_Dress:0.6>,(tightlacedressblackdeepvnecklayeredskirtankletdecorateholdflowersultryversatile:1.1),Girl,lightblue,(fullbody:1.2),(wholebody:1.2),(Lightweight:1.1),(Smile:1.1),Legs,Pure,Feet,(Lightmakeup:1.1),(45°LookUp:1.1),blackhair,Longhair,(whiteground:1.2),(Lightbackground:1.2),LongDress,(Stand.:1.2),Silkytexture,Cool,(Summer:1.1),(masterpiece:1,2),bestquality,masterpiece,highres,original,extremelydetailedwallpaper,perfectlighting,(extremelydetailedCG:1.2),drawing,paintbrush,Freedom,Romantic,mediumbreasts,

.
反向提示詞：

NSFW,(worst quality:2),(low quality:2),(normal quality:2),lowres,normal quality,((monochrome)),((grayscale)),skin spots,acnes,skin blemishes,age spot,(ugly:1.331),(duplicate:1.331),(morbid:1.21),(mutilated:1.21),(tranny:1.331),mutated hands,(poorly drawn hands:1.5),blurry,(bad anatomy:1.21),(bad proportions:1.331),extra limbs,(disfigured:1.331),(missing arms:1.331),(extra legs:1.331),(fused fingers:1.61051),(too many fingers:1.61051),(unclear eyes:1.331),lowers,bad hands,missing fingers,extra digit,bad hands,missing fingers,(((extra arms and legs))),

采樣：DPM++2MKarras
隨機(jī)種子：1172117708

CFGscale: 7
步數(shù)：22

生成結(jié)果：

8、qianlijiangshanXXMix-V1.0

下載鏈接：https://www.liblibai.com/modelinfo/2581675514bd453984cc0f544b8f6cfe
成圖參考：
基礎(chǔ)模型：SD 1.5
觸發(fā)詞語(yǔ)：1.用于風(fēng)景：no humans，mountain，outdoors，scenery，traditional media，landscape，river，birds，water，boat，moon 2.用于人物：1girl，solo，realistic，sky，hanfu，moon
說(shuō)明：

正向提示詞：

guofeng,illustration,1girl,full body,best quality,highly detailed,best quality,masterpiece,an extremely delicate and beautiful,extremely detailed,8k wallpaper,Amazing,finely detail,best quality,official art,extremely detailed CG unity 8k wallpaper,(style of Victo Ngai:0.5) <lora:qianlijiangshanXXMix-V1.0:0.65>, (moon:1.0),(river:1.2),(forest:1.0)(mountain :1.3)<lora:smokeanywhere:0.45>colorful smoke,

.
反向提示詞：

lowres,bad anatomy,bad hands,text,error,extra digit,fewer digits,cropped,worst quality,low quality,normal quality,jpeg artifacts,signature,watermark,username,two moons,six fingers

采樣：Euler a
隨機(jī)種子：329501653

CFGscale: 7
步數(shù)：25

總結(jié)

`以上就是今天要講的內(nèi)容。