目錄

一、keepalive基礎(chǔ)

1.1 作用

1.2 原理

1.3 功能

二、keepalive安裝

2.1 yum安裝

2.2 編譯安裝

三、配置文件

3.1?keepalived相關(guān)文件

3.2?主配置的組成

3.2.1?全局配置?

3.2.2 配置虛擬路由器

四、實(shí)際操作

4.1 lvs+keepalived高可用群集

4.2 keepalived+nginx

---

一、keepalive基礎(chǔ)

1.1 作用

用于解決系統(tǒng)的單點(diǎn)故障

1.2 原理

根據(jù)vrrp協(xié)議的工作原理，虛擬路由冗余協(xié)議

• 多個路由器組成一個路由器組，即虛擬路由器。
• 虛擬路由器使用虛擬IP與外網(wǎng)交互，虛擬MAC與內(nèi)網(wǎng)交互
• 獲得VIP的路由器為主路由器（Master狀態(tài)），其他路由器為備份路由器(Backup狀態(tài))
• 主路由器每隔 advertisement_intervl 秒（對應(yīng)圖中配置項(xiàng) advert_int），向組內(nèi)其他路由器發(fā)送通告消息（即心跳），告知本路由器的優(yōu)先級等信息；
• 組播的方式發(fā)送，組播地址224.0.0.18
• 只有主路由器會響應(yīng)ARP請求，組內(nèi)其他路由器會丟棄ARP請求；
• 只有主路由器會響應(yīng)對虛擬IP請求，組內(nèi)其他路由器會丟棄對虛擬IP的請求；

1.3 功能

- 基于vrrp協(xié)議完成地址流動
- 為vip地址所在的節(jié)點(diǎn)生成ipvs規(guī)則(在配置文件中預(yù)先定義)?
- 為ipvs集群的各RS做健康狀態(tài)檢測
- 基于腳本調(diào)用接口完成腳本中定義的功能，進(jìn)而影響集群事務(wù)，以此支持nginx、haproxy等服務(wù)

二、keepalive安裝

2.1 yum安裝

yum install -y keepalived

2.2 編譯安裝

yum install gcc curl openssl-devel libnl3-devel net-snmp-devel  -y
# 安裝依賴包環(huán)境https://keepalived.org/download.html
# 官網(wǎng)下載安裝包
wget https://keepalived.org/software/keepalived-2.2.2.tar.gztar  xf   keepalived-2.2.2.tar.gz
cd   keepalived-2.2.2/./configure --prefix=/usr/local/keepalived
make   &&  make  install自動生成service文件#編譯好后起不來原因是 沒有配置文件 /etc/keepalived/keepalived.confmkdir    /etc/keepalived 
cp /usr/local/keepalived/etc/keepalived/keepalived.conf  /etc/keepalived/keepalived.conf 需要修改配置文件的21行  網(wǎng)卡名稱， 否則起不來sed  -i   's/eth0/ens33/'   

三、配置文件

3.1?keepalived相關(guān)文件

• 軟件包名：keepalived

• 主程序文件：/usr/sbin/keepalived

• 主配置文件：/etc/keepalived/keepalived.conf

• 配置文件示例：/usr/share/doc/keepalived/

• Unit File：/lib/systemd/system/keepalived.service

• Unit File的環(huán)境配置文件：

• /etc/sysconfig/keepalived CentOS

3.2?主配置的組成

?/etc/keepalived/keepalived.conf配置組成

• GLOBAL CONFIGURATION

Global definitions（全局配置）：定義郵件配置，route_id ，vrrp配置，組播地址 等

• VRRP CONFIGURATION

VRRP instance(s)：定義vrrp協(xié)議中的每個vrrp虛擬路由器的規(guī)則，基本信息

• LVS CONFIGURATION（lvs調(diào)度服務(wù)器的規(guī)則設(shè)置）

Virtual server group(s)

Virtual server(s)：LVS集群的VS和RS

3.2.1?全局配置?

#/etc/keepalived/keepalived.conf 
global_defs {notification_email {root@localhost#keepalived 發(fā)生故障切換時郵件發(fā)送的目標(biāo)郵箱，可以按行區(qū)分寫多個root@localhost360601212@qq.com }notification_email_from keepalived@localhost ?#發(fā)郵件的地址smtp_server 127.0.0.1 ? ? #郵件服務(wù)器地址smtp_connect_timeout 30 ? #郵件服務(wù)器連接timeoutrouter_id   LVS01#每個keepalived主機(jī)唯一標(biāo)識，建議使用當(dāng)前主機(jī)名，但多節(jié)點(diǎn)重名不影響vrrp_skip_check_adv_addr ?#對所有通告報文都檢查，會比較消耗性能，啟用此配置后，如果收到的通告報文和上一個報文是同一個路由器，則跳過檢查，默認(rèn)值為全檢查
vrrp_strict 
#嚴(yán)格遵守VRRP協(xié)議,啟用此項(xiàng)后以下狀況將無法啟動服務(wù):1.無VIP地址 2.配置了單播鄰居 3.在VRRP版本2中有IPv6地址，開啟動此項(xiàng)并且沒有配置vrrp_iptables時會自動開啟iptables防火墻規(guī)則，默認(rèn)導(dǎo)致VIP無法訪問,建議不加此項(xiàng)配置。vrrp_garp_interval 0 #gratuitous ARP messages 免費(fèi)ARP報文發(fā)送延遲，0表示不延遲vrrp_gna_interval 0 #unsolicited NA messages （不請自來）消息發(fā)送延遲vrrp_mcast_group4 225.0.0.18#指定組播IP地址范圍：224.0.0.0到239.255.255.255,默認(rèn)值：224.0.0.18 vrrp_iptables ? ? ? ?#此項(xiàng)和vrrp_strict同時開啟時，則不會添加防火墻規(guī)則,如果無配置vrrp_strict項(xiàng),則無需啟用此項(xiàng)配置
}

3.2.2 配置虛擬路由器

vrrp_instance <STRING> {
#<String>為vrrp的實(shí)例名,一般為業(yè)務(wù)名稱配置參數(shù)......}
#配置參數(shù)：
state MASTER|BACKUP
#當(dāng)前節(jié)點(diǎn)在此虛擬路由器上的初始狀態(tài)，狀態(tài)為MASTER或者BACKUP
interface IFACE_NAME 
#綁定為當(dāng)前虛擬路由器使用的物理接口，如：eth0,bond0,br0,可以和VIP不在一個網(wǎng)卡
virtual_router_id VRID 
#每個虛擬路由器惟一標(biāo)識，范圍：0-255，每個虛擬路由器此值必須唯一，否則服務(wù)無法啟動，同屬一個虛擬路由器的多個keepalived節(jié)點(diǎn)必須相同,務(wù)必要確認(rèn)在同一網(wǎng)絡(luò)中此值必須唯一
priority 100 
#當(dāng)前物理節(jié)點(diǎn)在此虛擬路由器的優(yōu)先級，范圍：1-254，值越大優(yōu)先級越高,每個keepalived主機(jī)節(jié)點(diǎn)此值不同advert_int 1
#vrrp通告的時間間隔，默認(rèn)1sauthentication { 
#認(rèn)證機(jī)制auth_type AH|PASS ? #AH為IPSEC認(rèn)證(不推薦),PASS為簡單密碼(建議使用)auth_pass <PASSWORD> #預(yù)共享密鑰，僅前8位有效，同一個虛擬路由器的多個keepalived節(jié)點(diǎn)必須一樣
}
include    /etc/keealived/conf.d/*.conf
virtual_ipaddress { 
#虛擬IP,生產(chǎn)環(huán)境可能指定上百個IP地址<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>192.168.200.100 #指定VIP，不指定網(wǎng)卡，默認(rèn)為,注意：不指定/prefix,默認(rèn)為/32192.168.200.101/24 dev eth1 ? #指定VIP的網(wǎng)卡，建議和interface指令指定的崗卡不在一個網(wǎng)卡192.168.200.102/24 dev eth2 label eth2:1 #指定VIP的網(wǎng)卡label   ifconfig eth2:1   192.168.200.102/24
}
track_interface { 
#配置監(jiān)控網(wǎng)絡(luò)接口，一旦出現(xiàn)故障，則轉(zhuǎn)為FAULT狀態(tài)實(shí)現(xiàn)地址轉(zhuǎn)移eth0eth1…
}

?配置范例：

[root@localhost keepalived]#vim keepalived.conf
global_defs {notification_email {sysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1  #郵箱地址寫成自己smtp_connect_timeout 30router_id lvs_01   #修改成主機(jī)名vrrp_skip_check_adv_addr#vrrp_strict   注釋嚴(yán)格模式vrrp_garp_interval 0vrrp_gna_interval 0
}
include   /etc/keepalived/conf.d/*.conf  #添加子配置文件[root@localhost keepalived]#mkdir  conf.d
[root@localhost conf.d]# vim  conf.d/test.conf
vrrp_instance VI_1 {state MASTERinterface ens33   #修改網(wǎng)卡virtual_router_id 51  #路由器id 需要主從保持一致priority 100   advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.91.188/24   #設(shè)置虛擬ip}
}將配置文件復(fù)制到從節(jié)點(diǎn)
scp  keepalived.conf    192.168.91.11:/etc/keepalived/
scp  keepalived.conf    192.168.91.11:/etc/conf.d/test.conf修改配置文件
global_defs {notification_email {sysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1  smtp_connect_timeout 30router_id lvs_02  #修改成主機(jī)名vrrp_skip_check_adv_addr#vrrp_strict   注釋嚴(yán)格模式vrrp_garp_interval 0vrrp_gna_interval 0
}
include   /etc/keepalived/conf.d/*.conf  #添加子配置文件vrrp_instance VI_1 {state BACKUP       #修改類型為備節(jié)點(diǎn)interface ens33  virtual_router_id 51  #路由器id 需要主從保持一致   不用修改priority 80           #優(yōu)先級調(diào)小advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.91.188/24   #設(shè)置虛擬ip}
}

四、實(shí)際操作

4.1 lvs+keepalived高可用群集

環(huán)境配置

主keepalive：192.168.254.10? ? ? ? MASTER

從keepalive：192.168.254.20? ? ? ? SLAVE

web1：192.168.254.30

web2：192.168.254.40

vip：192.168.254.188

客戶：192.168.254.50

主keepalive配置

systemctl stop firewalld
setenforce 0yum install -y ipvsadm keepalived
#安裝lvs工具和keepalivedipvsadm-save > /etc/sysconfig/ipvsadm
vim /etc/keepalived/keepalived.confglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1#修改郵箱指向自己（10行）smtp_connect_timeout 30router_id LVS_01#指定服務(wù)器名稱主備需要不一樣（12行）vrrp_skip_check_adv_addr#vrrp_strict#14行需要注釋否則服務(wù)啟動有問題vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {state MASTER#指定服務(wù)器類型MASTER為主 BACKUP為備（20行）interface ens33#修改網(wǎng)卡名稱為ens33（21）virtual_router_id 10#指定虛擬路由器的ID號主備需要一致#nopreempt #非搶占模式兩個節(jié)點(diǎn)都需要配置去掉注釋priority 100#設(shè)定優(yōu)先級數(shù)字越大優(yōu)先級越高，準(zhǔn)備需要不一樣advert_int 1#通告間隔（查看是否存活）authentication {auth_type PASS#認(rèn)證類型auth_pass 123456#修改驗(yàn)證密碼，主備需要一樣（27行）}virtual_ipaddress {192.168.254.188#指定群集vip地址}
}
virtual_server 192.168.254.188 80 {delay_loop 6#健康間隔時間6秒lb_algo rr#調(diào)度算法輪詢lb_kind DR#lvs模式為DR persistence_timeout 0#連接保持時間改為0 否則 無法體現(xiàn)效果protocol TCP#采用協(xié)議real_server 192.168.254.30 80 {#43行修改地址為真實(shí)主機(jī)地址weight 1#45行刪除#節(jié)點(diǎn)權(quán)重TCP_CHECK {connect_port 80#檢查目標(biāo)端口connect_timeout 3#連接超時 nb_get_retry 3#重試次數(shù)delay_before_retry 3#重試間隔時間}}real_server 192.168.254.40 80 {#第二個weight 1TCP_CHECK{connect_port 80connect_timeout 3nb_get_retry 3delay_before_retry 3}}systemctl start keepalivedvim /etc/sysctl.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0 sysctl -p

從keepalive配置

systemctl stop firewalld
setenforce 0yum install -y ipvsadm keepalived
#安裝lvs工具和keepalivedipvsadm-save > /etc/sysconfig/ipvsadm
vim /etc/keepalived/keepalived.confglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1#修改郵箱指向自己（10行）smtp_connect_timeout 30router_id LVS_02#指定服務(wù)器名稱主備需要不一樣（12行）vrrp_skip_check_adv_addr#vrrp_strict#14行需要注釋否則服務(wù)啟動有問題vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {state BACKUP#指定服務(wù)器類型MASTER為主 BACKUP為備（20行）interface ens33#修改網(wǎng)卡名稱為ens33（21）virtual_router_id 10#指定虛擬路由器的ID號主備需要一致#nopreempt #非搶占模式兩個節(jié)點(diǎn)都需要配置去掉注釋priority 80#設(shè)定優(yōu)先級數(shù)字越大優(yōu)先級越高，準(zhǔn)備需要不一樣advert_int 1#通告間隔（查看是否存活）authentication {auth_type PASS#認(rèn)證類型auth_pass 123456#修改驗(yàn)證密碼，主備需要一樣（27行）}virtual_ipaddress {192.168.254.188#指定群集vip地址}
}
virtual_server 192.168.254.188 80 {delay_loop 6#健康間隔時間6秒lb_algo rr#調(diào)度算法輪詢lb_kind DR#lvs模式為DR persistence_timeout 0#連接保持時間改為0 否則 無法體現(xiàn)效果protocol TCP#采用協(xié)議real_server 192.168.254.30 80 {#43行修改地址為真實(shí)主機(jī)地址weight 1#45行刪除#節(jié)點(diǎn)權(quán)重TCP_CHECK {connect_port 80#檢查目標(biāo)端口connect_timeout 3#連接超時 nb_get_retry 3#重試次數(shù)delay_before_retry 3#重試間隔時間}}real_server 192.168.254.40 80 {#第二個weight 1TCP_CHECK{connect_port 80connect_timeout 3nb_get_retry 3delay_before_retry 3}}systemctl start keepalivedvim /etc/sysctl.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0 sysctl -p

web1：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo "web1" > /var/www/html/index.htmlifconfig lo:0 192.168.254.188 netmask 255.255.255.255vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2sysctl -proute add -host 192.168.254.188 dev lo:0

web2：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo "web2" > /var/www/html/index.htmlifconfig lo:0 192.168.254.188 netmask 255.255.255.255vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2sysctl -proute add -host 192.168.254.188 dev lo:0

客戶機(jī)：

curl 192.168.254.188web1
web2
web1
web2
...

4.2 keepalived+nginx

環(huán)境配置：

主keepalived+nginx：192.168.254.10

從keepalived+nginx：192.168.254.20

web1：192.168.254.30

web2：192.168.254.40

客戶機(jī)：192.168.254.50

?主keepalived+nginx

systemctl stop firewalld
setenforce 0yum install -y epel-release.noarch nginx keepalivedvim /etc/keepalived/keepalived.conf
global_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 178.168.200.1smtp_connect_timeout 30router_id LVS_01vrrp_skip_check_adv_addr#vrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_nginx {script "/etc/keepalived/nginx.sh"interval 1weight -30fall 3rise 2timeout 2
}vrrp_instance VI_1 {state MASTERinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.254.188}track_script {check_nginx
}
}vim /etc/keepalived/nginx.sh
#!/bin/bash
killall -0 nginx
if [ $(echo $0) == 1 ]
then echo "請開啟nginx"
else echo "已檢測nginx"
fivim /etc/nginx/nginx.conf
#在http模塊中，添加
upstream web {
server 192.168.254.30;
server 192.168.254.40;
}
#在server模塊中，添加
location / {
proxy_pass http://web;
}systemctl start keepalived nginx

從keepalived+nginx

systemctl stop firewalld
setenforce 0yum install -y epel-release.noarch nginx keepalivedvim /etc/keepalived/keepalived.conf
global_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 178.168.200.1smtp_connect_timeout 30router_id LVS_01vrrp_skip_check_adv_addr#vrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_nginx {script "/etc/keepalived/nginx.sh"interval 1weight -30fall 3rise 2timeout 2
}vrrp_instance VI_1 {state BACKUPinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.254.188}track_script {check_nginx
}
}vim /etc/keepalived/nginx.sh
#!/bin/bash
killall -0 nginx
if [ $(echo $0) == 1 ]
then echo "請開啟nginx"
else echo "已檢測nginx"
fivim /etc/nginx/nginx.conf
#在http模塊中，添加
upstream web {
server 192.168.254.30;
server 192.168.254.40;
}
#在server模塊中，添加
location / {
proxy_pass http://web;
}systemctl start keepalived nginx

web1 && web2

web1：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo web1 > /var/www/html/index.htmlsystemctl start httpd

?web2:

systemctl stop firewalld
setenforce 0yum install -y httpd
echo web2 > /var/www/html/index.htmlsystemctl start httpd

客戶機(jī)：

curl 192.168.254.188web1
web2
web1
web2
web1
web2
...