目前官方推薦有 2 種方式部署k8s apiserver 高可用

keepalived and haproxy

部署有2種方式，一種是systemd管理的，另一種是pod形式，使用那種可以根據(jù)實際情況選擇

服務(wù)部署

systemd方式

可以通過包管理工具安裝，正常啟動之后，通過 kubeadm init 安裝集群，配置好 apiserver 地址 VIP:${APISERVER_DEST_PORT}，配置參數(shù)見下面👇

pod方式

如果 keepalived 和 haproxy 將在控制平面節(jié)點上運行，則可以將它們配置作為靜態(tài) Pod 運行，這里所需要的只是在引導(dǎo)集群之前將相應(yīng)的清單文件放置在 /etc/kubernetes/manifests 目錄中，在引導(dǎo)過程中，kubelet 將啟動這些 pod，這種方式比較優(yōu)雅

keepalived

apiVersion: v1
kind: Pod
metadata:creationTimestamp: nullname: keepalivednamespace: kube-system
spec:containers:- image: osixia/keepalived:2.0.20name: keepalivedresources: {}securityContext:capabilities:add:- NET_ADMIN- NET_BROADCAST- NET_RAWvolumeMounts:- mountPath: /usr/local/etc/keepalived/keepalived.confname: config- mountPath: /etc/keepalived/check_apiserver.shname: checkhostNetwork: truevolumes:- hostPath:path: /etc/keepalived/keepalived.confname: config- hostPath:path: /etc/keepalived/check_apiserver.shname: check
status: {}

apiVersion: v1
kind: Pod
metadata:name: haproxynamespace: kube-system
spec:containers:- image: haproxy:2.8name: haproxylivenessProbe:failureThreshold: 8httpGet:host: localhostpath: /healthzport: ${APISERVER_DEST_PORT}scheme: HTTPSvolumeMounts:- mountPath: /usr/local/etc/haproxy/haproxy.cfgname: haproxyconfreadOnly: truehostNetwork: truevolumes:- hostPath:path: /etc/haproxy/haproxy.cfgtype: FileOrCreatename: haproxyconf
status: {}

${APISERVER_DEST_PORT} 值需要和配置文件一致，配置好后，執(zhí)行kubeadm init引導(dǎo)集群

服務(wù)配置

keepalived

keepalived配置包含服務(wù)配置文件和健康檢查文件，配置文件放在**/etc/keepalived**目錄，以下配置文件適用于2.0.20 and 2.2.4

! /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {router_id LVS_DEVEL
}
vrrp_script check_apiserver {script "/etc/keepalived/check_apiserver.sh"interval 3weight -2fall 10rise 2
}vrrp_instance VI_1 {state ${STATE}interface ${INTERFACE}virtual_router_id ${ROUTER_ID}priority ${PRIORITY}authentication {auth_type PASSauth_pass ${AUTH_PASS}}virtual_ipaddress {${APISERVER_VIP}}track_script {check_apiserver}
}

根據(jù)具體情況把上面配置文件變量替換成具體值

• ${STATE} 對于一臺主機來說是 MASTER，對于所有其他主機來說是 BACKUP，虛擬 IP 最初將分配給 MASTER。
• ${INTERFACE} VIP 需要綁定的網(wǎng)絡(luò)接口，例如eth0。
• ${ROUTER_ID} 應(yīng)該相同，但在同一子網(wǎng)中的所有集群中是唯一的。許多發(fā)行版將其值預(yù)先配置為 51，需要確認清楚
• ${PRIORITY} MASTER 節(jié)點應(yīng)高于BACKUP節(jié)點上的 ${PRIORITY}。因此 101 和 100 分別就足夠了。
  對于所有 keepalived 集群主機，
• ${AUTH_PASS} 驗證密碼，所有配置文件保持一致，例如123456
• ${APISERVER_VIP} 需要使用的 VIP 地址。

keepalived的健康檢查腳本放在/etc/keepalived/check_apiserver.sh。

#!/bin/sh
APISERVER_DEST_PORT=6443
errorExit() {echo "*** $*" 1>&2exit 1
}curl -sfk --max-time 2 https://localhost:${APISERVER_DEST_PORT}/healthz -o /dev/null || errorExit "Error GET https://localhost:${APISERVER_DEST_PORT}/healthz"

haproxy

配置文件目錄/etc/haproxy，適配版本 2.4 and 2.8

# /etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
globallog stdout format raw local0daemon#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaultsmode                    httplog                     globaloption                  httplogoption                  dontlognulloption http-server-closeoption forwardfor       except 127.0.0.0/8option                  redispatchretries                 1timeout http-request    10stimeout queue           20stimeout connect         5stimeout client          35stimeout server          35stimeout http-keep-alive 10stimeout check           10s#---------------------------------------------------------------------
# apiserver frontend which proxys to the control plane nodes
#---------------------------------------------------------------------
frontend apiserverbind *:${APISERVER_DEST_PORT}mode tcpoption tcplogdefault_backend apiserverbackend#---------------------------------------------------------------------
# round robin balancing for apiserver
#---------------------------------------------------------------------
backend apiserverbackendoption httpchkhttp-check connect sslhttp-check send meth GET uri /healthzhttp-check expect status 200mode tcpbalance     roundrobinserver ${HOST1_ID} ${HOST1_ADDRESS}:${APISERVER_SRC_PORT} check verify none# [...]

• ${APISERVER_DEST_PORT} haproxy 監(jiān)聽的端口，轉(zhuǎn)發(fā) apiserver 請求
• ${APISERVER_SRC_PORT} API Server 實例使用的端口
• ${HOST1_ID} 第一個負載平衡 API Server 主機名稱，隨意起，有多個可以再添加多個 server
• ${HOST1_ADDRESS} 第一個負載平衡 API Server 主機的可解析地址（DNS 名稱、IP 地址）

kube-vip

kube-vip官方文檔