高可用集群 KEEPALIVED

← 上一章：企业级反向代理 Haproxy ｜ 返回总目录 ｜ 下一章：企业级 NoSQL 数据库 Redis →

内容概述

• 高可用集群技术

• KeepAlived 安装

• KeepAlived 实现 VRRP

KeepAlived 实现 LVS 的高可用

KeepAlived 实现其它应用的高可用

1 高可用集群

1.1 集群类型

LB：Load Balance 负载均衡

LVS/HAProxy/nginx（http/upstream, stream/upstream）

HA：High Availability 高可用集群

数据库、Zookeeper、Redis

KeepAlived 通用的高可用集群

SPoF: Single Point of Failure，解决单点故障

HPC：High Performance Computing 高性能集群

https://www.top500.org

1.2 系统可用性

SLA：Service-Level Agreement 服务等级协议（提供服务的企业与客户之间就服务的品质、水准、性能等方面所达成的双方共同认可的协议或契约）

A = MTBF / (MTBF+MTTR）

99.95%:(60*24*30)*(1-0.9995)=21.6分钟  #一般按一个月停机时间统计

指标 ：99.9%, 99.99%, 99.999%,99.9999%

1.3 系统故障

硬件故障：设计缺陷、wear out（损耗）、自然灾害……

软件故障：设计缺陷 bug

1.4 实现高可用

提升系统高用性的解决方案：降低MTTR- Mean Time To Repair(平均故障时间)

解决方案：建立冗余机制

• active/passive 主/备

• active/active 双主 active --> HEARTBEAT --> passive active <--> HEARTBEAT <--> active

1.5 高可用相关技术

1.5.1 HA Service

资源：组成一个高可用服务的“组件”，比如：vip，service process，shared storage

• passive node的数量资源切换

1.5.2 Shared Storage

NAS(Network Attached Storage)：网络附加存储，基于网络的共享文件系统。 SAN(Storage Area Network)：存储区域网络，基于网络的块级别的共享

1.5.3 Network partition 网络分区

1.5.3.1 quorum 法定人数，仲裁

with quorum： > total/2

without quorum: <= total/2

1.5.3.2 隔离设备 fence

node：STONITH = Shooting The Other Node In The Head(强制下线/断电)

参考资料:

https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/7/html/high_availability _add-on_reference/s1-unfence-haar

1.5.4 双节点集群(TWO nodes Cluster)

辅助设备：仲裁设备，ping node, quorum disk

• Failover：故障切换，即某资源的主节点故障时，将资源转移至其它节点的操作

• Failback：故障移回，即某资源的主节点故障后重新修改上线后，将之前已转移至其它节点的资源重新切回的过程

1.5.5 HA Cluster 实现方案

1.5.5.1 AIS：Applicaiton Interface Specification 应用程序接口规范

RHCS：Red Hat Cluster Suite 红帽集群套件

参考资料：https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/5/html/ cluster_suite_overview/ch.gfscs.cluster-overview-cso

heartbeat：基于心跳监测实现服务高可用 pacemaker+corosync：资源管理与故障转移

1.5.5.2 VRRP：Virtual Router Redundancy Protocol

虚拟路由冗余协议,解决静态网关单点风险

物理层:路由器、三层交换机

软件层:keepalived

1.5.6 VRRP

1.5.6.1 VRRP 网络层硬件实现

参考链接：

https://support.huawei.com/enterprise/zh/doc/EDOC1000141382/19258d72/basic-
concepts-of-vrrp
https://wenku.baidu.com/view/dc0afaa6f524ccbff1218416.html
https://wenku.baidu.com/view/281ae109ba1aa8114431d9d0.html

1.5.6.2 VRRP 相关术语

• 虚拟路由器：Virtual Router

• 虚拟路由器标识：VRID(0-255)，唯一标识虚拟路由器

• VIP：Virtual IP

• VMAC：Virutal MAC (00-00-5e-00-01-VRID)

• 物理路由器：

master：主设备

backup：备用设备

priority：优先级

1.5.6.3 VRRP 相关技术

通告：心跳，优先级等；周期性

工作方式：抢占式，非抢占式

安全认证：

无认证

简单字符认证：预共享密钥

MD5

工作模式：

• 主/备：单虚拟路由器

• 主/主：主/备（虚拟路由器1），备/主（虚拟路由器2）

2 Keepalived 架构和安装

2.1 Keepalived 介绍

vrrp 协议的软件实现，原生设计目的为了高可用 ipvs服务

官网：http://keepalived.org/

功能：

• 基于vrrp协议完成地址流动

• 为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)

• 为ipvs集群的各RS做健康状态检测

• 基于脚本调用接口完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务

2.2 Keepalived 架构

官方文档：

https://keepalived.org/doc/
http://keepalived.org/documentation.html

用户空间核心组件：

vrrp stack：VIP消息通告

checkers：监测 Real Server

system call：实现 vrrp 协议状态转换时调用脚本的功能

SMTP：邮件组件

IPVS wrapper：生成 IPVS 规则

Netlink Reflector：网络接口

WatchDog：监控进程

控制组件：提供keepalived.conf 的解析器，完成Keepalived配置

IO复用器：针对网络目的而优化的自己的线程抽象

内存管理组件：为某些通用的内存管理功能（例如分配，重新分配，发布等）提供访问权限

Keepalived进程树

Keepalived  <-- Parent process monitoring children
\_ Keepalived   <-- VRRP child
\_ Keepalived   <-- Healthchecking child

2.3 Keepalived 环境准备

• 各节点时间必须同步：ntp, chrony

• 关闭防火墙及SELinux

• 各节点之间可通过主机名互相通信：非必须

• 建议使用/etc/hosts文件实现：非必须

• 各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信：非必须

2.4 Keepalived 相关文件

• 软件包名：keepalived

• 主程序文件：/usr/sbin/keepalived

• 主配置文件：/etc/keepalived/keepalived.conf

• 配置文件示例：/usr/share/doc/keepalived/

• Unit File：/lib/systemd/system/keepalived.service

• Unit File的环境配置文件：

  • /etc/sysconfig/keepalived CentOS

  • /etc/default/keepalived Ubuntu

注意：CentOS 7 上有 bug，可能有下面情况出现

systemctl restart keepalived                                #新配置可能无法生效
systemctl stop keepalived;systemctl start keepalived        #无法停止进程，需要 kill
停止

2.5 Keepalived 安装

2.5.1 包安装

#CentOS
[root@centos ~]#yum -y install keepalived

#ubuntu

[root@ubuntu1804 ~]#apt -y install keepalived

2.5.1.1 CentOS 安装 keepalived

[root@centos8 ~]#dnf -y install keepalived
[root@centos8 ~]#dnf info keepalived
Last metadata expiration check: 0:00:24 ago on Thu 26 Mar 202007:28:36 PM CST.
Installed Packages
Name         : keepalived
Version      : 2.0.10
Release      : 4.el8_0.2
Architecture : x86_64
Size         : 1.4 M
Source       : keepalived-2.0.10-4.el8_0.2.src.rpm
Repository   : @System
From repo    : AppStream
Summary      : High Availability monitor built upon LVS, VRRP and service
pollers
URL          : http://www.keepalived.org/
License      : GPLv2+
Description  : Keepalived provides simple and robust facilities for load
balancing
             : and high availability to Linux system and Linux based
infrastructures.
             : The load balancing framework relies on well-known and widely used
             : Linux Virtual Server (IPVS) kernel module providing Layer4 load
             : balancing. Keepalived implements a set of checkers to dynamically
and
             : adaptively maintain and manage load-balanced server pool
according
             : their health. High availability is achieved by VRRP protocol. VRRP
is
             : a fundamental brick for router failover. In addition, keepalived
             : implements a set of hooks to the VRRP finite state machine
providing
             : low-level and high-speed protocol interactions. Keepalived
frameworks
             : can be used independently or all together to provide resilient
             : infrastructures.
[root@centos8 ~]#systemctl start keepalived.service
[root@centos8 ~]#ps auxf |grep keepalived
root      128640.0  0.1  121081100 pts/0    S+19:25   0:00          |
\_ grep--color=auto keepalive
root      128350.0  0.3  914442484 ?        Ss   19:24   0:00
/usr/sbin/keepalived -D
root      128360.0  0.5  915764212 ?        S    19:24   0:00  \_
/usr/sbin/keepalived -D
root      128370.0  0.5  914444620 ?        S    19:24   0:00  \_
/usr/sbin/keepalived -D
[root@centos8 ~]#pstree -p
......
           ├─keepalived(12835)─┬─keepalived(12836)
           │                   └─keepalived(12837)
......

2.5.1.2 Ubuntu 安装 keepalived

范例：Ubuntu20.04

[root@ubuntu2004 ~]#apt list keepalived
Listing... Done
keepalived/focal-updates,now 1:2.0.19-2ubuntu0.2 amd64 [installed]
N: There are 2 additional versions. Please use the '-a' switch to see them.
[root@ubuntu2004 ~]#apt install -y keepalived
#默认没有配置文件无法启动
[root@ubuntu2004 ~]#systemctl status keepalived

#利用范例生成配置文件
[root@ubuntu2004 ~]#cp /usr/share/doc/keepalived/samples/keepalived.conf.sample
/etc/keepalived/keepalived.conf
[root@ubuntu2004 ~]#systemctl start keepalived
[root@ubuntu2004 ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP
group default qlen 1000
    link/ether 00:0c:29:0a:b2:20 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.100/24 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.11/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.12/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.13/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe0a:b220/64 scope link
       valid_lft forever preferred_lft forever
[root@ubuntu2004 ~]#pstree -p |grep keepalived
           |-keepalived(6260)-+-keepalived(6274)
           |                  `-keepalived(6275)
[root@ubuntu2004 ~]#ps auxf |grep keepalived
root        64600.0  0.0   6432720 pts/1    S+10:35   0:00  |       \_
grep--color=auto keepalived
root        62600.0  0.4  253728132 ?        Ss   10:29   0:00
/usr/sbin/keepalived --dont-fork
root        62740.0  0.1  253722812 ?        S    10:29   0:00  \_
/usr/sbin/keepalived --dont-fork
root        62750.0  0.1  253722916 ?        S    10:29   0:00  \_
/usr/sbin/keepalived --dont-fork

范例： Ubuntu18.04

[root@ubuntu1804 ~]#apt install keepalived -y
[root@ubuntu1804 ~]#dpkg -s keepalived
Package: keepalived
Status: install ok installed
Priority: extra
Section: admin
Installed-Size: 824
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Version: 1:1.3.9-1ubuntu0.18.04.2
Depends: iproute2, libc6 (>=2.27), libglib2.0-0 (>=2.26.0), libip4tc0 (>=
1.6.0+snapshot20161117), libip6tc0 (>=1.6.0+snapshot20161117), libnl-3-200 (>=
3.2.27), libnl-genl-3-200 (>=3.2.7), libnl-route-3-200 (>=3.2.7), libsnmp30
(>=5.7.3+dfsg-1.8ubuntu3.1~dfsg), libssl1.1 (>=1.1.0), libxtables12 (>=
1.6.0+snapshot20161117)
Recommends: ipvsadm
Conffiles:
 /etc/dbus-1/system.d/org.keepalived.Vrrp1.conf 6b020ff46c6425d3a9cfa179814d7253
 /etc/default/keepalived 6b2e3432e4ae31b444058ba2b0d1f06a
 /etc/init.d/keepalived 0312972e0718331b4c90b3b98e623624
Description: Failover and monitoring daemon for LVS clusters
 keepalived is used for monitoring real servers within a Linux
 Virtual Server (LVS) cluster.  keepalived can be configured to
 remove real servers from the cluster pool if it stops responding,
 as well as send a notification email to make the admin aware of
 the service failure.
 .
 In addition, keepalived implements an independent Virtual Router
 Redundancy Protocol (VRRPv2; see rfc2338 for additional info)
 framework for director failover.
 .
 You need a kernel >=2.4.28 or >=2.6.11 for keepalived.
 See README.Debian for more information.
Homepage: http://keepalived.org
Original-Maintainer: Alexander Wirt <formorer@debian.org>
[root@ubuntu1804 ~]#dpkg -L keepalived
/.
/etc
/etc/dbus-1
/etc/dbus-1/system.d
/etc/dbus-1/system.d/org.keepalived.Vrrp1.conf
/etc/default
/etc/default/keepalived
/etc/init.d
/etc/init.d/keepalived
/etc/keepalived
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/keepalived.service
/usr
/usr/bin
/usr/bin/genhash

/usr/sbin
/usr/sbin/keepalived
/usr/share
/usr/share/dbus-1
/usr/share/dbus-1/interfaces
/usr/share/dbus-1/interfaces/org.keepalived.Vrrp1.Instance.xml
/usr/share/dbus-1/interfaces/org.keepalived.Vrrp1.Vrrp.xml
/usr/share/doc
/usr/share/doc/keepalived
/usr/share/doc/keepalived/AUTHOR
/usr/share/doc/keepalived/CONTRIBUTORS
/usr/share/doc/keepalived/README
/usr/share/doc/keepalived/TODO
/usr/share/doc/keepalived/changelog.Debian.gz
/usr/share/doc/keepalived/copyright
/usr/share/doc/keepalived/keepalived.conf.SYNOPSIS.gz
/usr/share/doc/keepalived/samples
/usr/share/doc/keepalived/samples/client.pem
/usr/share/doc/keepalived/samples/dh1024.pem
/usr/share/doc/keepalived/samples/keepalived.conf.HTTP_GET.port
/usr/share/doc/keepalived/samples/keepalived.conf.IPv6
/usr/share/doc/keepalived/samples/keepalived.conf.SMTP_CHECK
/usr/share/doc/keepalived/samples/keepalived.conf.SSL_GET
/usr/share/doc/keepalived/samples/keepalived.conf.fwmark
/usr/share/doc/keepalived/samples/keepalived.conf.inhibit
/usr/share/doc/keepalived/samples/keepalived.conf.misc_check
/usr/share/doc/keepalived/samples/keepalived.conf.misc_check_arg
/usr/share/doc/keepalived/samples/keepalived.conf.quorum
/usr/share/doc/keepalived/samples/keepalived.conf.sample
/usr/share/doc/keepalived/samples/keepalived.conf.status_code
/usr/share/doc/keepalived/samples/keepalived.conf.track_interface
/usr/share/doc/keepalived/samples/keepalived.conf.virtual_server_group
/usr/share/doc/keepalived/samples/keepalived.conf.virtualhost
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.localcheck
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.lvs_syncd
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.routes
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.rules
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.scripts
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.static_ipaddress
/usr/share/doc/keepalived/samples/keepalived.conf.vrrp.sync
/usr/share/doc/keepalived/samples/root.pem
/usr/share/doc/keepalived/samples/sample.misccheck.smbcheck.sh
/usr/share/doc/keepalived/samples/sample_notify_fifo.sh
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/genhash.1.gz
/usr/share/man/man5
/usr/share/man/man5/keepalived.conf.5.gz
/usr/share/man/man8
/usr/share/man/man8/keepalived.8.gz
/usr/share/snmp
/usr/share/snmp/mibs
/usr/share/snmp/mibs/KEEPALIVED-MIB.txt
/usr/share/snmp/mibs/VRRP-MIB.txt
/usr/share/snmp/mibs/VRRPv3-MIB.txt
[root@ubuntu1804 ~]#cp /usr/share/doc/keepalived/samples/keepalived.conf.sample
/etc/keepalived/keepalived.conf

[root@ubuntu1804 ~]#systemctl start keepalived.service
[root@ubuntu1804 ~]#systemctl status keepalived.service
● keepalived.service - Keepalive Daemon (LVS and VRRP)
   Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor
preset: enabled)
   Active: active (running) since Thu 2020-03-2619:33:48 CST; 1min 9s ago
  Process: 3208ExecStart=/usr/sbin/keepalived $DAEMON_ARGS(code=exited,
status=0/SUCCESS)
 Main PID: 3209 (keepalived)
    Tasks: 3 (limit: 1084)
   CGroup: /system.slice/keepalived.service
           ├─3209 /usr/sbin/keepalived
           ├─3210 /usr/sbin/keepalived
           └─3211 /usr/sbin/keepalived

Mar 2619:34:04 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Timeout
connecting server [192.168.200.2]:tcp:1358.
Mar 2619:34:10 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Timeout
connecting server [192.168.200.2]:tcp:1358.
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Timeout
connecting server [192.168.200.2]:tcp:1358.
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Check on
service [192.168.200.2]:tcp:1358 failed after 3 retry.
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Removing
service [192.168.200.2]:tcp:1358 to VS [10.10.10.2]:tc
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Lost quorum
1-0=1 > 0for VS [10.10.10.2]:tcp:1358
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Adding
sorry server [192.168.200.200]:tcp:1358 to VS [10.10.10.
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Removing
alive servers from the pool for VS [10.10.10.2]:tcp:13
Mar 2619:34:16 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Remote SMTP
server [192.168.200.1]:25 connected.

Mar 2619:34:37 ubuntu1804.wang.org Keepalived_healthcheckers[3210]: Error
reading data from remote SMTP server [192.168.200.1]:25.
[root@ubuntu1804 ~]#ps auxf |grep keepalived
root       32240.0  0.1  144281040 pts/0    S+19:34   0:00      |   \_
grep--color=auto keepalived
root       32090.0  0.3  918122996 ?        Ss   19:33   0:00
/usr/sbin/keepalived
root       32100.0  0.5  961005276 ?        S    19:33   0:00  \_
/usr/sbin/keepalived
root       32110.0  0.5  961525420 ?        S    19:33   0:00  \_
/usr/sbin/keepalived

2.5.2 编译安装

#Ubuntu20.04安装相关包
[root@ubuntu2004 ~]#apt update

[root@ubuntu2004 ~]#apt -y install make gcc ipvsadm build-essential pkg-config
automake autoconf libipset-dev libnl-3-dev libnl-genl-3-dev libssl-dev
libxtables-dev libip4tc-dev libip6tc-dev libipset-dev libmagic-dev libsnmp-dev
libglib2.0-dev libpcre2-dev libnftnl-dev libmnl-dev libsystemd-dev

#Ubuntu18.04安装相关包

[root@ubuntu1804 ~]#apt update

[root@ubuntu1804 ~]#apt -y install gcc curl openssl libssl-dev libpopt-dev
daemon build-essential

#红帽系统安装相关包

[root@centos7 ~]#yum install  gcc curl openssl-devel libnl3-devel net-snmp-devel

#下载解压

[root@centos7 ~]#wget https://keepalived.org/software/keepalived-2.0.20.tar.gz
[root@centos7 ~]#tar xvf keepalived-2.0.20.tar.gz  -C /usr/local/src

[root@centos7 ~]#cd /usr/local/src/keepalived-2.0.20/

#选项--disable-fwmark 可用于禁用iptables规则,可防止VIP无法访问,无此选项默认会启用iptables
规则
[root@centos7 keepalived-2.0.20]#./configure --prefix=/usr/local/keepalived #--
disable-fwmark
[root@centos7 keepalived-2.0.20]#make && make install

[root@centos7 keepalived-2.0.20]#cd
[root@centos7 ~]#/usr/local/keepalived/sbin/keepalived -v
Keepalived v2.0.20 (01/22,2020)

Copyright(C) 2001-2020 Alexandre Cassen, <acassen@gmail.com>
Built with kernel headers for Linux 3.10.0
Running on Linux 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019
configure options: --prefix=/usr/local/keepalived

Config options:  LVS VRRP VRRP_AUTH OLD_CHKSUM_COMPAT FIB_ROUTING

System options:  PIPE2 SIGNALFD INOTIFY_INIT1 VSYSLOG EPOLL_CREATE1
IPV6_ADVANCED_API LIBNL3 RTA_ENCAP RTA_EXPIRES RTA_PREF FRA_SUPPRESS_PREFIXLEN
FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK FRA_OIFNAME IFA_FLAGS IP_MULTICAST_ALL
NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK
VRRP_VMAC IFLA_LINK_NETNSID CN_PROC SOCK_NONBLOCK SOCK_CLOEXEC O_PATH GLOB_BRACE
INET6_ADDR_GEN_MODE SO_MARK SCHED_RESET_ON_FORK

#默认会自动生成unit文件
[root@centos7 ~]#cp ./keepalived/keepalived.service /lib/systemd/system/
[root@centos7 ~]#cat /usr/lib/systemd/system/keepalived.service
[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/run/keepalived.pid
KillMode=process
EnvironmentFile=-/usr/local/keepalived/etc/sysconfig/keepalived
ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP$MAINPID

[Install]
WantedBy=multi-user.target

[root@centos7 ~]#cat /usr/local/keepalived/etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp               -P    Only run with VRRP subsystem.
# --check              -C    Only run with Health-checker subsystem.
# --dont-release-vrrp  -V    Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs  -I    Dont remove IPVS topology on daemon stop.
# --dump-conf          -d    Dump the configuration data.
# --log-detail         -D    Detailed log messages.
# --log-facility       -S    0-7 Set local syslog facility (default=LOG_DAEMON)
#

KEEPALIVED_OPTIONS="-D"

#默认无法启动
[root@centos7 ~]#systemctl start keepalived.service
Job for keepalived.service failed because the control process exited with error
code. See "systemctl status keepalived.service" and "journalctl -xe"for
details.

#查看日志，可以看到是因为缺少配置文件导致无法启动
[root@centos7 ~]#journalctl -xe
-- Subject: Unit keepalived.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit keepalived.service has begun starting up.
Mar 2900:38:17 centos7.wang.org Keepalived[1123]: Starting Keepalived v2.0.20
(01/22,2020)
Mar 2900:38:17 centos7.wang.org Keepalived[1123]: Running on Linux 3.10.0-
1062.el7.x86_64 #1 SMP Wed Aug 7
Mar 2900:38:17 centos7.wang.org Keepalived[1123]: Command line:
'/usr/local/keepalived/sbin/keepalived''-D
Mar 2900:38:17 centos7.wang.org Keepalived[1123]: Unable to find configuration
file /etc/keepalived/keepali   #默认配置文件路径
Mar 2900:38:17 centos7.wang.org Keepalived[1123]: Stopped Keepalived v2.0.20
(01/22,2020)
Mar 2900:38:17 centos7.wang.org systemd[1]: keepalived.service: control process
exited, code=exited status=
Mar 2900:38:17 centos7.wang.org systemd[1]: Failed to start LVS and VRRP High
Availability Monitor.
-- Subject: Unit keepalived.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit keepalived.service has failed.
--
-- The result is failed.
Mar 2900:38:17 centos7.wang.org systemd[1]: Unit keepalived.service entered
failed state.
Mar 2900:38:17 centos7.wang.org systemd[1]: keepalived.service failed.
Mar 2900:38:17 centos7.wang.org polkitd[565]: Unregistered Authentication Agent
for unix-process:1117:11546

#创建配置文件

[root@centos7 ~]#mkdir /etc/keepalived

[root@centos7 ~]#cp /usr/local/keepalived/etc/keepalived/keepalived.conf.sample
/etc/keepalived/keepalived.conf

# 再次启动成功

[root@centos7 ~]#systemctl enable --now  keepalived.service
Created symlink from /etc/systemd/system/multi-
user.target.wants/keepalived.service to
/usr/lib/systemd/system/keepalived.service.

[root@centos7 ~]#systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor
preset: disabled)
   Active: active (running) since Sun 2020-03-2900:44:33 CST; 4s ago
  Process: 1191ExecStart=/usr/local/keepalived/sbin/keepalived
$KEEPALIVED_OPTIONS(code=exited, status=0/SUCCESS)
   CGroup: /system.slice/keepalived.service
           ├─1192 /usr/local/keepalived/sbin/keepalived -D
           ├─1193 /usr/local/keepalived/sbin/keepalived -D
           └─1194 /usr/local/keepalived/sbin/keepalived -D

Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.18
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.16
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.17
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.18

Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.16
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.17
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.18
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.16
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.17
Mar 2900:44:37 centos7.wang.org Keepalived_vrrp[1194]: Sending gratuitous ARP
on eth0 for192.168.200.18
[root@centos7 ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
group default qlen 1000
    link/ether 00:0c:29:32:80:38 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.7/24 brd 10.0.0.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.16/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.17/32 scope global eth0

       valid_lft forever preferred_lft forever
    inet 192.168.200.18/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe32:8038/64 scope link
       valid_lft forever preferred_lft forever

[root@centos7 ~]#hostname -I
10.0.0.7 192.168.200.16 192.168.200.17 192.168.200.18
[root@centos7 ~]#ping 192.168.200.16
PING 192.168.200.16 (192.168.200.16) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
^C
---192.168.200.16 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1000ms

#默认生成iptables规则,无法访问VIP,编译时可以加--disable-fwmark禁用生成iptables规则
[root@centos7 ~]#iptables -vnL
Chain INPUT (policy ACCEPT 860 packets, 46129 bytes)
 pkts bytes target     prot opt in     out     source               destination
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.18
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.17
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.16

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 1737 packets, 1188K bytes)
 pkts bytes target     prot opt in     out     source               destination
4336 DROP       all  --  *      *       192.168.200.18       0.0.0.0/0
00 DROP       all  --  *      *       192.168.200.17       0.0.0.0/0
00 DROP       all  --  *      *       192.168.200.16       0.0.0.0/0

[root@centos7 ~]#vim /etc/keepalived/keepalived.conf
#注释下面一行
#vrrp_strict
#重启动不生效，有bug
[root@centos7 ~]#systemctl restart keepalived.service
[root@centos7 ~]#ping 192.168.200.16
PING 192.168.200.16 (192.168.200.16) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
^C
---192.168.200.16 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
[root@centos7 ~]#iptables -vnL
Chain INPUT (policy ACCEPT 1219 packets, 67647 bytes)

 pkts bytes target     prot opt in     out     source               destination
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.18
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.17
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.16
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
Chain OUTPUT (policy ACCEPT 2282 packets, 1233K bytes)
 pkts bytes target     prot opt in     out     source               destination
4336 DROP       all  --  *      *       192.168.200.18       0.0.0.0/0
00 DROP       all  --  *      *       192.168.200.17       0.0.0.0/0
4336 DROP       all  --  *      *       192.168.200.16       0.0.0.0/0

#无法关闭进程
[root@centos7 ~]#systemctl stop keepalived.service
[root@centos7 ~]#ps aux|grep keepalived
root       13830.0  0.1  696721020 ?        Ss   00:57   0:00
/usr/local/keepalived/sbin/keepalived -D
root       13840.0  0.2  698042308 ?        S    00:57   0:00
/usr/local/keepalived/sbin/keepalived -D
root       13850.0  0.1  696721308 ?        S    00:57   0:00
/usr/local/keepalived/sbin/keepalived -D
root       13920.0  0.0 112712964 pts/0    R+00:59   0:00 grep--
color=auto keepalived
[root@centos7 ~]#killall  keepalived
[root@centos7 ~]#systemctl start keepalived.service
[root@centos7 ~]#ping 192.168.200.16
PING 192.168.200.16 (192.168.200.16) 56(84) bytes of data.
64 bytes from 192.168.200.16: icmp_seq=1ttl=64time=0.093 ms
^C
---192.168.200.16 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev =0.093/0.093/0.093/0.000 ms
[root@centos7 ~]#iptables -vnL
Chain INPUT (policy ACCEPT 125 packets, 8493 bytes)
 pkts bytes target     prot opt in     out     source               destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
Chain OUTPUT (policy ACCEPT 135 packets, 20190 bytes)
 pkts bytes target     prot opt in     out     source               destination

2.6 KeepAlived 配置说明

2.6.1 配置文件组成部分

配置文件

/etc/keepalived/keepalived.conf

配置文件组成

• GLOBAL CONFIGURATION

Global definitions：定义邮件配置，route_id，vrrp配置，多播地址等

VRRP CONFIGURATION

VRRP instance(s)：定义每个vrrp虚拟路由器

LVS CONFIGURATION

Virtual server group(s)

Virtual server(s)：LVS集群的VS和RS

2.6.2 配置语法说明

帮助

man keepalived.conf

2.6.2.1 全局配置

#/etc/keepalived/keepalived.conf
global_defs {
  notification_email {
  root@localhost             #keepalived 发生故障切换时邮件发送的目标邮箱，可以按行区分写
多个
  root@wangxiaochun.com
29308620@qq.com
  }
  notification_email_from keepalived@localhost  #发邮件的地址
  smtp_server 127.0.0.1     #邮件服务器地址
  smtp_connect_timeout 30#邮件服务器连接timeout
  router_id ka1.example.com #每个keepalived主机唯一标识，建议使用当前主机名，如果多节点重
名可能会影响切换脚本执行

  vrrp_skip_check_adv_addr #对所有通告报文都检查，会比较消耗性能，启用此配置后，如果收到的
通告报文和上一个报文是同一个路由器，则跳过检查，默认值为全检查

  vrrp_strict #严格遵守VRRP协议,启用此项后以下状况将无法启动服务:1.无VIP地址 2.配置了单播邻
居 3.在VRRP版本2中有IPv6地址，开启动此项并且没有配置vrrp_iptables时会自动开启iptables防火
墙规则，默认导致VIP无法访问,建议不加此项配置

  vrrp_garp_interval 0#gratuitous ARP messages 报文发送延迟，0表示不延迟
  vrrp_gna_interval 0#unsolicited NA messages （不请自来）消息发送延迟
  vrrp_mcast_group4 224.0.0.18 #指定组播IP地址范围：224.0.0.0到239.255.255.255,默认
值：224.0.0.18

  vrrp_iptables   #此项和vrrp_strict同时开启时，则不会添加防火墙规则,如果无配置
vrrp_strict项,则无需启用此项配置，注意：新版加此项仍有iptables规则
}

2.6.2.2 配置虚拟路由器

vrrp_instance <STRING> { #<String>为vrrp的实例名,一般为业务名称
配置参数
    ......
 }
#配置参数：
state MASTER|BACKUP#当前节点在此虚拟路由器上的初始状态，状态为MASTER或者BACKUP
interface IFACE_NAME #绑定为当前虚拟路由器使用的物理接口，如：eth0,bond0,br0,可以和VIP不
在一个网卡

virtual_router_id VRID #每个虚拟路由器唯一标识，范围：0-255，每个虚拟路由器此值必须唯一，否
则服务无法启动，同属一个虚拟路由器的多个keepalived节点必须相同,务必要确认在同一网络中此值必须唯
一
priority 100#当前物理节点在此虚拟路由器的优先级，范围：1-254，每个keepalived主机节点此
值不同
advert_int 1#vrrp通告的时间间隔，默认1s
authentication { #认证机制
    auth_type AH|PASS   #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)
    auth_pass <PASSWORD> #预共享密钥，仅前8位有效，同一个虚拟路由器的多个keepalived节点必
须一样
}
virtual_ipaddress { #虚拟IP,生产环境可能指定几十上百个VIP地址
    <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
192.168.200.100         #指定VIP，不指定网卡，默认为eth0,注意：不指定/prefix,默认
为/32
192.168.200.101/24 dev eth1   #指定VIP的网卡，建议和interface指令指定的网卡不在一个
网卡
192.168.200.102/24 dev eth2 label eth2:1    #指定VIP的网卡label
}
track_interface { #配置监控网络接口，一旦出现故障，则转为FAULT状态实现地址转移
    eth0
    eth1
    …
}

范例：

[root@centos7 ~]#cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict                  #开启限制，会自动生效防火墙设置，导致无访问VIP

   vrrp_garp_interval 0
   vrrp_gna_interval 0
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 80#修改此行
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
192.168.200.16
192.168.200.17
192.168.200.18
    }
}
[root@centos7 ~]#systemctl start keepalived.service
[root@centos7 ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
group default qlen 1000
    link/ether 00:0c:29:33:b4:1a brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.17/24 brd 10.0.0.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.16/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.17/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.18/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe33:b41a/64 scope link
       valid_lft forever preferred_lft forever

[root@centos7 ~]#iptables -vnL
Chain INPUT (policy ACCEPT 59 packets, 3372 bytes)
 pkts bytes target     prot opt in     out     source               destination
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.16
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.17
00 DROP       all  --  *      *       0.0.0.0/0
192.168.200.18

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 33 packets, 6940 bytes)
 pkts bytes target     prot opt in     out     source               destination
[root@centos7 ~]#ping 192.168.200.16
PING 192.168.200.16 (192.168.200.16) 56(84) bytes of data.
^C
---192.168.200.16 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5002ms
[root@centos7 ~]#
# 如果是CentOS 8 ，会显示以下warning
[root@centos8 ~]#iptables -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
# Warning: iptables-legacy tables present, use iptables-legacy to see them
#无法访问VIP
[root@centos8 ~]#ping 192.168.200.16
PING 192.168.200.16 (192.168.200.16) 56(84) bytes of data.
^C
---192.168.200.16 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 143ms

2.7 启用 Keepalived 日志功能

范例：

[root@ka1 ~]#vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]#vim /etc/rsyslog.conf
local6.*                                                /var/log/keepalived.log
[root@ka1 ~]#systemctl restart keepalived.service rsyslog.service
[root@ka1 ~]#tail -f /var/log/keepalived.log
Apr 1409:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10
Apr 1409:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10
Apr 1409:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: (VI_1) Sending/queueing gratuitous
ARPs on eth0 for10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

Apr 1409:25:56 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

2.8 实现 Keepalived 独立子配置文件

当生产环境复杂时， /etc/keepalived/keepalived.conf 文件中保存所有集群的配置会导致内容过多，不易管理

可以将不同集群的配置，比如：不同集群的VIP配置放在独立的子配置文件中利用include 指令可以实现包含子配置文件

格式:

include /path/file

范例:

[root@ka1-centos8 ~]#mkdir /etc/keepalived/conf.d/
[root@ka1-centos8 ~]#vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
29308620@qq.com
   }
   notification_email_from 29308620@qq.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1.wang.org
   vrrp_skip_check_adv_addr
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}
include /etc/keepalived/conf.d/*.conf   #将VRRP相关配置放在子配置文件中
[root@ka1-centos8 ~]#vim /etc/keepalived/conf.d/cluster1.conf

3 Keepalived 实现 VRRP

3.1 实现master/slave的 Keepalived 单主架构

3.1.1 MASTER配置

[root@ka1-centos8 ~]#vim /etc/keepalived/keepalived.conf
global_defs {
  notification_email {
  root@localhost #keepalived 发生故障切换时邮件发送的对象，可以按行区分写多个
  }
  notification_email_from keepalived@localhost
  smtp_server 127.0.0.1
  smtp_connect_timeout 30
  router_id ka1.example.com
  vrrp_skip_check_adv_addr #所有报文都检查比较消耗性能，此配置为如果收到的报文和上一个报文
是同一个路由器则跳过检查报文中的源地址
#vrrp_strict #严格遵守VRRP协议,禁止状况:1.无VIP地址,2.配置了单播邻居,3.在VRRP版本2中有
IPv6地址
  vrrp_garp_interval 0#ARP报文发送延迟
  vrrp_gna_interval 0#消息发送延迟
  vrrp_mcast_group4 224.0.0.18 #默认组播IP地址，可指定组播范围：224.0.0.0到
239.255.255.255
 }
vrrp_instance VI_1 {
  state MASTER           #在另一个结点上为BACKUP
  interface eth0
  virtual_router_id 66#每个虚拟路由器必须唯一，同属一个虚拟路由器的多个keepalived节点必
须相同
  priority 100#在另一个结点上为80
  advert_int 1
  authentication {
    auth_type PASS       #预共享密钥认证，同一个虚拟路由器的keepalived节点必须一样
    auth_pass 12345678
  }
  virtual_ipaddress {
10.0.0.10 dev eth0 label eth0:0
  }
}

3.1.2 BACKUP配置

#配置文件和master基本一致，只需修改三行
[root@ka2-centos8 ~]#vim /etc/keepalived/keepalived.conf
global_defs {
  notification_email {
  root@localhost
  }
  notification_email_from keepalived@localhost
  smtp_server 127.0.0.1
  smtp_connect_timeout 30
  router_id ka2.example.com         #修改此行
  vrrp_skip_check_adv_addr
#vrrp_strict
  vrrp_garp_interval 0
  vrrp_gna_interval 0
  vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
  state BACKUP               #修改此行

  interface eth0
  virtual_router_id 66
  priority 80#修改此行
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass 12345678
  }
  virtual_ipaddress {
10.0.0.10 dev eth0 label eth0:0
  }
}

抓包观察

tcpdump -i eth0 -nn host 224.0.0.18

3.1.3 脑裂

主备节点同时拥有VIP，此时为脑裂理象

注意：脑裂现象原因

心跳线故障

防火墙错误配置 Keepalived 配置错误

范例: 发现脑裂

[root@centos7 ~]#arping  -I eth1 -c1 10.0.0.100
ARPING 10.0.0.100 from 192.168.10.130 eth1
Unicast reply from 10.0.0.100 [00:0C:29:7E:DA:E6]  0.801ms
Unicast reply from 10.0.0.100 [00:0C:29:97:AF:4F]  0.814ms
Sent 1 probes (1 broadcast(s))
Received 2 response(s)

3.2 抢占模式和非抢占模式

3.2.1 非抢占模式 nopreempt

默认为抢占模式 preempt，即当高优先级的主机恢复在线后，会抢占低先级的主机的master角色，造成网络抖动，建议设置为非抢占模式 nopreempt ，即高优先级主机恢复后，并不会抢占低优先级主机的 master 角色

注意: 非抢占模式下,如果原主机down机, VIP迁移至的新主机, 后续新主机也发生down时,VIP还会迁移回修复好的原主机

但如果新主机的服务down掉（keepalived服务正常），原主机也不会接管VIP，仍会由新主机拥有VIP 即非抢占式模式，只是适合当主节点宕机，切换到从节点的一次性的高可用性，后续即使当原主节点修复好，仍无法再次起到高用功能

注意：要关闭 VIP抢占，必须将各 Keepalived 服务器 state 配置为 BACKUP

#ha1主机配置

vrrp_instance VI_1 {
  state BACKUP              #都为BACKUP
  interface eth0
  virtual_router_id 66
  priority 100#优先级高
  advert_int 1
  nopreempt                 #添加此行，设为nopreempt
#ha2主机配置
vrrp_instance VI_1 {
  state BACKUP              #都为BACKUP
  interface eth0
  virtual_router_id 66
  priority 80#优先级低
  advert_int 1
#nopreempt                #注意：如果ka2主机也是非抢占式，会导致ka1即使优先级降低于
ka2，VIP也不会切换至ka2

3.2.2 抢占延迟模式 preempt_delay

抢占延迟模式，即优先级高的主机恢复后，不会立即抢回VIP，而是延迟一段时间（默认300s）再抢回 VIP

preempt_delay   #      #指定抢占延迟时间为#s，默认延迟300s

注意：需要各keepalived服务器state为BACKUP,并且不要启用 vrrp_strict

范例：

#ka1主机配置
vrrp_instance VI_1 {
  state BACKUP                      #都为BACKUP
  interface eth0
  virtual_router_id 66
  priority 100#优先级高
  advert_int 1
  preempt_delay 60#抢占延迟模式，默认延迟300s
#ka2主机配置
vrrp_instance VI_1 {
  state BACKUP                      #都为BACKUP
  interface eth0
  virtual_router_id 66
  priority 80#优先级低
  advert_int 1

3.3 VIP 单播配置

默认keepalived主机之间利用多播相互通告消息，会造成网络拥塞，可以替换成单播，减少网络流量另外：有些公有云不支持多播，可以单播实现

注意：启用 vrrp_strict 时，不能启用单播

#在所有节点vrrp_instance语句块中设置对方主机的IP，建议设置为专用于对应心跳线网络的地址，而非使
用业务网络
unicast_src_ip <IPADDR>  #指定发送单播的源IP
unicast_peer {
    <IPADDR>             #指定接收单播的对方目标主机IP
    ......
}

范例：

#master 主机配置
[root@ka1-centos8 ~]#cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id ka1.wang.org
   vrrp_skip_check_adv_addr
#vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 239.0.0.0  #单播优先于多播
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 66
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
10.0.0.10/24 dev eth0 label eth0:1
    }
    unicast_src_ip 10.0.0.8     #本机IP
    unicast_peer{
10.0.0.18               #指向对方主机IP
10.0.0.28               #如果有多个keepalived,再加其它节点的IP
    }
}
[root@ha1-centos8 ~]#hostname -I
10.0.0.8 10.0.0.10
#slave 主机配置
[root@ka2-centos8 ~]#cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id ka2.wang.org
   vrrp_skip_check_adv_addr
#vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 239.0.0.0  #单播优先于多播
}
vrrp_instance VI_1 {
    state SLAVE
    interface eth0
    virtual_router_id 66
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
10.0.0.10/24 dev eth0 label eth0:1
    }
    unicast_src_ip 10.0.0.18        #本机IP
    unicast_peer {
10.0.0.8                    #指向对方主机IP
    }
}
[root@ka2-centos8 ~]#hostname -I
10.0.0.18

范例: 抓包观察

root@centos6 ~]#tcpdump  -i eth0 -nn src host 10.0.0.8 and dst host 10.0.0.18
tcpdump: verbose output suppressed, use -v or -vvfor full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:37:48.069158 IP 10.0.0.8 > 10.0.0.18: VRRPv2, Advertisement, vrid 66, prio
100, authtype simple, intvl 1s, length 20
23:37:49.070013 IP 10.0.0.8 > 10.0.0.18: VRRPv2, Advertisement, vrid 66, prio
100, authtype simple, intvl 1s, length 20
23:37:50.071144 IP 10.0.0.8 > 10.0.0.18: VRRPv2, Advertisement, vrid 66, prio
100, authtype simple, intvl 1s, length 20

3.4 Keepalived 通知脚本配置

当keepalived的状态变化时，可以自动触发脚本的执行，比如：发邮件通知用户

默认以用户keepalived_script身份执行脚本，如果此用户不存在，以root执行脚本

可以用下面指令指定脚本执行用户的身份

global_defs {
    ......
    script_user <USER>
    ......
}

3.4.1 通知脚本类型

当前节点成为主节点时触发的脚本

notify_master <STRING>|<QUOTED-STRING>

当前节点转为备节点时触发的脚本

notify_backup <STRING>|<QUOTED-STRING>

当前节点转为“失败”状态时触发的脚本

notify_fault <STRING>|<QUOTED-STRING>

通用格式的通知触发机制，一个脚本可完成以上三种状态的转换时的通知

notify <STRING>|<QUOTED-STRING>

当停止VRRP时触发的脚本

notify_stop <STRING>|<QUOTED-STRING>

3.4.2 脚本的调用方法

在 vrrp_instance VI_1 语句块的末尾加下面行

notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"

3.4.3 实战案例1：实现 Keepalived 状态切换的通知脚本

以下脚本支持RHEL和Ubuntu系统

#在所有 keepalived节点配置如下
[root@ka1 ~]#cat /etc/keepalived/notify.sh
#!/bin/bash
#
#********************************************************************
#Author:            wangxiaochun
#QQ:                29308620
#Date:              2020-02-31