四台虚拟机KA1(172.25.254.50)、KA2(172.25.254.60)、rs1(172.25.254.10)、rs2(172.25.254.20)一、环境准备# KA1 当时间服务器 [rootKA1 ~]# vim /etc/chrony.conf allow 0.0.0.0/0 local stratum 10 [rootKA1 ~]# systemctl restart chronyd # KA2 同步 KA1 [rootKA2 ~]# vim /etc/chrony.conf pool 172.25.254.50 iburst [rootKA2 ~]# systemctl restart chronyd [rootKA2 ~]# chronyc sources -v MS Name/IP address Stratum Poll Reach LastRx Last sample ^* KA1 3 6 17 13 303ns[6125ns] /- 69msrs1、rs2 装 httpd[rootrs1 ~]# dnf install httpd -y [rootrs1 ~]# echo RS1 - 172.25.254.10 /var/www/html/index.html [rootrs1 ~]# systemctl enable --now httpd [rootrs2 ~]# dnf install httpd -y [rootrs2 ~]# echo RS2 - 172.25.254.20 /var/www/html/index.html [rootrs2 ~]# systemctl enable --now httpd测试[rootKA1 ~]# curl 172.25.254.10 RS1 - 172.25.254.10 [rootKA1 ~]# curl 172.25.254.20 RS2 - 172.25.254.20二、Keepalived 虚拟路由安装[rootKA1 ~]# dnf install keepalived.x86_64 -y [rootKA2 ~]# dnf install keepalived.x86_64 -yKA1 配置[rootKA1 ~]# vim /etc/keepalived/keepalived.confglobal_defs { notification_email { timinglee_zln163.com } notification_email_from timinglee_zln163.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id KA1 vrrp_skip_check_adv_addr #vrrp_strict vrrp_garp_interval 1 vrrp_gna_interval 1 vrrp_mcast_group4 224.0.0.44 } vrrp_instance WEB_VIP { state MASTER interface ens160 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }KA2 配置[rootKA2 ~]# vim /etc/keepalived/keepalived.confglobal_defs { notification_email { timinglee_zln163.com } notification_email_from timinglee_zln163.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id KA2 vrrp_skip_check_adv_addr #vrrp_strict vrrp_garp_interval 1 vrrp_gna_interval 1 vrrp_mcast_group4 224.0.0.44 } vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }启动[rootKA1 ~]# systemctl enable --now keepalived.service Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service. [rootKA2 ~]# systemctl enable --now keepalived.service Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.验证 VRRP 通告[rootKA1 ~]# tcpdump -i ens160 -nn host 224.0.0.44 tcpdump: verbose output suppressed, use -v for full protocol decode listening on ens160, link-type EN10MB (Ethernet), snapshot length 262144 bytes 07:35:12.183386 IP 172.25.254.50 224.0.0.44: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20 07:35:13.184051 IP 172.25.254.50 224.0.0.44: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20 07:35:14.184610 IP 172.25.254.50 224.0.0.44: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20 07:35:15.185验证 VIPKA1[rootKA1 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:26:33:d9 brd ff:ff:ff:ff:ff:ff inet 172.25.254.50/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet 172.25.254.100/24 scope global secondary ens160:0 valid_lft forever preferred_lft forever inet6 fe80::3901:aeea:786a:7227/64 scope link noprefixroute valid_lft forever preferred_lft foreverKA2[rootKA2 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:1e:fd:7a brd ff:ff:ff:ff:ff:ff inet 172.25.254.60/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet6 fe80::26df:35e5:539:56bc/64 scope link noprefixroute valid_lft forever preferred_lft forever故障切换停 KA1[rootKA1 ~]# systemctl stop keepalived.serviceKA1 VIP 消失[rootKA1 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:26:33:d9 brd ff:ff:ff:ff:ff:ff inet 172.25.254.50/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet6 fe80::3901:aeea:786a:7227/64 scope link noprefixroute valid_lft forever preferred_lft foreverKA2 VIP 漂移过来[rootKA2 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:1e:fd:7a brd ff:ff:ff:ff:ff:ff inet 172.25.254.60/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet 172.25.254.100/24 scope global secondary ens160:0 valid_lft forever preferred_lft forever inet6 fe80::26df:35e5:539:56bc/64 scope link noprefixroute valid_lft forever preferred_lft foreverKA2 日志[rootKA2 ~]# journalctl -u keepalived -n 5 Jul 31 07:38:16 KA2 Keepalived_vrrp[3157]: (WEB_VIP) Receive advertisement timeout Jul 31 07:38:16 KA2 Keepalived_vrrp[3157]: (WEB_VIP) Entering MASTER STATE Jul 31 07:38:16 KA2 Keepalived_vrrp[3157]: (WEB_VIP) setting protocol VIPs. Jul 31 07:38:16 KA2 Keepalived_vrrp[3157]: (WEB_VIP) Sending/queueing gratuitous ARPs on ens160 for 172.25.254.100恢复 KA1VIP 抢回。三、日志分离[rootKA1 ~]# vim /etc/sysconfig/keepalived KEEPALIVED_OPTIONS-D -S 6 [rootKA1 ~]# systemctl restart keepalived.service [rootKA1 ~]# vim /etc/rsyslog.conf local6.* /var/log/keepalived.log [rootKA1 ~]# systemctl restart rsyslog.service验证[rootKA1 ~]# ls -l /var/log/keepalived.log -rw------- 1 root root 1.2K Jul 31 07:42 /var/log/keepalived.log [rootKA1 ~]# tail /var/log/keepalived.log Jul 31 07:42:18 KA1 Keepalived_vrrp[2848]: (WEB_VIP) Entering MASTER STATE Jul 31 07:42:18 KA1 Keepalived_vrrp[2848]: (WEB_VIP) setting protocol VIPs.四、子配置文件[rootKA1 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { timinglee_zln163.com } notification_email_from timinglee_zln163.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id KA1 vrrp_skip_check_adv_addr #vrrp_strict vrrp_garp_interval 1 vrrp_gna_interval 1 vrrp_mcast_group4 224.0.0.44 } include /etc/keepalived/conf.d/*.conf[rootKA1 ~]# mkdir /etc/keepalived/conf.d -p [rootKA1 ~]# vim /etc/keepalived/conf.d/webvip.conf vrrp_instance WEB_VIP { state MASTER interface ens160 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }[rootKA1 ~]# keepalived -t -f /etc/keepalived/keepalived.conf [rootKA1 ~]# systemctl restart keepalived.service [rootKA1 ~]# ip addr show ens160 | grep 172.25.254.100 inet 172.25.254.100/24 scope global secondary ens160:0五、抢占模式5.1 非抢占模式KA1vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 nopreempt priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }KA2vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 nopreempt priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }[rootKA1 ~]# systemctl start keepalived.service [rootKA2 ~]# systemctl start keepalived.service [rootKA1 ~]# ip addr show ens160 | grep 172.25.254.100 inet 172.25.254.100/24 scope global secondary ens160:0 [rootKA1 ~]# systemctl stop keepalived.service [rootKA2 ~]# ip addr show ens160 | grep 172.25.254.100 inet 172.25.254.100/24 scope global secondary ens160:0 # KA1 恢复VIP 不抢回 [rootKA1 ~]# systemctl start keepalived.service [rootKA1 ~]# ip addr show ens160 | grep 172.25.254.100 # 无输出VIP 还在 KA25.2 延迟抢占KA1 和 KA2 都改成vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 preempt_delay 10 priority 100 # KA2 写 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }[rootKA1 ~]# watch -n 1 ip addr show ens160 # 停 KA1VIP 去 KA2 # 恢复 KA1等 10 秒 VIP 才飘回来六、单播模式组播不能跨网段换单播。KA1vrrp_instance WEB_VIP { state MASTER interface ens160 virtual_router_id 51 priority 100 advert_int 1 unicast_src_ip 172.25.254.50 unicast_peer { 172.25.254.60 } authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }KA2vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 priority 80 advert_int 1 unicast_src_ip 172.25.254.60 unicast_peer { 172.25.254.50 } authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } }[rootKA1 ~]# systemctl restart keepalived.service [rootKA2 ~]# systemctl restart keepalived.service抓包验证单播不是 224.0.0.44[rootKA1 ~]# tcpdump -i ens160 -nn src host 172.25.254.50 and dst 172.25.254.60 tcpdump: verbose output suppressed, use -v for full protocol decode listening on ens160, link-type EN10MB (Ethernet), snapshot length 262144 bytes 08:15:23.456789 IP 172.25.254.50 172.25.254.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20 08:15:24.457123 IP 172.25.254.50 172.25.254.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20停 KA1KA2 接管再启动 KA1VIP 抢回。单播模式下切换正常。七、VIP 迁移邮件告警7.1 邮件环境[rootKA1 ~]# dnf install s-nail postfix -y [rootKA1 ~]# systemctl start postfix.service [rootKA1 ~]# vim /etc/mail.rc set smtpsmtp.163.com set smtp-authlogin set smtp-auth-usertimin_zln163.com set smtp-auth-passwordTGfdKaJT7EB set fromtimie_zln163.com set ssl-verifyignore [rootKA1 ~]# echo hello | mailx -s test 1122334455qq.com [rootKA1 ~]# mailq Mail queue is empty7.2 告警脚本[rootKA1 ~]# mkdir -p /etc/keepalived/scripts [rootKA1 ~]# vim /etc/keepalived/scripts/waring.sh #!/bin/bash mail_dest111qq.com mail_send() { mail_subj$HOSTNAME to be $1 vip 转移 mail_messdate %F\ %T: vrrp 转移,$HOSTNAME 变为 $1 echo $mail_mess | mail -s $mail_subj $mail_dest } case $1 in master) mail_send master ;; backup) mail_send backup ;; fault) mail_send fault ;; *) exit 1 ;; esac [rootKA1 ~]# chmod x /etc/keepalived/scripts/waring.sh7.3 绑定 keepalivedglobal_defs { notification_email { timinglee_zln163.com } notification_email_from timinglee_zln163.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id KA1 vrrp_skip_check_adv_addr #vrrp_strict vrrp_garp_interval 1 vrrp_gna_interval 1 vrrp_mcast_group4 224.0.0.44 enable_script_security script_user root } vrrp_instance WEB_VIP { state MASTER interface ens160 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } notify_master /etc/keepalived/scripts/waring.sh master notify_backup /etc/keepalived/scripts/waring.sh backup notify_fault /etc/keepalived/scripts/waring.sh fault }[rootKA1 ~]# systemctl restart keepalived.service测试[rootKA1 ~]# systemctl stop keepalived.service # 邮箱收到KA1 to be backup vip 转移 [rootKA1 ~]# systemctl start keepalived.service # 邮箱收到KA1 to be master vip 转移八、双主模式KA1vrrp_instance WEB_VIP { state MASTER interface ens160 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } } vrrp_instance DB_VIP { state BACKUP interface ens160 virtual_router_id 52 priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.200/24 dev ens160 label ens160:1 } }KA2vrrp_instance WEB_VIP { state BACKUP interface ens160 virtual_router_id 51 priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.100/24 dev ens160 label ens160:0 } } vrrp_instance DB_VIP { state MASTER interface ens160 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.25.254.200/24 dev ens160 label ens160:1 } }[rootKA1 ~]# systemctl restart keepalived.service [rootKA2 ~]# systemctl restart keepalived.serviceKA1 结果[rootKA1 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:26:33:d9 brd ff:ff:ff:ff:ff:ff inet 172.25.254.50/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet 172.25.254.100/24 scope global secondary ens160:0 valid_lft forever preferred_lft forever inet6 fe80::3901:aeea:786a:7227/64 scope link noprefixroute valid_lft forever preferred_lft foreverKA2 结果[rootKA2 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:1e:fd:7a brd ff:ff:ff:ff:ff:ff inet 172.25.254.60/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet 172.25.254.200/24 scope global secondary ens160:1 valid_lft forever preferred_lft forever inet6 fe80::26df:35e5:539:56bc/64 scope link noprefixroute valid_lft forever preferred_lft forever停 KA1[rootKA1 ~]# systemctl stop keepalived.service [rootKA2 ~]# ip addr show ens160 2: ens160: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:1e:fd:7a brd ff:ff:ff:ff:ff:ff inet 172.25.254.60/24 brd 172.25.254.255 scope global noprefixroute ens160 valid_lft forever preferred_lft forever inet 172.25.254.100/24 scope global secondary ens160:0 valid_lft forever preferred_lft forever inet 172.25.254.200/24 scope global secondary ens160:1 valid_lft forever preferred_lft forever两个 VIP 都漂到 KA2验证成功。九、IPVS 高可用LVS Keepalived9.1 RS 端配置[rootrs1 ~]# cd /etc/NetworkManager/system-connections/ [rootrs1 system-connections]# cp eth0.nmconnection lo.nmconnection -p [rootrs1 system-connections]# vim lo.nmconnection [connection] idlo typeloopback interface-namelo [ipv4] methodmanual address1127.0.0.1/8 address2172.25.254.100/32 [rootrs1 system-connections]# nmcli connection reload [rootrs1 system-connections]# nmcli connection up lo [rootrs1 ~]# vim /etc/sysctl.conf net.ipv4.conf.all.arp_ignore1 net.ipv4.conf.all.arp_announce2 net.ipv4.conf.lo.arp_announce2 net.ipv4.conf.lo.arp_ignore1 [rootrs1 ~]# sysctl -p net.ipv4.conf.all.arp_ignore 1 net.ipv4.conf.all.arp_announce 2 net.ipv4.conf.lo.arp_announce 2 net.ipv4.conf.lo.arp_ignore 1rs2 同样操作。9.2 KA 端配置[rootKA1 ~]# dnf install ipvsadm -yvirtual_server 172.25.254.100 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP real_server 172.25.254.10 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 retry 3 delay_before_retry 1 } } real_server 172.25.254.20 80 { weight 1 TCP_CHECK { connect_timeout 5 retry 3 delay_before_retry 3 connect_port 80 } } }[rootKA1 ~]# systemctl restart keepalived.service9.3 验证 LVS 策略[rootKA1 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size4096) Prot LocalAddress:Port Scheduler Flags - RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.25.254.100:80 rr - 172.25.254.10:80 Route 1 0 0 - 172.25.254.20:80 Route 1 0 0客户端测试别在 KA1/KA2 上测[rootclient ~]# for i in {1..6}; do curl 172.25.254.100; done RS1 - 172.25.254.10 RS2 - 172.25.254.20 RS1 - 172.25.254.10 RS2 - 172.25.254.20 RS1 - 172.25.254.10 RS2 - 172.25.254.209.4 健康检查停 rs1 的 httpd[rootrs1 ~]# systemctl stop httpd [rootKA1 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size4096) Prot LocalAddress:Port Scheduler Flags - RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.25.254.100:80 rr - 172.25.254.20:80 Route 1 0 0rs1 被自动踢出启动 httpd 后又自动加回。停 KA1 keepalivedKA2 自动生成同样策略[rootKA2 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size4096) Prot LocalAddress:Port Scheduler Flags - RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.25.254.100:80 rr - 172.25.254.10:80 Route 1 0 0 - 172.25.254.20:80 Route 1 0 0
