架构基于WEB的负载均衡Nginx+keepalived

Nginx+keepalived架构基于WEB的负载均衡（转载）服务器故障：(服务器故障包括：服务器宕机、web服务终止、网线松动等等)①当lvs-master故障时，无法再接受用户请求并将请求转发给真实的web服务器(即便真实web服务器正常)从而导致整个web服务的瘫痪，也就是lvs控制器存在单点故障问题。②当lvs-master正常时，真实地web服务器如web1-realserver故障。此时lvs-master并不知道真实服务器是否在正常提供web服务，所以仍然在向故障的web1-realserver转发用户请求。这样的结果是用户请求无法被故障web服务器相应，某些用户可以访问网站有些则无法访问。基于以上的问题，我们需要想办法实现对lvs控制器和web服务器的健康监测,一旦服务出现问题能保证服务不中断的情况下排除故障。即增加lvs控制器实现主备模式避免单点故障以及自动删除故障web服务结点并当它恢复后再自动添加到群集中这样的功能，这就是LVS+keepalived能实现的功能。整个线上环境由Linuxtone的站长netseek提供，这里表示感谢，整个系统的拓补如下：名称IPLVS-DR-Master61.164.122.6LVS-DR-BACKUP61.164.122.7LVS-DR-VIP61.164.122.8WEB1-Realserver61.164.122.9WEB2-Realserver61.164.122.10实施步骤：①在realserver主机上实行脚本realserver,为lo:0绑定VIP地址61.164.122.8，这步分别在二个web主机上61.164.122.9、61.164.122.10实施。这步提前做，是因为以后的过程中这一步是不会发生更改的。#vim/usr/local/sbin/realserver#!/bin/bashSNS_VIP=61.164.122.8./etc/rc.d/init.d/functionscase$1instart)ifconfiglo:0$SNS_VIPnetmask255.255.255.255broadcast$SNS_VIP/sbin/routeadd-host$SNS_VIPdevlo:0echo1/proc/sys/net/ipv4/conf/lo/arp_ignoreecho2/proc/sys/net/ipv4/conf/lo/arp_announceecho1/proc/sys/net/ipv4/conf/all/arp_ignoreecho2/proc/sys/net/ipv4/conf/all/arp_announcesysctl-p/dev/null2&1echoRealServerStartOK;;stop)ifconfiglo:0downroutedel$SNS_VIP/dev/null2&1echo0/proc/sys/net/ipv4/conf/lo/arp_ignoreecho0/proc/sys/net/ipv4/conf/lo/arp_announceecho0/proc/sys/net/ipv4/conf/all/arp_ignoreecho0/proc/sys/net/ipv4/conf/all/arp_announceechoRealServerStoped;;*)echoUsage:$0{start|stop}exit1esacexit0②为二台lvs主机安装lvs脚本，这步并非必要。做完这步时可以检测到用软件实现了LVS/DR的双机互备机制，但并不能实现的智能自动增加删除故障的web站点，所以这个靠keepalived来实现。过程如下：#mkdir/usr/local/src/lvs#cd/usr/local/src/lvs#wget脚本如下/usr/local/src/lvs/lvs#!/bin/bash#websitedirectorvip.SNS_VIP=61.164.122.8SNS_RIP1=61.164.122.9SNS_RIP2=61.164.122.10./etc/rc.d/init.d/functionslogger$0calledwith$1case$1instart)#setsquidvip/sbin/ipvsadm--set30560/sbin/ifconfigeth0:0$SNS_VIPbroadcast$SNS_VIPnetmask255.255.255.255broadcast$SNS_VIPup/sbin/routeadd-host$SNS_VIPdeveth0:0/sbin/ipvsadm-A-t$SNS_VIP:80-swrr-p3/sbin/ipvsadm-a-t$SNS_VIP:80-r$SNS_RIP1:80-g-w1/sbin/ipvsadm-a-t$SNS_VIP:80-r$SNS_RIP2:80-g-w1touch/var/lock/subsys/ipvsadm/dev/null2&1;;stop)/sbin/ipvsadm-C/sbin/ipvsadm-Zifconfigeth0:0downifconfigeth0:1downroutedel$SNS_VIProutedel$SS_VIPrm-rf/var/lock/subsys/ipvsadm/dev/null2&1echoipvsadmstoped;;status)if[!-e/var/lock/subsys/ipvsadm];thenechoipvsadmstopedexit1elseechoipvsadmOKfi;;*)echoUsage:$0{start|stop|status}exit1esacexit0③关闭lvs脚本./lvsstop，编辑keepalived.conf文件，用keepalived实现负载均衡及高可用性。a)Keepalved的安装#wget将keepalived做成启动脚务，方便管理:#cp/usr/local/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/#cp/usr/local/etc/sysconfig/keepalived/etc/sysconfig/#mkdir/etc/keepalived#cp/usr/local/etc/keepalived/keepalived.conf/etc/keepalived/#cp/usr/local/sbin/keepalived/usr/sbin/#servicekeepalivedstart|stopb)Keealived的配置1、配置在主负载均衡服务器上配置keepalived.conf#vim/etc/keepalived/keepalived.conf!ConfigurationFileforkeepalivedglobal_defs{notification_email{[e-mail]}[e-mail]_server127.0.0.1#smtp_connect_timeout30router_idLVS_DEVEL}vrrp_instanceVI_1{stateMASTER#备份服务器上将MASTER改为BACKUPinterfaceeth0virtual_router_id51priority100#备份服务上将100改为99advert_int1authentication{auth_typePASSauth_pass1111}virtual_ipaddress{61.164.122.8}}virtual_server61.164.122.880{delay_loop6#(每隔10秒查询realserver状态)lb_algowrr#lvs算法lb_kindDR#DirectRoutepersistence_timeout60#同一IP的连接60秒内被分配到同一台realserverprotocolTCP#用TCP协议检查realserver状态real_server61.164.122.980{weight3#权重TCP_CHECK{connect_timeout10#10秒无响应超时nb_get_retry3delay_before_retry3connect_port80}}real_server61.164.122.1080{weight3TCP_CHECK{connect_timeout10nb_get_retry3delay_before_retry3connect_port80}}}2、BACKUP服务器同上配置，先安装lvs再按装keepalived,仍后配置/etc/keepalived/keepalived.conf，只需将红色标示的部分改一下即可。3、停掉lvs脚本，分别在二台lvs机上启动servciekeepalivedstart就可实现负载均衡及高可用集群。※值得注意的是：1、你必须向你的服务器所在机房IDC多申请一个IP供VIP使用。2、服务器的iptables、SElinux均关闭，在生产过程中，我就遇到了iptables的NAT转发问题，导致了lvs失败。3、修改keepalved.conf文件请直接修改/etc/keealived/keepalived.conf，因为keepalived启动时会以这个为默认。4、系统排障时多用ipvsadm-ln和tail-f/var/log/messages。以上架构我在生产环境、局域网、虚拟机下均实现，有兴趣的同志可以对照实验，有问题的话欢迎来信交流[e-mail](抚琴煮酒)