Keepalived+Nginx+Tomcat 實現高可用Web集群的示例代碼

Keepalived+Nginx+Tomcat 實現高可用Web集群

 

一、Nginx的安裝過程

1.下載Nginx安裝包,安裝依賴環境包

(1)安裝 C++編譯環境

yum -y install gcc  #C++

(2)安裝pcre

yum -y install pcre-devel

(3)安裝zlib

yum -y install zlib-devel

(4)安裝Nginx

定位到nginx 解壓文件位置,執行編譯安裝命令

[[email protected] nginx-1.12.2]# pwd
/usr/local/nginx/nginx-1.12.2
[[email protected] nginx-1.12.2]# ./configure && make && make install

(5)啟動Nginx

安裝完成后先尋找那安裝完成的目錄位置

[[email protected] nginx-1.12.2]# whereis nginx
nginx: /usr/local/nginx
[[email protected] nginx-1.12.2]# 

進入Nginx子目錄sbin啟動Nginx

[[email protected] sbin]# ls
nginx
[[email protected] sbin]# ./nginx &
[1] 5768
[[email protected] sbin]# 

查看Nginx是否啟動

或通過進程查看Nginx啟動情況

[[email protected] sbin]# ps -aux|grep nginx
root    5769 0.0 0.0 20484  608 ?    Ss  14:03  0:00 nginx: master process ./nginx
nobody   5770 0.0 0.0 23012 1620 ?    S  14:03  0:00 nginx: worker process
root    5796 0.0 0.0 112668  972 pts/0  R+  14:07  0:00 grep --color=auto nginx
[1]+ 完成         ./nginx
[[email protected] sbin]# 

到此Nginx安裝完成并啟動成功。

(6)Nginx快捷啟動和開機啟動配置

編輯Nginx快捷啟動腳本【 注意Nginx安裝路徑 需要根據自己的NGINX路徑進行改動

[[email protected] init.d]# vim /etc/rc.d/init.d/nginx
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemon
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# config: /usr/local/nginx/conf/nginx.conf
# pidfile: /usr/local/nginx/logs/nginx.pid
 
# Source function library.
. /etc/rc.d/init.d/functions
 
# Source networking configuration.
. /etc/sysconfig/network
 
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/local/nginx/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"
[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
lockfile=/var/lock/subsys/nginx
 
make_dirs() {
  # make required directories
  user=`$nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -`
  if [ -z "`grep $user /etc/passwd`" ]; then
  useradd -M -s /bin/nologin $user
  fi
  options=`$nginx -V 2>&1 | grep 'configure arguments:'`
  for opt in $options; do
  if [ `echo $opt | grep '.*-temp-path'` ]; then
  value=`echo $opt | cut -d "=" -f 2`
  if [ ! -d "$value" ]; then
  # echo "creating" $value
  mkdir -p $value && chown -R $user $value
  fi
  fi
  done
}
 
start() {
  [ -x $nginx ] || exit 5
  [ -f $NGINX_CONF_FILE ] || exit 6
  make_dirs
  echo -n $"Starting $prog: "
  daemon $nginx -c $NGINX_CONF_FILE
  retval=$?
  echo
  [ $retval -eq 0 ] && touch $lockfile
  return $retval
}
 
stop() {
  echo -n $"Stopping $prog: "
  killproc $prog -QUIT
  retval=$?
  echo
  [ $retval -eq 0 ] && rm -f $lockfile
  return $retval
}
 
restart() {
  #configtest || return $?
  stop
  sleep 1
  start
}
 
reload() {
  #configtest || return $?
  echo -n $"Reloading $prog: "
  killproc $nginx -HUP
  RETVAL=$?
  echo
}
 
force_reload() {
  restart
}
 
configtest() {
  $nginx -t -c $NGINX_CONF_FILE
}
 
rh_status() {
  status $prog
}
 
rh_status_q() {
  rh_status >/dev/null 2>&1
}
 
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
 
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac

為啟動腳本授權 并加入開機啟動

[[email protected] init.d]# chmod -R 777 /etc/rc.d/init.d/nginx 
[[email protected] init.d]# chkconfig nginx 

啟動Nginx

[[email protected] init.d]# ./nginx start

將Nginx加入系統環境變量

[[email protected] init.d]# echo 'export PATH=$PATH:/usr/local/nginx/sbin'>>/etc/profile && source /etc/profile

Nginx命令 [ service nginx (start|stop|restart) ]

[[email protected] init.d]# service nginx start
Starting nginx (via systemctl):              [ 確定 ]

Tips: 快捷命令

service nginx (start|stop|restart)

二、KeepAlived安裝和配置

1.安裝Keepalived依賴環境

yum install -y popt-devel   
yum install -y ipvsadm
yum install -y libnl*
yum install -y libnf*
yum install -y openssl-devel

2.編譯Keepalived并安裝

[[email protected] keepalived-1.3.9]# ./configure
[[email protected] keepalived-1.3.9]# make && make install

3.將Keepalive 安裝成系統服務

[[email protected] etc]# mkdir /etc/keepalived
[[email protected] etc]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

手動復制默認的配置文件到默認路徑

[[email protected] etc]# mkdir /etc/keepalived
[[email protected] etc]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[[email protected] etc]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

為keepalived 創建軟鏈接

[[email protected] sysconfig]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/

設置Keepalived開機自啟動

[[email protected] sysconfig]# chkconfig keepalived on
注意:正在將請求轉發到“systemctl enable keepalived.service”。
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service

啟動Keepalived服務

[[email protected] keepalived]# keepalived -D -f /etc/keepalived/keepalived.conf

關閉Keepalived服務

[[email protected] keepalived]# killall keepalived

三、集群規劃和搭建

環境準備:

CentOS 7.2

Keepalived   Version 1.4.0 - December 29, 2017

Nginx           Version: nginx/1.12.2

Tomcat         Version:8

集群規劃清單

虛擬機 IP 說明
Keepalived+Nginx1[Master] 192.168.43.101 Nginx Server 01
Keeepalived+Nginx[Backup] 192.168.43.102 Nginx Server 02
Tomcat01 192.168.43.103 Tomcat Web Server01
Tomcat02 192.168.43.104 Tomcat Web Server02
VIP 192.168.43.150 虛擬漂移IP

1.更改Tomcat默認歡迎頁面,用于標識切換Web

更改TomcatServer01 節點ROOT/index.jsp 信息,加入TomcatIP地址,并加入Nginx值,即修改節點192.168.43.103信息如下:

<div id="asf-box">
  <h1>${pageContext.servletContext.serverInfo}(192.168.224.103)<%=request.getHeader("X-NGINX")%></h1>
</div>

更改TomcatServer02 節點ROOT/index.jsp信息,加入TomcatIP地址,并加入Nginx值,即修改節點192.168.43.104信息如下:

<div id="asf-box">
  <h1>${pageContext.servletContext.serverInfo}(192.168.224.104)<%=request.getHeader("X-NGINX")%></h1>
</div>

2.啟動Tomcat服務,查看Tomcat服務IP信息,此時Nginx未啟動,因此request-header沒有Nginx信息。

 

3.配置Nginx代理信息

1.配置Master節點[192.168.43.101]代理信息

upstream tomcat {
  server 192.168.43.103:8080 weight=1;
  server 192.168.43.104:8080 weight=1;
}
server{
  location / {
    proxy_pass http://tomcat;
  proxy_set_header X-NGINX "NGINX-1";
  }
  #......其他省略
}

2.配置Backup節點[192.168.43.102]代理信息

upstream tomcat {
  server 192.168.43.103:8080 weight=1;
  server 192.168.43.104:8080 weight=1;
}
server{
  location / {
    proxy_pass http://tomcat;
  proxy_set_header X-NGINX "NGINX-2";
  }
  #......其他省略
}

3.啟動Master 節點Nginx服務

[[email protected] init.d]# service nginx start
Starting nginx (via systemctl):              [ 確定 ]

此時訪問 192.168.43.101 可以看到103和104節點Tcomat交替顯示,說明Nginx服務已經將請求負載到了2臺tomcat上。

 

4.同理配置Backup[192.168.43.102] Nginx信息,啟動Nginx后,訪問192.168.43.102后可以看到Backup節點已起到負載的效果。

 

4.配置Keepalived 腳本信息

1. 在Master節點和Slave節點 /etc/keepalived目錄下添加check_nginx.sh 文件,用于檢測Nginx的存貨狀況,添加keepalived.conf文件

check_nginx.sh文件信息如下:

#!/bin/bash
#時間變量,用于記錄日志
d=`date --date today +%Y%m%d_%H:%M:%S`
#計算nginx進程數量
n=`ps -C nginx --no-heading|wc -l`
#如果進程為0,則啟動nginx,并且再次檢測nginx進程數量,
#如果還為0,說明nginx無法啟動,此時需要關閉keepalived
if [ $n -eq "0" ]; then
    /etc/rc.d/init.d/nginx start
    n2=`ps -C nginx --no-heading|wc -l`
    if [ $n2 -eq "0" ]; then
        echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
        systemctl stop keepalived
    fi
fi

添加完成后,為check_nginx.sh 文件授權,便于腳本獲得執行權限。

[[email protected] keepalived]# chmod -R 777 /etc/keepalived/check_nginx.sh 

2.在Master 節點 /etc/keepalived目錄下,添加keepalived.conf 文件,具體信息如下:

vrrp_script chk_nginx { 
 script "/etc/keepalived/check_nginx.sh"  //檢測nginx進程的腳本 
 interval 2 
 weight -20 
} 

global_defs { 
 notification_email { 
   //可以添加郵件提醒 
 } 
} 
vrrp_instance VI_1 { 
 state MASTER         #標示狀態為MASTER 備份機為BACKUP
 interface ens33        #設置實例綁定的網卡(ip addr查看,需要根據個人網卡綁定)
 virtual_router_id 51     #同一實例下virtual_router_id必須相同  
 mcast_src_ip 192.168.43.101  
 priority 250         #MASTER權重要高于BACKUP 比如BACKUP為240 
 advert_int 1         #MASTER與BACKUP負載均衡器之間同步檢查的時間間隔,單位是秒
 nopreempt           #非搶占模式
 authentication {       #設置認證
    auth_type PASS     #主從服務器驗證方式
    auth_pass 123456 
 } 
 track_script { 
    check_nginx 
 } 
 virtual_ipaddress {      #設置vip
    192.168.43.150     #可以多個虛擬IP,換行即可
 } 
}

3.在Backup節點 etc/keepalived目錄下添加 keepalived.conf 配置文件

信息如下:

vrrp_script chk_nginx { 
 script "/etc/keepalived/check_nginx.sh"  //檢測nginx進程的腳本 
 interval 2 
 weight -20 
} 

global_defs { 
 notification_email { 
   //可以添加郵件提醒 
 } 
} 
vrrp_instance VI_1 { 
 state BACKUP         #標示狀態為MASTER 備份機為BACKUP
 interface ens33        #設置實例綁定的網卡(ip addr查看)
 virtual_router_id 51     #同一實例下virtual_router_id必須相同  
 mcast_src_ip 192.168.43.102  
 priority 240         #MASTER權重要高于BACKUP 比如BACKUP為240 
 advert_int 1         #MASTER與BACKUP負載均衡器之間同步檢查的時間間隔,單位是秒
 nopreempt           #非搶占模式
 authentication {       #設置認證
    auth_type PASS     #主從服務器驗證方式
    auth_pass 123456 
 } 
 track_script { 
    check_nginx 
 } 
 virtual_ipaddress {      #設置vip
    192.168.43.150     #可以多個虛擬IP,換行即可
 } 
}

Tips: 關于配置信息的幾點說明

  • state - 主服務器需配成MASTER,從服務器需配成BACKUP
  • interface - 這個是網卡名,我使用的是VM12.0的版本,所以這里網卡名為ens33
  • mcast_src_ip - 配置各自的實際IP地址
  • priority - 主服務器的優先級必須比從服務器的高,這里主服務器配置成250,從服務器配置成240
  • virtual_ipaddress - 配置虛擬IP(192.168.43.150)
  • authentication - auth_pass主從服務器必須一致,keepalived靠這個來通信
  • virtual_router_id - 主從服務器必須保持一致

 5.集群高可用(HA)驗證

Step1 啟動Master機器的Keepalived和 Nginx服務

[[email protected] keepalived]# keepalived -D -f /etc/keepalived/keepalived.conf
[[email protected] keepalived]# service nginx start

查看服務啟動進程

[[email protected] keepalived]# ps -aux|grep nginx
root    6390 0.0 0.0 20484  612 ?    Ss  19:13  0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nobody   6392 0.0 0.0 23008 1628 ?    S  19:13  0:00 nginx: worker process
root    6978 0.0 0.0 112672  968 pts/0  S+  20:08  0:00 grep --color=auto nginx

查看Keepalived啟動進程

[[email protected] keepalived]# ps -aux|grep keepalived
root    6402 0.0 0.0 45920 1016 ?    Ss  19:13  0:00 keepalived -D -f /etc/keepalived/keepalived.conf
root    6403 0.0 0.0 48044 1468 ?    S  19:13  0:00 keepalived -D -f /etc/keepalived/keepalived.conf
root    6404 0.0 0.0 50128 1780 ?    S  19:13  0:00 keepalived -D -f /etc/keepalived/keepalived.conf
root    7004 0.0 0.0 112672  976 pts/0  S+  20:10  0:00 grep --color=auto keepalived

使用 ip add 查看虛擬IP綁定情況,如出現192.168.43.150 節點信息則綁定到Master節點

[[email protected] keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
  inet6 ::1/128 scope host 
    valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  link/ether 00:0c:29:91:bf:59 brd ff:ff:ff:ff:ff:ff
  inet 192.168.43.101/24 brd 192.168.43.255 scope global ens33
    valid_lft forever preferred_lft forever
  inet 192.168.43.150/32 scope global ens33
    valid_lft forever preferred_lft forever
  inet6 fe80::9abb:4544:f6db:8255/64 scope link 
    valid_lft forever preferred_lft forever
  inet6 fe80::b0b3:d0ca:7382:2779/64 scope link tentative dadfailed 
    valid_lft forever preferred_lft forever
  inet6 fe80::314f:5fe7:4e4b:64ed/64 scope link tentative dadfailed 
    valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
  link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff
  inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
    valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
  link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff

Step 2 啟動Backup節點Nginx服務和Keepalived服務,查看服務啟動情況,如Backup節點出現了虛擬IP,則Keepalvied配置文件有問題,此情況稱為腦裂。

[[email protected] keepalived]# clear
[[email protected] keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
  inet6 ::1/128 scope host 
    valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  link/ether 00:0c:29:14:df:79 brd ff:ff:ff:ff:ff:ff
  inet 192.168.43.102/24 brd 192.168.43.255 scope global ens33
    valid_lft forever preferred_lft forever
  inet6 fe80::314f:5fe7:4e4b:64ed/64 scope link 
    valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
  link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff
  inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
    valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
  link/ether 52:54:00:2b:74:aa brd ff:ff:ff:ff:ff:ff

Step 3 驗證服務

瀏覽并多次強制刷新地址: http://192.168.43.150 ,可以看到103和104多次交替顯示,并顯示Nginx-1,則表明 Master節點在進行web服務轉發。

 Step 4 關閉Master keepalived服務和Nginx服務,訪問Web服務觀察服務轉移情況

[[email protected] keepalived]# killall keepalived
[[email protected] keepalived]# service nginx stop

此時強制刷新192.168.43.150發現 頁面交替顯示103和104并顯示Nginx-2 ,VIP已轉移到192.168.43.102上,已證明服務自動切換到備份節點上。

Step 5 啟動Master Keepalived 服務和Nginx服務

此時再次驗證發現,VIP已被Master重新奪回,并頁面交替顯示 103和104,此時顯示Nginx-1

四、Keepalived搶占模式和非搶占模式

keepalived的HA分為搶占模式和非搶占模式,搶占模式即MASTER從故障中恢復后,會將VIP從BACKUP節點中搶占過來。非搶占模式即MASTER恢復后不搶占BACKUP升級為MASTER后的VIP。

非搶占模式配置:

1> 在vrrp_instance塊下兩個節點各增加了nopreempt指令,表示不爭搶vip

2> 節點的state都為BACKUP 兩個keepalived節點都啟動后,默認都是BACKUP狀態,雙方在發送組播信息后,會根據優先級來選舉一個MASTER出來。由于兩者都配置了nopreempt,所以MASTER從故障中恢復后,不會搶占vip。這樣會避免VIP切換可能造成的服務延遲。

以上就是本文的全部內容,希望對大家的學習有所幫助,也希望大家多多支持龍方網絡。

鄭重聲明:本文版權包含圖片歸原作者所有,轉載文章僅為傳播更多信息之目的,如作者信息標記有誤,請第一時間聯系我們([email protected])修改或刪除,多謝。

鄭重聲明:本文版權歸原作者所有,轉載文章僅為傳播更多信息之目的,如作者信息標記有誤,請第一時間聯系我們修改或刪除,多謝。

留言與評論(共有 0 條評論)
   
驗證碼:
彩票店