下面是一个丑到爆的新方案架构图,已经在公司测试环境的部署,并且进行了不完全充分的测试。 架构拓扑: 简单描述: 两台 NFS 服务器,通过 em1 网卡与内网的其他业务服务器进行通信,em2网卡主要负责两台 NFS 服务器之间心跳通信,em3网卡主要负责drbd数据同步的传输。 前面的2台图片服务器通过 NFS 集群提供出来的一个VIP 192.168.0.219 来使用 NFS 集群服务。 一、项目基础设施及信息介绍 1、设备信息 现有的两台 NFS 存储服务器的硬件配置信息: CPU: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz MEM: 16G Raid: RAID 1 Disk: SSD 200G x 2 网卡:集成的 4 个千兆网卡 Link is up at 1000 Mbps, full duplex 前端两台静态图片服务器硬件配置信息: 略 2、网络 浮动 VIP : 192.168.0.219 # 漂浮在M1和M2上,负责对外提供服务 现有的两台 NFS 存储服务器的网络配置信息: 主机名:M1.redhat.sx em1:192.168.0.210 内网 em2:172.16.0.210 心跳线 em3:172.16.100.210 DRBD千兆数据传输 主机名:M2.redhat.sx em1:192.168.0.211 内网 em2:172.16.0.211 心跳线 em3:172.16.100.211 DRBD千兆数据传输 3、系统环境 内核版本:2.6.32-504.el6.x86_64 系统版本:CentOS 6.5 系统位数:x86_64 防火墙规则清空 selinux关闭 4、软件版本 heartbeat-3.0.4-2.el6.x86_64 drbd-8.4.3 rpcbind-0.2.0-11.el6.x86_64 nfs-utils-1.2.3-54.el6.x86_64 二、基础服务配置 这里仅以 M1 服务的配置为例,M2 服务器配置与此相同。 1、配置时间同步 M1端: [root@M1 ~]# ntpdate pool.ntp.org 12 Nov 14:45:15 ntpdate[27898]: adjust time server 42.96.167.209 offset 0.044720 secM2端: [root@M2 ~]# ntpdate pool.ntp.org 12 Nov 14:45:06 ntpdate[24447]: adjust time server 42.96.167.209 offset 0.063174 sec2、配置/etc/hosts文件 M1端: [root@M1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.210 M1.redhat.sx 192.168.0.211 M2.redhat.sxM2端: [root@M2 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.210 M1.redhat.sx 192.168.0.211 M2.redhat.sx3、增加主机间路由 首先先验证 M1 和 M2 的服务器 IP 是否合乎规划 M1端: [root@M1 ~]# ifconfig|egrep 'Link encap|inet addr' # 验证现有 IP 信息 em1 Link encap:Ethernet HWaddr B8:CA:3A:F1:00:2F inet addr:192.168.0.210 Bcast:192.168.0.255 Mask:255.255.255.0 em2 Link encap:Ethernet HWaddr B8:CA:3A:F1:00:30 inet addr:172.16.0.210 Bcast:172.16.0.255 Mask:255.255.255.0 em3 Link encap:Ethernet HWaddr B8:CA:3A:F1:00:31 inet addr:172.16.100.210 Bcast:172.16.100.255 Mask:255.255.255.0 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0M2端: [root@M2 ~]# ifconfig|egrep 'Link encap|inet addr' em1 Link encap:Ethernet HWaddr B8:CA:3A:F1:DE:37 inet addr:192.168.0.211 Bcast:192.168.0.255 Mask:255.255.255.0 em2 Link encap:Ethernet HWaddr B8:CA:3A:F1:DE:38 inet addr:172.16.0.211 Bcast:172.16.0.255 Mask:255.255.255.0 em3 Link encap:Ethernet HWaddr B8:CA:3A:F1:DE:39 inet addr:172.16.100.211 Bcast:172.16.100.255 Mask:255.255.255.0 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0查看现有路由,然后增加相应的心跳线和drbd数据传输线路的端到端的静态路由条目。目的是为了让心跳检测和数据同步不受干扰。 M1端: [root@M1 network-scripts]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 em3 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em2 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 em2 169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 em3 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 em1 [root@M1 network-scripts]# /sbin/route add -host 172.16.0.211 dev em2 [root@M1 network-scripts]# /sbin/route add -host 172.16.100.211 dev em3 [root@M1 network-scripts]# echo '/sbin/route add -host 172.16.0.211 dev em2' >> /etc/rc.local [root@M1 network-scripts]# echo '/sbin/route add -host 172.16.100.211 dev em3' >> /etc/rc.local [root@M1 network-scripts]# tail -2 /etc/rc.local /sbin/route add -host 172.16.0.211 dev em1 /sbin/route add -host 172.16.100.211 dev em1 [root@M1 network-scripts]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.0.211 0.0.0.0 255.255.255.255 UH 0 0 0 em2 172.16.100.211 0.0.0.0 255.255.255.255 UH 0 0 0 em3 172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 em3 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em2 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 em2 169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 em3 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 em1 [root@M1 network-scripts]# traceroute 172.16.0.211 traceroute to 172.16.0.211 (172.16.0.211), 30 hops max, 60 byte packets 1 172.16.0.211 (172.16.0.211) 0.820 ms 0.846 ms 0.928 ms [root@M1 network-scripts]# traceroute 172.16.100.211 traceroute to 172.16.100.211 (172.16.100.211), 30 hops max, 60 byte packets 1 172.16.100.211 (172.16.100.211) 0.291 ms 0.273 ms 0.257 msM2端: [root@M2 network-scripts]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 em3 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em2 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 em2 169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 em3 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 em1 [root@M2 network-scripts]# /sbin/route add -host 172.16.0.210 dev em2 [root@M2 network-scripts]# /sbin/route add -host 172.16.100.210 dev em3 [root@M2 network-scripts]# echo '/sbin/route add -host 172.16.0.210 dev em2' >> /etc/rc.local [root@M2 network-scripts]# echo '/sbin/route add -host 172.16.100.210 dev em3' >> /etc/rc.local [root@M2 network-scripts]# tail -2 /etc/rc.local /sbin/route add -host 172.16.0.210 dev em1 /sbin/route add -host 172.16.100.210 dev em1 [root@M2 network-scripts]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.0.210 0.0.0.0 255.255.255.255 UH 0 0 0 em2 172.16.100.210 0.0.0.0 255.255.255.255 UH 0 0 0 em3 172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 em3 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em2 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 em1 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 em2 169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 em3 0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 em1 [root@M2 network-scripts]# traceroute 172.16.0.210 traceroute to 172.16.0.210 (172.16.0.210), 30 hops max, 60 byte packets 1 172.16.0.210 (172.16.0.210) 0.816 ms 0.843 ms 0.922 ms [root@M2 network-scripts]# traceroute 172.16.100.210 traceroute to 172.16.100.210 (172.16.100.210), 30 hops max, 60 byte packets 1 172.16.100.210 (172.16.100.210) 0.256 ms 0.232 ms 0.215 ms 转载请保留固定链接: https://linuxeye.com/architecture/2045.html |