LinuxEye - Linux系统教程

LinuxEye - Linux系统教程

当前位置: 主页 > Linux配置 >

基于RHCS+iscsi+glvm构建IP SAN共享存储配置

时间:2012-12-03 10:26来源:51cto 编辑:老广 点击:
目的: 四台主机,rs1、rs2、rs3、target,rs1,rs2,rs3提供http服务,target主机把自己的磁盘做成IP SAN 的target供rs1,rs2,rs3使用 规划: rs1 172.16.1.11 hostname rs1.laoguang.mers2 172.16.1.12 hostname rs2.laoguang.me
目的:
四台主机,rs1、rs2、rs3、target,rs1,rs2,rs3提供http服务,target主机把自己的磁盘做成IP SAN 的target供rs1,rs2,rs3使用

规划:
rs1 172.16.1.11 hostname rs1.laoguang.me
rs2 172.16.1.12 hostname rs2.laoguang.me
rs3 172.16.1.13 hostname rs3.laoguang.me
target 172.16.1.14 hostname target.laoguang.me
VIP 172.16.1.1
  

一.先构建我们的RHCS高可用集群

准备工作:hostname设置,/etc/hosts下做好解析,ssh 互信,date统一,这是所有集群必须具备的。

1.1 配置准备工作.在rs1上演示,target不是高可用集群中的一个节点,可不配置,不过建议配置
配置hostname
hostname rs1.laoguang.me
vim /etc/hosts
172.16.1.11     rs1.laoguang.me rs1  
172.16.1.12     rs2.laoguang.me rs2  
172.16.1.13     rs3.laoguang.me rs3
172.16.1.14     target.laoguang.me      target

配置互信
ssh-keygen -t rsa -P ""    ##会生成密钥在/root/.ssh下,公钥拷贝到rs2的/root/.ssh/authorized_keys
ssh-copy-id -i /root/.ssh/id_rsa.pub rs2    ##rs1登录rs2不再需要密码
ssh-copy-id -i /root/.ssh/id_rsa.pub target

时间同步:(如果有NTP就同步NTP的时间)
date -s 20121127;ssh rs2 'date -s 20121127';ssh rs3 'date -s 20121127';\
ssh target 'date -s 20121127'

1.2 rs1,rs2,rs3安装httpd
yum -y install httpd

建立测试页面rs1,rs2,rs3上分别为
echo "<h1>RS1</h1>" > /var/www/html/index.html
echo "<h1>RS2</h1>" > /var/www/html/index.html
echo "<h1>RS3</h1>" > /var/www/html/index.html

启动服务测试各各网站是否运行正常,如果正常继续。
service httpd start

关闭httpd,并关闭的开机自启动
service httpd stop
chkconfig httpd off

1.3 安装RHCS套件需要的openais,cman,rgmanager,gfs2-utils(集群文件系统),system-config-cluster(此为GUI配置工具)
yum -y install cman rgmanager gfs2-utils

1.4 建立集群WebCluster
ccs_tool create WebCluster   ##会在/etc/cluster中生成cluster.conf集群的配置文件(xml格式)

1.5 加入node到WebCluster集群
ccs_tool addfence Manual fence_manual  ##RHCS需要先定义隔离设备
ccs_tool addnode -v 1 -n 1 -f Manual rs1.laoguang.me  ##加入rs1
ccs_tool addnode -v 1 -n 2 -f Manual rs2.laoguang.me  ##加入rs2
ccs_tool addnode -v 1 -n 3 -f Manual rs3.laoguang.me  ##加入rs3
-v 指定每个节点的票数
-n 指定节点ID
-f 指定fence设备

1.6 rs1,rs2,rs3上启动cman
service cman start;     
##注:如果局域网内多台机器玩cman的话cman可能启动不成功,其它机器可能是同步的其它主机的CIB
service rgmanager start; ##启动rgmanager

1.7 启动system-config-cluster 配置cluster ##由于cman的配置文件为xml格式,笔者还不会用
system-config-cluster

1.7.1 定义故障转移域


1.7.2 定义IP资源与httpd资源                                                                     


 

httpd资源                                                                                    

                                                                               
1.7.3 定义服务                                                                                  

                                                                                   
查看集群状态
clustat 
Cluster Status for WebCluster @ Wed Nov 28 18:38:19 2012                                    
Member Status: Quorate                                                                      
                                                                                            
Member Name                                     ID   Status                                 
------ ----                                     ---- ------                                 
rs1.laoguang.me                                     1 Online, rgmanager                     
rs2.laoguang.me                                     2 Online, rgmanager                     
rs3.laoguang.me                                     3 Online, Local, rgmanager              
                                                                                            
Service Name                           Owner (Last)                State                    
------- ----                           ----- ------                    -----                
service:WebService                     rs1.laoguang.me                 started
                                       
由此可知WebService在rs1上运行,访问http://172.16.1.1测试                                        
                                                                   

二,target中安装scsi-target-utils                                                                                                                                                            

2.1 安装rpm包                                                                                                                                                                                   
yum -y install scsi-target-utils 
                                                         
                                                                                                
2.2 建立一2G分区,我建的是/dev/sda5  ##不要格式化,记得partprobe                                
                                                                                                
2.3 编辑配置文件/etc/tgt/targets.conf,添加一个target,如下
<target iqn.2012-11.me.laoguang.target:sda5>                                                
    backing-store /dev/sda5           ##后备存储,一般指一个分区,或lvm设备             
    initiator-address 172.16.0.0/16   ##指定intiaor只能是这个地址范围                   
    #direct-store /dev/sdb          ##直接存储,通常指一个磁盘如/dev/sda5               
    #incominguser hadoop redhat    ##定义基于账户认证Initiator,后面的是账号与密码       
    #outgoinguser userA secretpassA  ##Initiator也可以验证target,这个target的账号密码  
</target>                                                                               
##targetname 格式 iqn-name: iqn.yyyy-mm.<reversed domain name>[identifer]
我们也可以有命令行来配置,只不过重启后会失效哦                                          
用法:tgtadm --lld [driver] --op [operation] --mode [mode] [OPTION]..                    
新增target:                                                                            
tgtadm --lld iscsi --op new --mode target --tid 1 --targetname iqn.2012-11.me.laoguang.target:sda5
tgtadm --lld iscsi --op show --mode target                                              
新增一localunit:                                                                        
tgtadm --lld -iscsi --op new --mode localunit --tid 1 --lun 1 --backing-store /dev/sda5
新增ACL允许172.16.0.0/16网段使用:                                                      
tgtadm --lld iscsi --op bind --tid 1 --initiator-address 172.16.0.0/16                  
新增账户认证:                                                                           
建立账户:                                                                              
tgtadm --lld iscsi --op new --mode account  --user hadoop --password redhat             
查看新建账户:                                                                           
tgtadm --lld iscsi --op show --mode account                                             
绑定账户到target:                                                                      
tgtadm --lld iscsi --op bind --mode account --tid 1 --user hadoop                       
查看target信息:                                                                         
tgtadm --lld iscsi --op show --mode target 
                                                                                    
2.4 启动tgtd服务,验证target是否分享出去
service tgtd start                                                                          
tgtadm --lld iscsi --op show --mode target                                                  
                                                                                            
    Target 1: iqn.2012-11.me.laoguang.target:sda5                                           
    System information:                                                                     
        Driver: iscsi                                                                       
        State: ready                                                                        
    I_T nexus information:                                                                  
    LUN information:                                                                        
        LUN: 0                                                                              
            Type: controller                                                                
            SCSI ID: IET     00010000                                                       
            SCSI SN: beaf10                                                                 
            Size: 0 MB, Block size: 1                                                       
            Online: Yes                                                                     
            Removable media: No                                                             
            Readonly: No                                                                    
            Backing store type: null                                                        
            Backing store path: None                                                        
            Backing store flags:                                                            
        LUN: 1                                                                              
            Type: disk                                                                      
            SCSI ID: IET     00010001                                                       
            SCSI SN: beaf11                                                                 
            Size: 1012 MB, Block size: 512                                                  
            Online: Yes                                                                     
            Removable media: No                                                             
            Readonly: No                                                                    
            Backing store type: rdwr                                                        
            Backing store path: /dev/sda5                                                   
            Backing store flags:                                                            
    Account information:                                                                    
    ACL information:                                                                        
        172.16.0.0/16
                                                                   
验证正常,禁止tgt开机启动
chkconfig tgtd off
                                                 

三,在rs1,rs2,rs3上安装iscsi-initiator-utils

yum -y install iscsi-initator-utils ##很可能已经安装了                                      
echo "InitiatorAlias=rs1" >> /etc/iscsi/initiatorname.iscsi  ##分别给个别名                 
echo "InitiatorAlias=rs2" >> /etc/iscsi/initiatorname.iscsi                                 
echo "InitiatorAlias=rs3" >> /etc/iscsi/initiatorname.iscsi 


3.1 启动iscsi,发现共享磁盘target

service iscsi start                                                                         
iscsiadm -m discovery -t sendtargets -p 172.16.1.14   ##发现target                          
172.16.1.14:3260,1 iqn.2012-11.me.laoguang.target:sda5
                                                  
3.2 挂载(登录)共享磁盘
iscsiadm -m node -T  iqn.2012-11.me.laoguang.target:sda5 -p 172.16.1.14 -l                  
-m 指定模式 man 一下                                                                        
-t sendtargets 在discovery模式下使用                                                        
-p portal 指定target的IP与端口,如果是默认端口3260可省略                                    
-l login  登录                                                                              
-u logout 个为登出
fdisk -l   ##查看新磁盘我的是sda5
                                                                                           
rs1,rs2,rs3同样登录                                                                             
                                                                                                
禁止它们开机启动
chkconfig iscsi off;chkconfig iscsid off

3.3 安装lvm2-cluster,让gfs2支持卷组
lvmconf --enable-cluster                                                                    
service clvmd start 

3.4 rs1上对/data/sdb创建lvm卷组
pvcreate /dev/sdb                                                                           
vgcreate myvg /dev/sdb                                                                      
lvcreate -L 800M --name mydata myvg

查看其它节点中是否生成了lvm
ls /dev/myvg/mydata

3.5 rs1,rs2,rs3启动gfs2,在rs1中格式化gfs格式的磁盘
mkfs.gfs2 -p lock_dlm -t WebCluster:mylvm -j 3 /dev/myvg/mydata

3.6 rs1,rs2,rs3挂载/dev/myvg/mydata到/var/www/html中,建立测试文件
mount /dev/myvg/mydata /var/www/html                                                        
echo "GFS Storage" > /var/www/html/index.html

rs1,rs2同时打开/var/www/html/index.html 查看是否有上锁警告                                                                                                                                     
访问http://172.16.1.1查看是否能显示新主页

转载请保留固定链接: https://linuxeye.com/configuration/987.html

------分隔线----------------------------
标签:iSCSIRHCSglvm
栏目列表
推荐内容