Rhel Linux multipath + OpenfIler raid iscsi推荐

前段时间学习了关于Redhat Linux上的多路径Multipath，以及为了方便学习在本机电脑上安装了Openfiler模拟存储，配置多块网卡实现多路径冗余识别存储。在这里以我个人的理解简单的描述我所认知的理论知识。

Multipath是Linux系统上开源免费的多路径软件，实现服务器可以通过多条物理路径连接到块存储。

Openfiler是底层以Linux系统模拟的存储架构，可以配置多块网卡，同时可以配置软件Raid。

多路径是避免设备之间单条链路的单点故障隐患，同时可以实现流量分摊，实现性能提升。

当因为主机HBA卡、线缆、交换机或者存储设备的RAID控制器故障等原因造成一条物理路径失效时,服务器可以将通过此物理路径的I/O转移到其他正常的物理路径上面,而应用程序不会觉察到底层的改变，从而提高系统的可用性。

在这里通过模拟现网环境配置Multipath实现多路径冗余；

实验环境：

RedhatEnterpriseLinux7.2//安装配置Multipath多路径识别ISCSI存储设备;Openfileresa-2.99.1-x86_64//配置多网卡，模拟配置ISCSI存储;

在VMworkstation上Openfiler系统已经安装完成并且配置了四块物理网卡用于实现存储多路径，相应的IP地址已经配置规划；

首先配置存储设备的磁盘阵列Raid，由于在这里用的是模拟存储设备没有相应的RAID控制器，所以在这里配置软件Raid，模拟现网中存储设备的配置；

在Openfiler中配置添加逻辑卷，以及分配相应的空间大小给卷组。在这里配置了四个lv磁盘组，为了便于后续Oracle ASM使用；

创建相应的ISCSI并关联LUN，并且配置允许192.168.60.0/24网段访问，在此Openfiler存储相应的配置就完成了；

在Redhat Linux上首先挂载并配置YUM源，安装iscsi initiator软件；

[root@bogon~]#mount/dev/sr0/mntmount:/dev/sr0iswrite-protected,mountingread-only[root@bogon~]#vi/etc/yum.conf[root@bogon~]#yumrepolistLoadedplugins:langpacks,product-id,search-disabled-repos,subscription-managerThissystemisnotregisteredtoRedHatSubscriptionManagement.Youcanusesubscription-managertoregister.Repository'local'ismissingnameinconfiguration,usingidlocal|4.1kB00:00:00(1/2):local/group_gz|136kB00:00:00(2/2):local/primary_db|3.6MB00:00:00repoidreponamestatuslocallocal4,620repolist:4,620[root@bogon~]#cd/mnt/Packages/[root@bogonPackages]#rpm-ivhiscsi-initiator-utils-6.2.0.873-32.el7.x86_64.rpmwarning:iscsi-initiator-utils-6.2.0.873-32.el7.x86_64.rpm:HeaderV3RSA/SHA256Signature,keyIDfd431d51:NOKEYPreparing...#################################[100%]packageiscsi-initiator-utils-6.2.0.873-32.el7.x86_64isalreadyinstalled

启动iSCSI相关服务，并且配置开机启动项；

[root@bogonPackages]#systemctllist-unit-files|grepiscsiiscsi-shutdown.servicestaticiscsi.serviceenablediscsid.servicedisablediscsiuio.servicedisablediscsid.socketenablediscsiuio.socketenabled[root@bogonPackages]#systemctlenableiscsi.service

扫描并识别ISCSI存储信息，由于Openfiler所用端口号为3260，所以在此扫描添加端口号；

[root@bogonPackages]#iscsiadm-mdiscovery-tst-p192.168.60.11:3260192.168.60.11:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06192.168.60.12:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06192.168.60.13:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06192.168.60.14:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06

修改配置ISCSI软件开机扫描信息；

vi/etc/iscsi/initiatorname.iscsiInitiatorName=iqn.2006-01.com.openfiler:tsn.6524fccc7d06

对扫描到的ISCSI识别存储进行登陆；

[root@bogonPackages]#iscsiadm-mnode-Tiqn.2006-01.com.openfiler:tsn.6524fccc7d06-p192.168.60.11:3260-lLogginginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.11,3260](multiple)Loginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.11,3260]successful.[root@bogonPackages]#iscsiadm-mnode-Tiqn.2006-01.com.openfiler:tsn.6524fccc7d06-p192.168.60.12:3260-lLogginginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.12,3260](multiple)Loginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.12,3260]successful.[root@bogonPackages]#iscsiadm-mnode-Tiqn.2006-01.com.openfiler:tsn.6524fccc7d06-p192.168.60.13:3260-lLogginginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.13,3260](multiple)Loginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.13,3260]successful.[root@bogonPackages]#iscsiadm-mnode-Tiqn.2006-01.com.openfiler:tsn.6524fccc7d06-p192.168.60.14:3260-lLogginginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.14,3260](multiple)Loginto[iface:default,target:iqn.2006-01.com.openfiler:tsn.6524fccc7d06,portal:192.168.60.14,3260]successful.

查看ISCSI当前登陆会话是否正常；

[root@bogonPackages]#iscsiadm-msessiontcp:[1]192.168.60.11:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06(non-flash)tcp:[2]192.168.60.12:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06(non-flash)tcp:[3]192.168.60.13:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06(non-flash)tcp:[4]192.168.60.14:3260,1iqn.2006-01.com.openfiler:tsn.6524fccc7d06(non-flash)

安装Multipath多路径软件，并配置开机自动启动；

[root@bogonPackages]#rpm-ivhdevice-mapper-libs-1.02.107-5.el7.x86_64.rpmwarning:device-mapper-libs-1.02.107-5.el7.x86_64.rpm:HeaderV3RSA/SHA256Signature,keyIDfd431d51:NOKEYPreparing...#################################[100%]packagedevice-mapper-libs-7:1.02.107-5.el7.x86_64isalreadyinstalled[root@bogonPackages]#rpm-ivhdevice-mapper-multipath-0.4.9-85.el7.x86_64.rpmwarning:device-mapper-multipath-0.4.9-85.el7.x86_64.rpm:HeaderV3RSA/SHA256Signature,keyIDfd431d51:NOKEYPreparing...#################################[100%]packagedevice-mapper-multipath-0.4.9-85.el7.x86_64isalreadyinstalled[root@bogonPackages]#systemctlenablemultipathd.service[root@bogonPackages]#systemctllist-unit-files|grepmultipathmultipathd.serviceenabled[root@bogonPackages]#modprobedm-multipath[root@bogonPackages]#modprobedm-round-robin[root@bogonPackages]#systemctlstartmultipathd.service

查看并生成Multipath配置文件；

[root@bogonPackages]#multipath-llJun0911:28:53|/etc/multipath.confdoesnotexist,blacklistingalldevices.Jun0911:28:53|Adefaultmultipath.conffileislocatedatJun0911:28:53|/usr/share/doc/device-mapper-multipath-0.4.9/multipath.confJun0911:28:53|Youcanrun/sbin/mpathconftocreateormodify/etc/multipath.conf[root@bogonPackages]#cp/usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf/etc/multipath.conf[root@bogonPackages]#multipathstart

查看多路径所产生的磁盘并获取其WWID，为配置Multipath.conf做准备；

[root@bogonPackages]#fdisk-l|grep"10.7GB"Disk/dev/sdb:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdc:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdd:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdf:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdg:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdh:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdj:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdk:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdl:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdn:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdo:10.7GB,10737418240bytes,20971520sectorsDisk/dev/sdp:10.7GB,10737418240bytes,20971520sectors[root@bogonPackages]#fdisk-l|grep"21.5GB"Disk/dev/sde:21.5GB,21474836480bytes,41943040sectorsDisk/dev/sdi:21.5GB,21474836480bytes,41943040sectorsDisk/dev/sdm:21.5GB,21474836480bytes,41943040sectorsDisk/dev/sdq:21.5GB,21474836480bytes,41943040sectors[root@bogonPackages]#ls-l/dev/disk/by-id/total0lrwxrwxrwx.1rootroot9Jun911:09ata-VMware_Virtual_SATA_CDRW_Drive_01000000000000000001- ../../sr0lrwxrwxrwx.1rootroot10Jun911:09dm-name-rhel-root- ../../dm-0lrwxrwxrwx.1rootroot10Jun911:09dm-name-rhel-swap- ../../dm-1lrwxrwxrwx.1rootroot10Jun911:09dm-uuid-LVM-lqbtrHjC1YDN1WRYptlr8UJDHwBSKqgAgdEPZJZeAyoqfX9sJjEm7gO9aIji71Gb- ../../dm-0lrwxrwxrwx.1rootroot10Jun911:09dm-uuid-LVM-lqbtrHjC1YDN1WRYptlr8UJDHwBSKqgAweG1tRVIKfpakJ0csD3TE1coXJbDB8r2- ../../dm-1lrwxrwxrwx.1rootroot10Jun911:09lvm-pv-uuid-JuUZYx-8bPC-zoP1-4riV-qVxw-4qhJ-2ddb6j- ../../sda2lrwxrwxrwx.1rootroot9Jun911:23scsi-14f504e46494c45524e77537957762d34737a312d4d716339- ../../sdnlrwxrwxrwx.1rootroot9Jun911:23scsi-14f504e46494c4552554c536b71612d474539552d4942357a- ../../sdolrwxrwxrwx.1rootroot9Jun911:23scsi-14f504e46494c45526a33566a58622d6b6d4b562d73776f64- ../../sdqlrwxrwxrwx.1rootroot9Jun911:23scsi-14f504e46494c45527748533276382d707978772d686c7a43- ../../sdp[root@bogonPackages]#multipath-v3..........14f504e46494c45524e77537957762d34737a312d4d71633933:0:0:0sdb8:161undef14f504e46494c4552554c536b71612d474539552d4942357a33:0:0:1sdc8:321undef14f504e46494c45527748533276382d707978772d686c7a4333:0:0:2sdd8:481undef14f504e46494c45526a33566a58622d6b6d4b562d73776f6433:0:0:3sde8:641undef14f504e46494c45524e77537957762d34737a312d4d71633934:0:0:0sdf8:801undef14f504e46494c4552554c536b71612d474539552d4942357a34:0:0:1sdg8:961undef14f504e46494c45527748533276382d707978772d686c7a4334:0:0:2sdh8:1121undef14f504e46494c45526a33566a58622d6b6d4b562d73776f6434:0:0:3sdi8:1281undef14f504e46494c45524e77537957762d34737a312d4d71633935:0:0:0sdj8:1441undef14f504e46494c4552554c536b71612d474539552d4942357a35:0:0:1sdk8:1601undef14f504e46494c45527748533276382d707978772d686c7a4335:0:0:2sdl8:1761undef14f504e46494c45526a33566a58622d6b6d4b562d73776f6435:0:0:3sdm8:1921undef14f504e46494c45524e77537957762d34737a312d4d71633936:0:0:0sdn8:2081undef14f504e46494c4552554c536b71612d474539552d4942357a36:0:0:1sdo8:2241undef14f504e46494c45527748533276382d707978772d686c7a4336:0:0:2sdp8:2401undef14f504e46494c45526a33566a58622d6b6d4b562d73776f6436:0:0:3sdq65:01undef

在此上面的查找WWID，可以识别为：

14f504e46494c45524e77537957762d34737a312d4d716339sdbsdfsdjsdn10.7GB14f504e46494c4552554c536b71612d474539552d4942357asdcsdgsdksdo10.7GB14f504e46494c45527748533276382d707978772d686c7a43sddsdhsdlsdp10.7GB14f504e46494c45526a33566a58622d6b6d4b562d73776f64sdesdisdmsdq21.5GB

紧接着就可以配置Multipath.conf配置文件了；

[root@bogonPackages]#vi/etc/multipath.confdefaults{user_friendly_namesyesfind_multipathsyes#blacklist{#wwid26353900f02796769#devnode"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"#devnode"^hd[a-z]"multipaths{multipath{wwid14f504e46494c45524e77537957762d34737a312d4d716339aliasoracle-gridpath_grouping_policymultibuspath_selector"round-robin0"failbackimmediaterr_weightprioritiesno_path_retry5}multipath{wwid14f504e46494c4552554c536b71612d474539552d4942357aaliasoracle-syspath_grouping_policymultibuspath_selector"round-robin0"failbackimmediaterr_weightprioritiesno_path_retry5}multipath{wwid14f504e46494c45527748533276382d707978772d686c7a43aliasoracle-archpath_grouping_policymultibuspath_selector"round-robin0"failbackimmediaterr_weightprioritiesno_path_retry5}multipath{wwid14f504e46494c45526a33566a58622d6b6d4b562d73776f64aliasoracle-datapath_grouping_policymultibuspath_selector"round-robin0"failbackimmediaterr_weightprioritiesno_path_retry5}}devices{device{vendor"openfiler"product"virtual-disk"path_grouping_policymultibuspath_checkerreadsector0path_selector"round-robin0"hardware_handler"0"failback15rr_weightprioritiesno_path_retryqueue}}

配置完成后重新启动Multipath并检查是否正常；

[root@bogonPackages]#multipathstop[root@bogonPackages]#multipathstart[root@bogonPackages]#multipath-F[root@bogonPackages]#multipath-v2create:oracle-grid(14f504e46494c45524e77537957762d34737a312d4d716339)undefOPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=undef`-+-policy='round-robin0'prio=1status=undef|-33:0:0:0sdb8:16undefreadyrunning|-34:0:0:0sdf8:80undefreadyrunning|-35:0:0:0sdj8:144undefreadyrunning`-36:0:0:0sdn8:208undefreadyrunningcreate:oracle-sys(14f504e46494c4552554c536b71612d474539552d4942357a)undefOPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=undef`-+-policy='round-robin0'prio=1status=undef|-33:0:0:1sdc8:32undefreadyrunning|-34:0:0:1sdg8:96undefreadyrunning|-35:0:0:1sdk8:160undefreadyrunning`-36:0:0:1sdo8:224undefreadyrunningcreate:oracle-arch(14f504e46494c45527748533276382d707978772d686c7a43)undefOPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=undef`-+-policy='round-robin0'prio=1status=undef|-33:0:0:2sdd8:48undefreadyrunning|-34:0:0:2sdh8:112undefreadyrunning|-35:0:0:2sdl8:176undefreadyrunning`-36:0:0:2sdp8:240undefreadyrunningcreate:oracle-data(14f504e46494c45526a33566a58622d6b6d4b562d73776f64)undefOPNFILER,VIRTUAL-DISKsize=20Gfeatures='0'hwhandler='0'wp=undef`-+-policy='round-robin0'prio=1status=undef|-33:0:0:3sde8:64undefreadyrunning|-34:0:0:3sdi8:128undefreadyrunning|-35:0:0:3sdm8:192undefreadyrunning`-36:0:0:3sdq65:0undefreadyrunning[root@bogonPackages]#multipath-lloracle-sys(14f504e46494c4552554c536b71612d474539552d4942357a)dm-3OPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=rw`-+-policy='round-robin0'prio=1status=active|-33:0:0:1sdc8:32activereadyrunning|-34:0:0:1sdg8:96activereadyrunning|-35:0:0:1sdk8:160activereadyrunning`-36:0:0:1sdo8:224activereadyrunningoracle-data(14f504e46494c45526a33566a58622d6b6d4b562d73776f64)dm-5OPNFILER,VIRTUAL-DISKsize=20Gfeatures='0'hwhandler='0'wp=rw`-+-policy='round-robin0'prio=1status=active|-33:0:0:3sde8:64activereadyrunning|-34:0:0:3sdi8:128activereadyrunning|-35:0:0:3sdm8:192activereadyrunning`-36:0:0:3sdq65:0activereadyrunningoracle-arch(14f504e46494c45527748533276382d707978772d686c7a43)dm-4OPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=rw`-+-policy='round-robin0'prio=1status=active|-33:0:0:2sdd8:48activereadyrunning|-34:0:0:2sdh8:112activereadyrunning|-35:0:0:2sdl8:176activereadyrunning`-36:0:0:2sdp8:240activereadyrunningoracle-grid(14f504e46494c45524e77537957762d34737a312d4d716339)dm-2OPNFILER,VIRTUAL-DISKsize=10Gfeatures='0'hwhandler='0'wp=rw`-+-policy='round-robin0'prio=1status=active|-33:0:0:0sdb8:16activereadyrunning|-34:0:0:0sdf8:80activereadyrunning|-35:0:0:0sdj8:144activereadyrunning`-36:0:0:0sdn8:208activereadyrunning

检查相应的多路径对应是否正常；

[root@bogonPackages]#cd/dev/mapper/[root@bogonmapper]#ls-lsatotal00drwxr-xr-x.2rootroot180Jun912:05.0drwxr-xr-x.20rootroot3940Jun912:05..0crw-------.1rootroot10,236Jun911:09control0lrwxrwxrwx.1rootroot7Jun912:05oracle-arch- ../dm-40lrwxrwxrwx.1rootroot7Jun912:05oracle-data- ../dm-50lrwxrwxrwx.1rootroot7Jun912:05oracle-grid- ../dm-20lrwxrwxrwx.1rootroot7Jun912:05oracle-sys- ../dm-30lrwxrwxrwx.1rootroot7Jun911:09rhel-root- ../dm-00lrwxrwxrwx.1rootroot7Jun911:09rhel-swap- ../dm-1

格式化相应的磁盘，为后续ASM磁盘使用；

[root@bogonmapper]#fdisk-c/dev/mapper/oracle-arch[root@bogonmapper]#fdisk-c/dev/mapper/oracle-data[root@bogonmapper]#fdisk-c/dev/mapper/oracle-grid[root@bogonmapper]#fdisk-c/dev/mapper/oracle-sys[root@bogonmapper]#kpartx-a/dev/mapper/oracle-arch[root@bogonmapper]#kpartx-a/dev/mapper/oracle-sys[root@bogonmapper]#kpartx-a/dev/mapper/oracle-data[root@bogonmapper]#kpartx-a/dev/mapper/oracle-grid[root@bogonmapper]#ls-lsatotal00drwxr-xr-x.2rootroot260Jun912:22.0drwxr-xr-x.20rootroot4020Jun912:22..0crw-------.1rootroot10,236Jun911:09control0lrwxrwxrwx.1rootroot7Jun912:16oracle-arch- ../dm-40lrwxrwxrwx.1rootroot7Jun912:21oracle-arch1- ../dm-60lrwxrwxrwx.1rootroot7Jun912:17oracle-data- ../dm-50lrwxrwxrwx.1rootroot7Jun912:22oracle-data1- ../dm-80lrwxrwxrwx.1rootroot7Jun912:17oracle-grid- ../dm-20lrwxrwxrwx.1rootroot7Jun912:22oracle-grid1- ../dm-90lrwxrwxrwx.1rootroot7Jun912:17oracle-sys- ../dm-30lrwxrwxrwx.1rootroot7Jun912:22oracle-sys1- ../dm-70lrwxrwxrwx.1rootroot7Jun911:09rhel-root- ../dm-00lrwxrwxrwx.1rootroot7Jun911:09rhel-swap- ../dm-1

在此，实验所用的模拟存储Openfiler使用四块物理网卡，配置软件Raid，逻辑卷配置多块Lun。Redhat Enterprise Linux配置Multipath多路径，所有均已实现。

相关文章：

你感兴趣的文章：

标签云：