1.软raid的实现方式：
软Raid本身是一个软件程序，这个程序是内核模块MD来模拟实现的。由于它的运行是依赖于操作系统的，软件程序和系统出现故障，RAID就有可能出现问题。性能上不及硬件RAID，还会对CPU造成一定的压力。在生产环境中，不建议使用软Raid，使用raid卡或者Raid芯片实现Raid功能。Linux软体RAID设备通过MD(多个设备)设备驱动程序来实现。
2.mdadm命令介绍：
mdadm:模式化的命令
mdadm [mode] raiddevice [options] component-devices
-A：–assemble 装配模式,组装一个预先存在的数组。
-C：–build 创建模式,创建一个新的数组
-F：–follow,–monitor 监控模式,选择监控模式。
-G: –grow 改变的有源阵列的大小或形状。
-I: –incremental 添加/从适当的数组中删除单个器件/，并可能启动阵列。
创建的软Raid：/dev/md0, /dev/md1, /dev/md2…
-a {yes|no}:是否为新建的raid设备自动创建设备文件/dev/md#
-l #：指定Raid级别；
-n #:指定用于Raid的块设备的个数，不包括备盘；
-c:指定chunks块的大小
-x #:指定用于备盘的块设备个数；
DEVICE…
-D，–detail :详细信息 显示一个Raid设备的详细信息。
用法示例：
mdadm -D /dev/md0 等同于mdadm -detail /dev/md0
设备文件：
major:主设备号，用于区分设备类别;
minor:此设备号，用于区分同一种类别下不同的具体设备；
一、通过mdadm命令，调用内核MD模块创建Raid0
1.首先，在虚拟机上新添加两块20G的物理硬盘，用来做在不同磁盘上的Raid。
注意：在同一块磁盘的不同分区上，不能做raid，当磁盘出现故障，Raid就无法使用了。做Raid的时候，要在不同的物理硬盘上。
1).查看添加好的物理硬盘。
[root@Vmware5 ~]# fdisk -l /dev/sdbDisk /dev/sdb: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000[root@Vmware5 ~]# fdisk -l /dev/sdcDisk /dev/sdc: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000
2).创建一个5G大小的sdb1新分区，文件分区类型为fd。
[root@Vmware5 ~]# fdisk /dev/sdbDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x19603682.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)Partition number (1-4): 1First cylinder (1-2610, default 1):Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +5GCommand (m for help): tSelected partition 1Hex code (type L to list codes): fdChanged system type of partition 1 to fd (Linux raid autodetect)Command (m for help): pDisk /dev/sdb: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x19603682 Device Boot Start End Blocks Id System/dev/sdb1 1 654 5253223+ fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.
3).创建一个大小为5G的sdc1新分区，文件分区类型为fd。
[root@Vmware5 ~]# fdisk /dev/sdcDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x07b84e42.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)Partition number (1-4): 1First cylinder (1-2610, default 1):Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +5GCommand (m for help): tSelected partition 1Hex code (type L to list codes): fdChanged system type of partition 1 to fd (Linux raid autodetect)Command (m for help): pDisk /dev/sdc: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x07b84e42 Device Boot Start End Blocks Id System/dev/sdc1 1 654 5253223+ fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.
注意：创建好分区之后，不要格式化，格式化的是创建好的软Raid,不是它下边的分区。
4).查看下创建好的分区。
[root@Vmware5 ~]# fdisk -l /dev/sd{b,c}1Disk /dev/sdb1: 5379 MB, 5379300864 bytes255 heads, 63 sectors/track, 653 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdc1: 5379 MB, 5379300864 bytes255 heads, 63 sectors/track, 653 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000000005）
5).分别添加/dev/sdb1、/dev/sdc1到/dev/sdb、/dev/sdc。
[root@Vmware5 ~]# partx -a /dev/sdb1 /dev/sdb[root@Vmware5 ~]# partx -a /dev/sdc1 /dev/sdc[root@Vmware5 ~]# cat /proc/partitionsmajor minor #blocks name 8 0 10485760 sda 8 1 204800 sda1 8 2 7920640 sda2 8 3 2097152 sda3 8 4 1 sda4 8 5 261120 sda5 8 16 20971520 sdb 8 17 5253223 sdb1 8 32 20971520 sdc 8 33 5253223 sdc1
6).看下有没有其他设备，md0是否被使用。
[root@Vmware5 ~]# cat /proc/mdstatPersonalities :unused devices: none
7).创建软Raid,名字为md0。
[root@Vmware5 ~]# mdadm -C /dev/md0 -a yes -l 0 -n 2 /dev/sdb1 /dev/sdc1mdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md0 started.
8).看下是否有了md0这个软Raid设备。
[root@Vmware5 ~]# cat /proc/mdstatPersonalities : [raid0]md0 : active raid0 sdc1[1] sdb1[0] 10506240 blocks super 1.2 512k chunksunused devices: none
9).软Raid0的详细信息。
[root@Vmware5 ~]# ls /dev/mdmd-device-map[root@Vmware5 ~]# cat /dev/md/md-device-mapmd0 1.2 e002e742:690d9a6f:f5fe3e08:bdeac269 /dev/md0
10).在上文提到过，分好区不要急着格式化/dev/sdb1和/dev/sdc1,此时需格式化/dev/md0就行了。
[root@Vmware5 ~]# mke2fs -t ext4 /dev/md0mke2fs 1.41.12 (17-May-2010)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=128 blocks, Stripe width=256 blocks657072 inodes, 2626560 blocks131328 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=269274316881 block groups32768 blocks per group, 32768 fragments per group8112 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632Writing inode tables: done Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 22 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.
11).创建一个挂在目录，名字为tt1，然后把格式化好的md0分区挂载到tt1上去。
[root@Vmware5 ~]# mkdir -p /tt1[root@Vmware5 ~]# mount /dev/md0 /tt1[root@Vmware5 ~]# mount/dev/sda2 on / type ext4 (rw)proc on /proc type proc (rw)sysfs on /sys type sysfs (rw)devpts on /dev/pts type devpts (rw,gid=5,mode=620)tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")/dev/sda1 on /boot type ext4 (rw)/dev/sda3 on /home type ext4 (rw)none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)/dev/md0 on /tt1 type ext4 (rw)[root@Vmware5 ~]# cd /tt1/[root@Vmware5 tt1]# lslost+found
12).监控md0的详细信息。
[root@Vmware5 tt1]# mdadm -D /dev/md0/dev/md0: Version : 1.2 版本 Creation Time : Fri Oct 25 05:59:07 2013 创建时间 Raid Level : raid0 Raid类型 Array Size : 10506240 (10.02 GiB 10.76 GB) raid大小 Raid Devices : 2 ##一共有几个设备创建的Raid Total Devices : 2 ##一共有多少设备 Persistence : Superblock is persistent ##超级快是否是持久的 Update Time : Fri Oct 25 05:59:07 2013 ##更新时间 State : clean ##数据是完整地 Active Devices : 2Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Chunk Size : 512K ##Chunk的大小 Name : Vmware5:0 (local to host Vmware5) 当前主机名 UUID : 42e702e0:6f9a0d69:083efef5:69c2eabd ##设备的UUID Events : 0 Number Major Minor RaidDevice State ##标识号码，主设备号，次设备号 以及磁盘状态 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1
13).拷贝到tt1目录一下文件，看下是否可以正常查看。
[root@Vmware5 ~]# cp /etc/fstab /tt1/[root@Vmware5 ~]# ls /tt1fstab lost+found[root@Vmware5 ~]# cat /tt1/fstab# /etc/fstab# Created by anaconda on Thu Oct 24 23:49:23 2013# Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more infoUUID=b49ee2b3-75aa-4227-a9ff-5d0d990af0fd / ext4 defaults 1 1UUID=3a69daa4-b393-4694-abbb-b856345b376d /boot ext4 defaults 1 2UUID=34f85ed8-5f68-4fdc-8aa0-e50d2f9f012e /home ext4 defaults 1 2UUID=95d97c70-9291-499b-ac16-a38508a85e4d swap swap defaults 0 0tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0
可以正常查看，说明软Raid0做成功了。
二、通过mdadm命令，创建一个大小为10G的软Raid1。
1).创建一个大小为10G的sdb2新分区，文件分区类型为fd。
[root@Vmware5 ~]# fdisk /dev/sdbWARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)Partition number (1-4): 2First cylinder (655-2610, default 655):Using default value 655Last cylinder, +cylinders or +size{K,M,G} (655-2610, default 2610): +5GCommand (m for help): tPartition number (1-4): 2Hex code (type L to list codes): fdChanged system type of partition 2 to fd (Linux raid autodetect)Command (m for help): pDisk /dev/sdb: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x35c1f03a Device Boot Start End Blocks Id System/dev/sdb1 1 654 5253223+ fd Linux raid autodetect/dev/sdb2 655 1308 5253255 fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.WARNING: Re-reading the partition table failed with error 16: Device or resource busy.The kernel still uses the old table. The new table will be used atthe next reboot or after you run partprobe(8) or kpartx(8)Syncing disks.
2).创建一个大小为10G的sdc2新分区，文件分区类型为fd。
[root@Vmware5 ~]# fdisk /dev/sdcWARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): nCommand action e extended p primary partition (1-4)Partition number (1-4): 2First cylinder (655-2610, default 655):Using default value 655Last cylinder, +cylinders or +size{K,M,G} (655-2610, default 2610): +10GCommand (m for help): tPartition number (1-4): 2Hex code (type L to list codes): fdChanged system type of partition 2 to fd (Linux raid autodetect)Command (m for help): pDisk /dev/sdc: 21.5 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x87e3e792 Device Boot Start End Blocks Id System/dev/sdc1 1 654 5253223+ fd Linux raid autodetect/dev/sdc2 655 1960 10490445 fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.WARNING: Re-reading the partition table failed with error 16: Device or resource busy.The kernel still uses the old table. The new table will be used atthe next reboot or after you run partprobe(8) or kpartx(8)Syncing disks.
3).分别添加/dev/sdb2、/dev/sdc2到/dev/sdb、/dev/sdc。
[root@Vmware5 ~]# partx -a /dev/sdb2 /dev/sdb[root@Vmware5 ~]# partx -a /dev/sdc2 /dev/sdc
4).用mdadm命令，创建软Raid1，名字为md1。
[root@Vmware5 ~]# mdadm -C /dev/md1 -a yes -l 1 -n 2 /dev/sd{b,c}2mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90mdadm: largest drive (/dev/sdc2) exceeds size (5249088K) by more than 1%Continue creating array? yesmdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md1 started.[root@Vmware5 ~]# cat /proc/mdstatPersonalities : [raid0] [raid1]md1 : active raid1 sdc2[1] sdb2[0] 5249088 blocks super 1.2 [2/2] [UU]md127 : active raid0 sdb1[0] sdc1[1] 10506240 blocks super 1.2 512k chunksunused devices: none
5).格式化新建的软raid1名字为md1。
[root@Vmware5 ~]# mke2fs -t ext4 /dev/md1 -L MD1mke2fs 1.41.12 (17-May-2010)Filesystem label=MD1OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=0 blocks, Stripe width=0 blocks328656 inodes, 1312272 blocks65613 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=134637158441 block groups32768 blocks per group, 32768 fragments per group8016 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736Writing inode tables: done Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 20 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.
6).用mount命令，临时挂载到/mnt目录下。
[root@Vmware5 ~]# mount /dev/md1 /mnt/[root@Vmware5 ~]# mount/dev/sda2 on / type ext4 (rw)proc on /proc type proc (rw)sysfs on /sys type sysfs (rw)devpts on /dev/pts type devpts (rw,gid=5,mode=620)tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")/dev/sda1 on /boot type ext4 (rw)/dev/sda3 on /home type ext4 (rw)none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)/dev/md1 on /mnt type ext4 (rw)
7).看下md1的监控信息。
[root@Vmware5 ~]# mdadm -D /dev/md1/dev/md1: Version : 1.2 Creation Time : Fri Oct 25 09:08:45 2013 Raid Level : raid1 Array Size : 5249088 (5.01 GiB 5.38 GB) Used Dev Size : 5249088 (5.01 GiB 5.38 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Oct 25 09:13:10 2013 State : clean Active Devices : 2Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : Vmware5:1 (local to host Vmware5) UUID : 91d7c869:e5b6aa2e:7515e116:dac07875 Events : 17 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 1 8 34 1 active sync /dev/sdc2
8).模拟软Raid1中/dev/sdb2这块硬盘损坏。
[root@Vmware5 ~]# mdadm /dev/md1 -f /dev/sdb2mdadm: set /dev/sdb2 faulty in /dev/md1[root@Vmware5 ~]# mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Oct 25 09:08:45 2013 Raid Level : raid1 Array Size : 5249088 (5.01 GiB 5.38 GB) Used Dev Size : 5249088 (5.01 GiB 5.38 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Oct 25 09:14:38 2013 State : clean, degraded Active Devices : 1Working Devices : 1 Failed Devices : 1 Spare Devices : 0 Name : Vmware5:1 (local to host Vmware5) UUID : 91d7c869:e5b6aa2e:7515e116:dac07875 Events : 18 Number Major Minor RaidDevice State 0 0 0 0 removed 1 8 34 1 active sync /dev/sdc2 0 8 18 - faulty spare /dev/sdb2
9).把/dev/sdb2这块硬盘移除。
[root@Vmware5 ~]# mdadm /dev/md1 -r /dev/sdb2mdadm: hot removed /dev/sdb2 from /dev/md1[root@Vmware5 ~]# mdadm -D /dev/md1/dev/md1: Version : 1.2 Creation Time : Fri Oct 25 09:08:45 2013 Raid Level : raid1 Array Size : 5249088 (5.01 GiB 5.38 GB) Used Dev Size : 5249088 (5.01 GiB 5.38 GB) Raid Devices : 2 Total Devices : 1 Persistence : Superblock is persistent Update Time : Fri Oct 25 09:17:14 2013 State : clean, degraded Active Devices : 1Working Devices : 1 Failed Devices : 0 Spare Devices : 0 Name : Vmware5:1 (local to host Vmware5) UUID : 91d7c869:e5b6aa2e:7515e116:dac07875 Events : 33 Number Major Minor RaidDevice State 0 0 0 0 removed 1 8 34 1 active sync /dev/sdc2
此时，sdb2这块磁盘显示removed。
10).重建一下sdb2这块磁盘。
[root@Vmware5 ~]# mdadm /dev/md1 -a /dev/sdb2mdadm: added /dev/sdb2[root@Vmware5 ~]# mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Oct 25 09:08:45 2013 Raid Level : raid1 Array Size : 5249088 (5.01 GiB 5.38 GB) Used Dev Size : 5249088 (5.01 GiB 5.38 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Oct 25 09:18:05 2013 State : clean, degraded, recovering Active Devices : 1Working Devices : 2 Failed Devices : 0 Spare Devices : 1 Rebuild Status : 8% complete Name : Vmware5:1 (local to host Vmware5) UUID : 91d7c869:e5b6aa2e:7515e116:dac07875 Events : 40 Number Major Minor RaidDevice State 2 8 18 0 spare rebuilding /dev/sdb2 1 8 34 1 active sync /dev/sdc2
此时，/dev/sdb2磁盘显示正在重建中。
如果大家想查看同步的动态效果：
[root@Vmware5 ~]# watch -n 1 'cat /proc/mdstat'
11).此时，再模拟/dev/sdc这硬盘损坏。
[root@Vmware5 ~]# mdadm /dev/md1 -f /dev/sdc2mdadm: set /dev/sdc2 faulty in /dev/md1[root@Vmware5 ~]# mdadm -D /dev/md1 /dev/md1: Version : 1.2 Creation Time : Fri Oct 25 09:08:45 2013 Raid Level : raid1 Array Size : 5249088 (5.01 GiB 5.38 GB) Used Dev Size : 5249088 (5.01 GiB 5.38 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Fri Oct 25 09:20:38 2013 State : clean, degraded Active Devices : 1Working Devices : 1 Failed Devices : 1 Spare Devices : 0 Name : Vmware5:1 (local to host Vmware5) UUID : 91d7c869:e5b6aa2e:7515e116:dac07875 Events : 59 Number Major Minor RaidDevice State 2 8 18 0 active sync /dev/sdb2 1 0 0 1 removed 1 8 34 - faulty spare /dev/sdc2
12).md1是软Raid1，两块硬盘其中一块出现问题，不耽误使用。看下可否查看mnt目录下的文件。
[root@Vmware5 ~]# cp /etc/inittab /mnt/[root@Vmware5 ~]# cat /mnt/inittab# inittab is only used by upstart for the default runlevel.# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.# System initialization is started by /etc/init/rcS.conf# Individual runlevels are started by /etc/init/rc.conf# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,# with configuration in /etc/sysconfig/init.# For information on how to write upstart event handlers, or how# upstart works, see init(5), init(8), and initctl(8).# Default runlevel. The runlevels used are:# 0 - halt (Do NOT set initdefault to this)# 1 - Single user mode# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)# 3 - Full multiuser mode# 4 - unused# 5 - X11# 6 - reboot (Do NOT set initdefault to this)id:3:initdefault:
可以正常查看，说明了Raid1创建成功了。
到此为止，实验结。
命运掌握在自己手中
