Saturday, November 13, 2010

CentOS / Redhat: Create Software RAID 1 Array

SkyHi @ Saturday, November 13, 2010
Recently, I've added another 73GB SAS disk to my Linux server after the installation. I've 30GB empty partition on old hard disk. How do I convert old /dev/sda3 and new /dev/sdb1 (both 30GB) into RAID 1 to improve NFS server speed and reliability?

RAID devices are virtual devices created from two or more real block devices. Linux supports RAID1 and other levels. You need to have same size partition on both disks i.e. on second disk create partitions exactly the same size as those on the first disk, and set the type as fd (Linux raid autodetect). You need to use the following commands to create RAID 1.
WARNING! These examples may crash your computer or may result into data loss if not executed properly. These examples involves disk partition manipulation using the fdisk command and building (formatting) file systems using the mkfs.ext3 command. So make sure you backup all data before typing any one of the following command.
Our sample RAID -1 setup with two partitions:
+------------+
|            |
| /dev/sda3  | ===============+
|            |                |
+------------+                |
  37GB                        | =======> RAID-1 /dev/md0
                              |
+------------+                |
|            |                |
| /dev/sdb1  | ===============+
|            |
+------------+
  37GB

Step # 1: List Partitions

Type the following commands:
# fdisk -l
# fdisk -l /dev/sda

Sample outputs:
Disk /dev/sda: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        3929    31455270   83  Linux
/dev/sda3            3930        8779    38957625   83  Linux
/dev/sda4            8780        8910     1052257+   5  Extended
/dev/sda5            8780        8910     1052226   82  Linux swap / Solaris
For example, purpose you will use /dev/sda3 (size 37G). Now, list second disk partitions, enter:
# fdisk -l /dev/sdb
Sample outputs:
Disk /dev/sdb: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

Step # 2: Create 37G Partitions For /dev/sdb

Type the following command:
# fdisk /dev/sdb
Sample outputs:
The number of cylinders for this disk is set to 8910.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): 
To add a new partition, type n command followed by p command as follows:
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-8910, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-8910, default 8910): +37G
To print the partition table type p command, enter:
Command (m for help): p

Disk /dev/sdb: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          37      297171   83  Linux
Finally, set partition type to "Software Raid" i.e. type 0xfd *(type t command followed by fd code to change partition type to Software RAID):
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd

Command (m for help): p

Disk /dev/sdb: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          37      297171   fd  Linux raid autodetect
Save the changes and exit to shell prompt by typing w command:
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Step # 3: Create / Update 37G Partitions For /dev/sda

You need to delete /dev/sda3 (make sure you backup all data) and/or set it to software raid as follows using the fdisk command:
# fdisk /dev/sda
Sample outputs:
The number of cylinders for this disk is set to 8910.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        3929    31455270   83  Linux
/dev/sda3            3930        8779    38957625   83  Linux
/dev/sda4            8780        8910     1052257+   5  Extended
/dev/sda5            8780        8910     1052226   82  Linux swap / Solaris

Command (m for help): t
Partition number (1-5): 3
Hex code (type L to list codes): fd
Changed system type of partition 3 to fd (Linux raid autodetect)

Command (m for help): p

Disk /dev/sda: 73.2 GB, 73295462400 bytes
255 heads, 63 sectors/track, 8910 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        3929    31455270   83  Linux
/dev/sda3            3930        8779    38957625   fd  Linux raid autodetect
/dev/sda4            8780        8910     1052257+   5  Extended
/dev/sda5            8780        8910     1052226   82  Linux swap / Solaris

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

Reload Partition Table Changes

Use the partprobe command to reload the changes without rebooting the box:
# partprobe /dev/sda
OR
# echo 1 > /sys/block/sdb/device/rescan

Step #3: Create RAID 1 Array

To create RAID1 using /dev/sda3 and /dev/sdb1, enter:
# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb1
To see RAID build progress, enter:
# watch cat /proc/mdstat
Sample outputs:

Fig.01: RAID 1 mirroring
Fig.01: RAID 1 mirroring

Once build completed create a filesystem on the new software raid devices, enter:
# mkfs.ext3 /dev/md0
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
4872896 inodes, 9737360 blocks
486868 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
298 block groups
32768 blocks per group, 32768 fragments per group
16352 inodes per group
Superblock backups stored on blocks:
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
 4096000, 7962624

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Mount RAID1 at /raid1

Type the following commands:
# mkdir /raid1
# mount /dev/md0 /raid1/
# df -H /raid1/

Sample outputs:
Filesystem             Size   Used  Avail Use% Mounted on
/dev/md0                40G   185M    38G   1% /raid1

How Do I Test My RAID 1 Array Speed?

Simply use the dd command as follows:
# cd /raid1/
# dd if=/dev/zero of=output bs=8k count=256k
# rm output

262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 6.01068 seconds, 357 MB/s

Update /etc/fstab

Make sure RAID1 get mounted when you reboot the system (note >> append redirection):
# echo '/dev/md0 /raid1 ext3 noatime,rw 0 0' >> /etc/fstab

How Do I Get Detailed Information About RAID 1?

Type the following command:
# mdadm --query --detail /dev/md0
Sample outputs:
/dev/md0:
        Version : 0.90
  Creation Time : Wed May 26 01:58:53 2010
     Raid Level : raid1
     Array Size : 38949440 (37.15 GiB 39.88 GB)
  Used Dev Size : 38949440 (37.15 GiB 39.88 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed May 26 02:18:22 2010
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : b6f207e4:7f73484f:53f64bfd:e70b77d8
         Events : 0.2

    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       1       8       17        1      active sync   /dev/sdb1
REFERENCES
http://www.cyberciti.biz/faq/centos-redhat-rhel-linux-setup-create-raid1/