Creating RAID1 on a remote Debian box:
------ Goal
To create a system which boots off a RAID1 and has swap, '/' and '/home' on a raid as well. Configure grub to boot from either disk in case of failure.
------ Prepare
#> fdisk -l /dev/hda
Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 10 80293+ 83 Linux
/dev/hda2 11 140 1044225 82 Linux swap / Solaris
/dev/hda3 141 9729 77023642+ 83 Linux
#> fdisk -l /dev/hdc
Disk /dev/hdc: 80.0 GB, 80026361856 bytes
16 heads, 63 sectors/track, 155061 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Device Boot Start End Blocks Id System
/dev/hdc1 1 19377 9765976+ fd Linux raid autodetect
/dev/hdc2 19378 27128 3906504 fd Linux raid autodetect
/dev/hdc3 27129 104634 39063024 fd Linux raid autodetect
/dev/hdc4 104635 155061 25415208 83 Linux
------ Create
#> mdadm --create --verbose /dev/md0 --level raid1 --raid-disks 2 missing /dev/hdc1
mdadm: size set to 9765888K
mdadm: array /dev/md0 started.
#> mdadm --create --verbose /dev/md1 --level raid1 --raid-disks 2 missing /dev/hdc2
mdadm: size set to 3906432K
mdadm: array /dev/md1 started.
#> mdadm --create --verbose /dev/md2 --level raid1 --raid-disks 2 missing /dev/hdc3
mdadm: size set to 39062912K
mdadm: array /dev/md2 started.
------ Format
#> mkfs.ext3 /dev/md0
mke2fs 1.37 (21-Mar-2005)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1221600 inodes, 2441472 blocks
122073 blocks (5.00%) reserved for the super user
First data block=0
75 block groups
32768 blocks per group, 32768 fragments per group
16288 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 21 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
#> mkswap /dev/md1
Setting up swapspace version 1, size = 4000182 kB
no label, UUID=62084b83-9a70-4e31-a79f-a1bd50cad225
#> mkfs.ext3 /dev/md2
------ Copy the data over
#> cd /; find . -xdev | cpio -pm /mnt/md0/
8605674 blocks
#> cd /boot/; find . -xdev | cpio -pm /mnt/md0/boot/
------ Prepare for reboot
#> strings initrd.img-2.4.18-sb-smp | grep -i raid
shows raid1.o included in - so I think initrd is ready
(It shows it, but it doesn't mean that it works...)
Still need to create an initrd image (and again later):
#> sudo mkinitrd -o /boot/initrd.img-2.4.18-sb-smp-raid 2.4.18-sb-smp
Install Grub:
#> grub
grub> root (hd0,0)
grub> setup (hd0)
grub> root (hd1,0)
grub> setup (hd1)
grub> quit
Include the following lines in menu.lst
default 0
fallback 1 # fallback goes to non-raid for now
...
title Debian GNU/Linux, kernel 2.4.18-sb-smp RAID
root (hd1,0)
kernel /boot/vmlinuz-2.4.18-sb-smp root=/dev/md0 ro nomce
initrd /boot/initrd.img-2.4.18-sb-smp-raid
savedefault
Copy the menu.lst to both places (raid and non-raid)
Edit the fstab on the RAID disk:
/dev/md0 / ext3 errors=remount-ro 0 1
proc /proc proc defaults 0 0
/dev/md1 none swap sw 0 0
/dev/md2 /mnt/md2 ext3 errors=remount-ro 0 1
Check everything again!!!
Reboot and you will have a RAID system or a pumpkin
------ reboot
If the server comes up - do 'df' and cat /proc/mdstat and see that it is indeed in the RAID setup
Now start cleaning up and finishing up
------ Repartition the 1st drive
#> fdisk /dev/hda
Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 1217 9775521 fd Linux raid autodetect
/dev/hda2 1218 1704 3911827+ fd Linux raid autodetect
/dev/hda3 1705 6568 39070080 fd Linux raid autodetect
/dev/hda4 6569 9729 25390732+ 83 Linux
------ Reconstruct the array
#> mdadm --manage /dev/md0 --add /dev/hda1
mdadm: hot added /dev/hda1
#> mdadm --manage /dev/md1 --add /dev/hda2
mdadm: hot added /dev/hda2
#> mdadm --manage /dev/md2 --add /dev/hda3
mdadm: hot added /dev/hda3
Check the status:
#> cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 ide/host0/bus0/target0/lun0/part3[2] ide/host0/bus1/target0/lun0/part3[1]
39062912 blocks [2/1] [_U]
[>....................] recovery = 3.0% (1203712/39062912) finish=15.7min speed=40123K/sec
md1 : active raid1 ide/host0/bus0/target0/lun0/part2[2] ide/host0/bus1/target0/lun0/part2[1]
3906432 blocks [2/1] [_U]
md0 : active raid1 ide/host0/bus0/target0/lun0/part1[0] ide/host0/bus1/target0/lun0/part1[1]
9765888 blocks [2/2] [UU]
unused devices: <none>
Reconstruction takes some time and only processes one device at a time.
------ Configure Grub
Fix Grub to boot from either disk into RAID:
title Debian GNU/Linux, kernel 2.4.18-sb-smp RAID
root (hd0,0)
kernel /boot/vmlinuz-2.4.18-sb-smp root=/dev/md0 ro nomce
initrd /boot/initrd.img-2.4.18-sb-smp-raid
savedefault
title Debian GNU/Linux, kernel 2.4.18-sb-smp RAID Fail-over
root (hd1,0)
kernel /boot/vmlinuz-2.4.18-sb-smp root=/dev/md0 ro nomce
initrd /boot/initrd.img-2.4.18-sb-smp-raid
savedefault
Run:
#> grub-install /dev/hda
#> grub-install /dev/hdc
(these commands take some time to complete)
Then simply start the grub shell - and run:
#> grub
grub> find /boot/grub/stage1
(hd0,0)
(hd1,0)
This shows that grub can find the boot loader on both disks
------ Configure mdadm.conf
#> mdadm --detail --scan
Add the DEVICE line and the rest can be copied from the output
my /etc/mdadm/mdadm.conf:
DEVICE /dev/hda* /dev/hdc*
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=3ba82ac4:c23cf3ac:66c11f5a:c93a083d devices=/dev/hda1,/dev/hdc1
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=7679314e:d5b48d0d:901ccafc:fa214592 devices=/dev/hda2,/dev/hdc2
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=16135445:b2797fcf:b5301ec2:7b70049e devices=/dev/hda3,/dev/hdc3
------ Make the initrd again
that initrd.img needs to regenerated after everything is working. I don't know why, but I am going to do it anyway:
#> cp /boot/initrd.img-2.4.18-sb-smp-raid /boot/initrd.img-2.4.18-sb-smp-raid.old
#> mkinitrd -o /boot/initrd.img-2.4.18-sb-smp-raid 2.4.18-sb-smp
Reboot again... if it still works - you are done
Enjoy,
Alex Pachikov © 2005
p.s. Check hdparm after reboot to see that DMA comes up as 1, or edit /etc/hdparm.conf
p.p.s. Check that the monitoring part of mdadm is working.