Linux/mdadm

From Omnia
Jump to: navigation, search

mdadm

"mdadm is a Linux utility by Neil Brown that is used to manage RAID devices, previously known as mdctl. Besides managing, it can create, delete, or monitor Linux software RAIDs. Available under version 2 or later of the GNU General Public License, mdadm is free software. Mdadm derives its name from the “md” (multiple disk) device nodes it manages." [1]

Commands

Show all md devices status:

cat /proc/mdstat

Show md device details:

mdadm --detail /dev/md0

Create RAID 0 (stripe):

mdadm --create --verbose /dev/md0 --level=raid0 --raid-devices=3 /dev/hd[cde]1

Create RAID 5 (stripe with parity):

mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sd[cde]1
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 --spare-devices=1 /dev/sd[cdef]1

Add partition to md device:

mdadm /dev/md0 --add /dev/sdb1

Fail and remove partition of md device:

mdadm /dev/md0 --fail /dev/sdb1
mdadm /dev/md0 --remove /dev/sdb1

Start drive without resync:

mdadm -S /dev/md0
mdadm -A --assume-clean /dev/md0

Create /etc/mdadm.conf (high to low detail, any will work)

mdadm --detail --scan --verbose > /etc/mdadm.conf
mdadm --detail --scan > /etc/mdadm.conf
mdadm --examine --scan > /etc/mdadm.conf

Stop array:

mdadm --stop /dev/md0

Start (assemble) array:

mdadm --assemble /dev/md0

Add disk to RAID 5:

unmount /dev/md1
mdadm --add /dev/md1 /dev/sdb3
mdadm --grow --raid-devices=4 /dev/md1

# wait for rebuild...
watch -dc cat /proc/mdstat
e2fsck -f /dev/md3
resize2fs /dev/md3

Common mdadm commands

Generate mdadm.conf

cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.save
/usr/share/mdadm/mkconf --generate > /etc/mdadm/mdadm.conf

Create RAID

mdadm --create /dev/md2 --raid-devices=3 --spare-devices=0 --level=5 --run /dev/sd[cde]1

Remove disk from RAID

mdadm --fail /dev/md0 /dev/sda1
mdadm --remove /dev/md0 /dev/sda1

Copy the partition structure (when replacing a failed drive)

sfdisk -d /dev/sda | sfdisk /dev/sdb 
mdadm --zero-superblock /dev/sdb

Add a disk to a RAID array (to replace a removed failed drive)

mdadm --add /dev/md0 /dev/sdf1

Check RAID status

cat /proc/mdstat
mdadm --detail /dev/md0

Reassemble a group of RAID disks

#This works to move an assembly from one physical machine to another.
mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

Steps to emulate mdrun (which has been depreciated)

# haven't tested this. Use with care
mdadm --examine --scan --config=partitions > /tmp/mdadm.conf
mdadm --assemble --scan --config=/tmp/mdadm.conf

Add a disk to an existing RAID and resize the filesystem

mdadm --add /dev/md0 /dev/sdg1
mdadm --grow /dev/md0 -n 5
e2fsck -f /dev/md0
resize2fs /dev/md0
e2fsck -f /dev/md0

Replace all disks in an array with larger drives and resize

# For each drive in the existing array
mdadm --fail /dev/md0 /dev/sda1
mdadm --remove /dev/md0 /dev/sda1
# physically replace the drive
mdadm --add /dev/md0 /dev/sda1
# now, wait until md0 is rebuilt.
# this can literally take days
# All drives have been replaced and sync'd, but they still use the original size.
# Issue the following command to use all available space:
mdadm --grow /dev/md0  --size=max
# Do not forget to resized the file system which sits on the raid set:
# for ext2/3/4
e2fsck -f /dev/md0 && resize2fs /dev/md0 && e2fsck -f /dev/md0
# for lvm pv
pvresize /dev/md0
# for ntfs
ntfsresize /dev/md0
# note, most likely ntfs is NOT exported as a single partition. In the case
# of a Xen hvm machine, it is a "disk device" so you will need to resize the
# partition itself, then resize ntfs.

Stop and remove the RAID device

mdadm --stop /dev/md0
mdadm --remove /dev/md0

Destroy an existing array

mdadm --manage /dev/md2 --fail /dev/sd[cde]1
mdadm --manage /dev/md2 --remove /dev/sd[cde]1
mdadm --manage /dev/md2 --stop
mdadm --zero-superblock /dev/sd[cde]1

Speed up a sync (after drive replacement)

cat /proc/sys/dev/raid/speed_limit_max
200000

Rename an existing array

mdadm --stop /dev/md127
mdadm -A /dev/md0 -m127 --update=super-minor /dev/sd[bcd]

Source: Linux Server Tech FAQ - Common mdadm commands - http://wiki.linuxservertech.com/index.php?action=artikel&cat=7&id=11&artlang=en

LVM

See LVM

Handle Drive Failure

Fail device:

mdadm /dev/md0 --fail /dev/sdb1    # -f
  mdadm: set /dev/sdb1 faulty in /dev/md0

Remove failed device:

mdadm /dev/md0 --remove /dev/sdb1

Verify failed device:

mdadm --detail /dev/md0

Add device to md:

mdadm /dev/md0 --add /dev/sdb1    # -a

Verify rebuild:

mdadm --detail /dev/md0
cat /proc/mdstat

References:

Force Partial Failure Recovery

inactive array (missing 2 devices due to io failure)

md4 : inactive sdf3[5] sde3[4] sdd3[3] sdc3[2]
      5780957696 blocks

/etc/mdadm.conf

ARRAY /dev/md4 level=raid5 num-devices=6 UUID=f0ce1e02:2cd38a68:6ffb704c:fe6e32b0

Stop array:

mdadm --stop /dev/md4

Forcefully Rebuild Array:

mdadm -A --force /dev/md4 /dev/sd[acdef]3
# notice 'b' is missing as it is dead dead
# but 'a' is only partially dead (random io errors)

References:

Notes

See mdadm/Notes

keywords