Migrating Windows Fileservers: Preserving NTFS and Domain Controller Permissions

A client was running a Windows 2008 fileserver. Share folders have been created and used for years, and permissions were set per group or per user over the many folders, linked to Active Directory domain controller.

For this particular case, the shares were on a dedicated drive that is a storage mapped volume. Reassigning the volume from the old server to a new one is quite easy, but the tricky part was preserving the share settings and the NTFS permissions along with the active directory domain controller security permissions.

After much digging around, the solution was a builtin command line that can backup the permissions, and exporting a registry key to backup the configurations of the shares and their paths. Make sure you test this on a test machine first before applying to production! This will save both local users and domain controller security permissions.

Pay attention to back slashes (\) as it makes a difference to the tool.

Step 0: Backup and Restore Shares and their Permissions

• Run regedit with administrator permission: search for regedit then right click and choose "Run as Administrator"
• Go to this location:  HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Shares
  Note: HKLM = HKey_Local_Machine
• Right click the Shares registry key and export. The key that looks like a folder in the left pane, not the content inside on the right pane.

To restore the shares and their permissions, double click the saved exported file.

Step 1: Backup and Restore NTFS and Active Directory Security Permissions

• Open the command prompt (cmd.exe) with administrator permission.
• To backup: icacls "path to folder" /save ntfsperms.txt /t /c 2> errors.txt
  Example: icacls d:\data /save ntfsperms.txt /t /c 2> errors.txt
  /t for recursion to include subfolders of the main one.
  /c to continue even when errors occur, but they'll printed and written to the errors text file.
  
  Note: If you put multiple folders directly on the root of the drive, the command should look like this: icacls d:\* /save ntfsperms.txt /t /c 2> errors.txt

To restore: icacls d:\ /restore ntfsperms.txt

Yes, there's a difference in the way you restore the permissions and the path. Even if you had backed up d:\data, you restore to the root directory/folder d:\. That's how the tool works.

Keep in mind that the text file you'll save the permissions to will exist in the same place where it's showing the command prompt. If you run the cmd as Admin, you'll be in C:\Windows\System32 by default.

Review: Pretec 8GB microSD Card

I bought this 8GB class 10 microSD card that also comes with an adapter from newegg for $8 (free delivery when choosing 5-7 day delivery). Pretec is a Taiwanese company.

The microSD card comes pre-formatted as FAT16 and preloaded with software for Android from a company called NQ Mobile:

• Mobile Security: Protects against mobile viruses, trojans and spyware
• Free features: anti-malware, Contacts backup, System optimization, Privacy Protection (checks app permissions), Traffic Manager
• Premium: Anti-theft, Virus database update
• Mobile Manager: Suite of tools for managing phone-related functions
• Free: Call Manager (block calls/SMS), Backup Contacts, Erase call history/SMS
• Premium: Hidden private space, spam database update
• Mobile Booster: Smartphone performance optimization
• Free: Single-key optimization, Task monitor, traffic monitor, power monitor, software manager, file manager

The sd card also comes with a Windows application for photo recovery and free recovery of 25 pictures, after which you have to pay.

The default capacity is 7.5GB, and the used capacity by the apps above is 20MB. fdisk output:

fdisk -l /dev/sdc

Disk /dev/sdc: 7964 MB, 7964983296 bytes
91 heads, 18 sectors/track, 9497 cylinders, total 15556608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            8192    15556607     7774208    6  FAT16

Writing 1GB to the sd card:

dd if=/dev/zero of=./test.dd bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 0.974053 s, 1.1 GB/s

Writing 100MB to the sd card with fdatasync enabled to avoid caching (realistic number for cameras):

dd if=/dev/zero of=./test.dd bs=10M count=10 conv=fdatasync
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 8.36829 s, 12.5 MB/s

Writing 500MB to the sd card with fdatasync enabled:


dd if=/dev/zero of=./test.dd bs=10M count=50 conv=fdatasync
50+0 records in
50+0 records out
524288000 bytes (524 MB) copied, 63.5707 s, 8.2 MB/s

Putting ext4 instead of FAT16 on the card. First I changed the partition type using fdisk/cfdisk, then I used mkfs.ext4 with no options (defaults) on the partition.

mkfs.ext4 /dev/sdc1
mke2fs 1.42.5 (29-Jul-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
486720 inodes, 1943552 blocks
97177 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1992294400
60 block groups
32768 blocks per group, 32768 fragments per group
8112 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                           
Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now running the same write tests as above:

dd if=/dev/zero of=./test.dd bs=10M count=100             
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 0.698984 s, 1.5 GB/s

dd if=/dev/zero of=./test.dd bs=10M count=10 conv=fdatasync
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 9.40124 s, 11.2 MB/s

dd if=/dev/zero of=./test.dd bs=10M count=50 conv=fdatasync
50+0 records in
50+0 records out
524288000 bytes (524 MB) copied, 42.9945 s, 12.2 MB/s

Now testing XFS file system:

mkfs.xfs -f /dev/sdc1
meta-data=/dev/sdc1              isize=256    agcount=4, agsize=485888 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=1943552, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

dd if=/dev/zero of=./test.dd bs=10M count=100              
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 0.590697 s, 1.8 GB/s

dd if=/dev/zero of=./test.dd bs=10M count=10 conv=fdatasync
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 8.03924 s, 13.0 MB/s

dd if=/dev/zero of=./test.dd bs=10M count=50 conv=fdatasync
50+0 records in
50+0 records out
524288000 bytes (524 MB) copied, 42.0262 s, 12.5 MB/s

Unfortunately, there's no native exFAT driver for Linux so I cannot test the write speeds properly. exFAT is the filesystem used by some video cameras and is newer than FAT16.


In conclusion, the card does perform to the class 10 speed standards: a minimum of 10 MB/s. Your experience will vary depending on the filesystem you choose.

Note: Do not be fooled by the 1GB/s speeds. Those happened because my machine has l6GB RAM. Whenever a program writes to storage, the operating system will write to RAM then sync the data to disk, for the best performance. So, the more free memory you have, the better the performance.

Sorting Docx Files By Date in Shell

A friend of mine asked me to recover data from a formatted disk and because the disk is formatted, there's no way to know the file name, directory or some other attributes that are saved in the Master File Table (MFT) of the NTFS filesystem.

Docx files are basically a collection of XML files that are zipped together. Knowing this fact, all I had to do is write a script that unzips the control file that has the time stamps, create the directories based on year & month (yyyy-mm), then move the file inside it.

One required package to be installed is the "xmlstarlet" package. Use your package manager to install it before running the script.

You don't need to modify this script. Just save it, chmod +x the script then use it with "find" like the example. One thing, if you intend to traverse subdirectories, then remove the maxdepth option and make sure your destination directory (DocxSorted) is located in an upper level:

find /media/MyDisk/Unsorted -iname "*.docx" -exec /home/user/Desktop/sortDocxByDate.sh "{}" /media/MyDisk/DocxSorted \;

The script:

#!/bin/bash
# This is to be used by "find -exec" and pass the files as argument then the destination
# This is for .docx only!
# Example: find . -maxdepth 1 -iname "*.docx" -exec ./sortDocxByDate.sh "{}" DocxSorted \;
# Written by MBH http://mbhtech.blogspot.com -- free to use and modify
theFile=$1
dst=$2

DirDate=`unzip -p $theFile docProps/core.xml | xmlstarlet sel -t -m //dcterms:modified -v . | cut -d- -f1,2`

if [ ! -d "$dst/$DirDate" ]; then
mkdir -p "$dst/$DirDate"
echo created dir $DirDate
fi

mv $theFile $dst/$DirDate/`basename $theFile`
#echo moving $theFile to $dst/$DirDate/`basename $theFile`

Software RAID vs. LVM: Quick Speed Test

Table of Contents

1. Introduction
2. System Setup
3. Speed Test Methods
   1. hdparm
   2. dd
4. Speed Test #1: RAID vs. LVM
5. Speed Test #2: Disks Separately
6. Conclusion

Introduction

Currently, I have a fileserver that is setup this way:

Filesystem
      ^
Logical Volume Manager
      ^
Software RAID Arrays
      ^
Physical Disks

In my case, the LVM is an extra layer and it's not useful since I only have one physical entity that belongs to a Volume Group: A single RAID5 array.
So you could put your filesystem on top of a Logical Volume, or directly on the RAID array device. It depends on how you want to manage your data and devices.

So, is this hampering performance? The tables below will do the talking, but first: the setup.

System Setup

Processor	Intel Pentium Dual CPU E2160 @ 1.80GHz
Motherboard	MSI (MS-7514) P43 Neo3-F
	North Bridge: Intel P43
	South Bridge: Intel ICH10
SATA Controller 1	JMicron 20360/20363 AHCI Controller
	AHCI Mode: Enabled
	Ports: 6-7
Sata Controller 2	82801JI (ICH10 Family) SATA AHCI Controller
	Ports: 0-5
RAM	1GB @ CL 5
Video Card	GeForce 7300 GS
Disk sda	WDC WD10EACS-00D6B1
Disk sdb	WDC WD10EACS-00D6B1
Disk sdc	WDC WD10EACS-00ZJB0
Disk sdd	WDC WD10EADS-65L5B1
Disk sde	WDC WD10EADS-65L5B1
Disk sdf	MAXTOR STM31000340AS
Disk sdg	WDC WD10EACS-00ZJB0
Disk sdh	WDC WD10EADS-00L5B1
Disk sdi	Hitachi HDS721680PLAT80 (OS)
Chunk size	256kB
LVM: Physical Extent Size	1GB
LVM: Read ahead sectors	Auto (set to 256)

Speed Test Methods

A quick and easy way to run a speed test is by using a tool called hdparm and another called dd.
Note that these two utilities don't take the filesystem performance into account, as they read directly from the device, not a certain file. It doesn't matter in this case, as I'm about to show comparisons to show the magnitude of difference speed only, not show very exact results ;)

hdparm

hdparm -tT /dev/xxx
-t: Perform timings of device reads for benchmark and comparison purposes.
Displays  the  speed of reading through the buffer cache to the disk without any prior caching of data.
This measurement is an indication of how fast the drive can sustain sequential data reads under Linux, without any filesystem overhead.

-T: Perform timings of cache reads for benchmark and comparison purposes.
This displays the speed of reading directly from the Linux buffer cache without disk access.
This measurement is essentially an indication of the throughput of the processor, cache, and memory of the system under test.

dd

dd if=/dev/xxx of=/dev/null bs=10M count=400
This will read from the device and dump the data to a null device (just reading). Block size=10 Megabytes (2^20).
This will read 4GB of data. I specified 4GB to make sure that it surpasses the RAM size.

Before running dd, I flushed the read cache by entering: hdparm -f /dev/sd[a-h], which flushes the cache of all RAID disks.

Speed Test #1: RAID vs. LVM

LVM	root@Adam:~/mdadm-3.0# dd if=/dev/mapper/arrays-storage of=/dev/null bs=10M count=400 2097152000 bytes (2.1 GB) copied, 41.1147 s, 43.0 MB/s
	root@Adam:~/mdadm-3.0# hdparm -tT /dev/mapper/arrays-storage  Timing cached reads:   1926 MB in  2.00 seconds = 962.65 MB/sec  Timing buffered disk reads:  146 MB in  3.00 seconds =  48.62 MB/sec
RAID	root@Adam:~/mdadm-3.0# dd if=/dev/md0 of=/dev/null bs=10M count=400 2097152000 bytes (2.1 GB) copied, 10.9341 s, 125 MB/s
	root@Adam:~/mdadm-3.0# hdparm -tT /dev/md0 Timing cached reads:   1998 MB in  2.00 seconds = 998.73 MB/sec Timing buffered disk reads:  538 MB in  3.01 seconds = 178.98 MB/sec

The above numbers are the average of 3 runs.

Speed Test #2: Disks Separately

root@Adam:~# for i in {a,b,c,d,e,f,g,h}; do dd if=/dev/sd"$i"1 of=/dev/null bs=10M count=400; done
root@Adam:~# for i in {a,b,c,d,e,f,g,h}; do hdparm -I /dev/sd"$i" | grep Firmware; done

Disk	Model	Firmware	Speed Test Result
sda	WDC WD10EACS-00D6B1	01.01A01	46.3106 s, 90.6 MB/s
sdb	WDC WD10EACS-00D6B1	01.01A01	48.6391 s, 86.2 MB/s
sdc	WDC WD10EACS-00ZJB0	01.01B01	70.8184 s, 59.2 MB/s
sdd	WDC WD10EADS-65L5B1	01.01A01	46.9733 s, 89.3 MB/s
sde	WDC WD10EADS-65L5B1	01.01A01	44.2861 s, 94.7 MB/s
sdf	MAXTOR STM31000340AS	MX15	77.1797 s, 54.3 MB/s
sdg	WDC WD10EACS-00ZJB0	01.01B01	50.5498 s, 83.0 MB/s
sdh	WDC WD10EADS-00L5B1	01.01A01	46.747 s, 89.7 MB/s

As you can see, though sdc & sdg have the same model and firmware, their speed differs! I have no clue why and I searched in Western Digital's website for firmwares to download, but their site leads no where to any firmware download link.

The Maxtor disk has a newer firmware released. I'll checkout its changelog before installing it. Also, as a precaution, I'll clone the Maxtor disk to sdg since it's not being used now; just in case the new firmware doesn't play nice!

Conclusion

From the above numbers, it's clear that LVM, in my setup, has crippled the performance by a huge margin (~66%). So for my next setup, I'm going to skip LVM and slap the filesystem directly on top of the RAID5 array.

On one of my PCs (Adrenalin), I already have XFS filesystem running on top of the RAID array and LVM is not being used. I get double the speed of hard disks out of the array (140 MB/s) when tested it last year with hdparm.

I don't claim that this is a typical problem of LVM. I did a quick search and didn't find numbers. I'm too lazy right now to find anything really. But I have the numbers on that MSI crap board (caused me so many problems with the SATA ports), and I'll skip LVM on that board. If I keep the board & not smash it to smithereens.

Irrelevant note: I'm loving posting to my blog through Google Docs.

DB2 Container Rebalancing: Choosing the right filesystems

Our ERP software's database spans over multiple filesystems for better load balancing, but mixing high-load and low-load database tables on the same filesystem. Also, filesystem are grouped in volume groups that reside on different physical disks.

To better understand this, here's how our production SAN is devised:

Array 2 is one of two database data arrays, and is created over 6 hard disks in RAID level 5.
Array 2 contains two Logical Drives: SAPappShared and SAPdataDrv2. These will be mapped to the operating system (OS) as physical disks.

 

Here you can see that the 3rd array spans over different disks. This is to optimize data load times when requesting data from the database.
The two red/white disks are hot-spares that will substitute any failed disk immediately.
You can also see that this array reside on a different controller (B), while the previous is on (A). The controller handles I/O requests from the OS.

 

Database logs are written to a RAID level 1 array spanned over 2 disks. Logs are written always so it's better to keep them away from data to not hamper the performance.






From what you saw above, you can count 6 Logical Drives that relate to the database. The database uses only 4 of them: Log, Data1-1, Data1-2 and Data2. The Heartbeat drives are used by the cluster services of the OS.



Now, moving to the OS: IBM's AIX. We run version 5.3L.

On AIX, you cannot create a filesystem on a physical disk directly. The physical disk has to belong to a Volume Group. A volume group can house multiple physical disks, whether internal disks or attached through storage or network. A volume group can contain multiple filesystems. This all falls under the topic of the Logical Voume Manager, which I will not talk about here.

Back to our setup above, from AIX's point of view, the Logical Drives are shown as physical disks:

# lspv
hdisk0          00c3b3f0feb38826                    rootvg          active
hdisk1          00c3b3f0d832abbc                    rootvg          active
hdisk2          00c3b3e0fbad39e6                    hrtvg1
hdisk3          00c3b3e0fbb076c8                    saplogsvg       active
hdisk4          00c3b3e0fbef90aa                    sapdatavg1      active
hdisk5          00c3b3e0fbf2e9bf                    sapdatavg2      active
hdisk6          00c3b3f0168bedbb                    sapdatavg3      active

You can see from the list above that each Logical Drive is a physical disk and moreover, each has been put to a separate volume group.

These are filesystems spanned over the volume group on hdisk4:

# lspv -l hdisk4
hdisk4:
LV NAME               LPs   PPs   DISTRIBUTION          MOUNT POINT
db2lv                 2     2     00..02..00..00..00    /db2/EHP/db2ehp
db2dumplv             24    24    00..01..00..00..23    /db2/EHP/db2dump
sapdata6lv            280   280   123..00..00..135..22  /db2/EHP/sapdata6
sapdata3lv            340   340   13..132..135..00..60  /db2/EHP/sapdata3
loglv2                1     1     00..01..00..00..00    N/A

Before continuing, let's talk about database usage, to be able to justify the next moves.

Since the database tables have been split over multiple filesystems, some of the filesystems have high growth rate. Our database (IBM's DB2) can span the contents of a table over multiple filesystems; a load balancing technique. It does this through its concept of rebalancing containers. (In DB2, tables reside in Tablespaces and a tablespace consists of one or more containers).

So, if we create a second container of a high growth table, it will take half of it and dump it over the new container and in the future, it will write to both in a load balancing form.

Now, to be able to achieve a true sense of load balancing, in filesystem utilization and database performance, ideally these two filesystems (that house the high growth table) should be on different controllers, so that data requested residing on both filesystems can be served in parallel by the controllers.

If you're doing the rebalancing after going live with the system (which is the case mostly), then you'd need to know which physical disk on the OS goes to which controller.

Here, you see that each Logical Drive has a unique identifier called Logical Unit Number (LUN).

The LUN is represented as a hex value on AIX.





From the shell, type "lspv" to list the physical volumes, then "lsattr -El " to list the attributes of one of the disks. The value "lun_id" is what you want. See the output below.

# lspv
hdisk0          00c3b3f0feb38826                    rootvg          active
hdisk1          00c3b3f0d832abbc                    rootvg          active
hdisk2          00c3b3e0fbad39e6                    hrtvg1
hdisk3          00c3b3e0fbb076c8                    saplogsvg       active
hdisk4          00c3b3e0fbef90aa                    sapdatavg1      active
hdisk5          00c3b3e0fbf2e9bf                    sapdatavg2      active
hdisk6          00c3b3f0168bedbb                    sapdatavg3      active

# lsattr -El hdisk4
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B80003260360000A92E472997CA IEEE Unique volume name                False
lun_id         0x0005000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           00c3b3e0fbef90aa0000000000000000 Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     5                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10400                          SCSI ID                                False
size           173670                           Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False

# lsattr -El hdisk6
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B80003260360000A9AA472EA49C IEEE Unique volume name                False
lun_id         0x000c000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           00c3b3f0168bedbb0000000000000000 Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     5                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10500                          SCSI ID                                False
size           294225                           Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False

Using the above commands, finding which disk belongs to which controller on a SAN on AIX becomes like baby steps ;)

We did a rebalance on one of the containers today. The container's size was about 50GB and it took about an hour on an IBM DS4800 SAN hooked with fiber to the database node residing on an IBM pSeries (570) machine: 4 POWER5 processors allocated and 30GB of RAM.

I tried looking up time estimation for rebalancing containers on DB2 on Google, but didn't find anything useful. Luckily, things went on smoothly.
The error file "db2diag.log" had the following with regards to rebalancing:

2009-05-18-01.05.59.552039+180 E106271503A342     LEVEL: Warning
PID     : 799110               TID  : 1           PROC : db2rebal 0
INSTANCE: db2ehp               NODE : 000
FUNCTION: DB2 UDB, buffer pool services, sqlb_rebalance, probe:2876
MESSAGE : ADM6062I  Rebalance for table space "EHP#BTABD" (ID "17") has been
 completed.

2009-05-18-01.05.59.552281+180 I106271846A295     LEVEL: Warning
PID     : 799110               TID  : 1           PROC : db2rebal 0
INSTANCE: db2ehp               NODE : 000
FUNCTION: DB2 UDB, buffer pool services, sqlb_rebalance, probe:2876
MESSAGE : PoolID 17: Last extent moved was #1866949

During the rebalancing period, the container being worked on was not showing in the list on the ERP software (SAP) and threw some errors. It was working fine on the database level however. After the process completed, SAP reported the new containers properly.

Making Bootable USB Drives Recognizable on Windows, Again

A couple of weeks back, I "borrowed" my sister's 256MB USB flash memory and put Slax on it.

Now, my sister wants her "USB" back, which I tried to explain that if I give her just the USB part it won't work as intended. As funny as I was, she didn't think so, and she wanted the WHOLE thing back. And working.

Windows didn't recognize the whole device and couldn't even see it as a removable disk, so formatting was not an option, yet.

Using a Windows XP bootable CD and trying to use fixmbr failed, because the recovery console failed to see the USB drive.

I fixed it on Linux, by using "dd" to write zeros on the disk (/dev/sdf in my case), then used fdisk (not cfdisk) to write a DOS partition table, then created a new primary partition with type "Windows 95 FAT" (b).

root@adrenalin:~# dd if=/dev/zero of=/dev/sdf
dd: writing to `/dev/sdf': No space left on device
517121+0 records in
517120+0 records out
264765440 bytes (265 MB) copied, 110.158 s, 2.4 MB/s

root@adrenalin:~# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF
disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)

Command (m for help): p

Disk /dev/sdf: 264 MB, 264765440 bytes
9 heads, 57 sectors/track, 1008 cylinders
Units = cylinders of 513 * 512 = 262656 bytes

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help): o
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1008, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1008, default 1008):
Using default value 1008

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): b
Changed system type of partition 1 to b (W95 FAT32)

Command (m for help): p

Disk /dev/sdf: 264 MB, 264765440 bytes
9 heads, 57 sectors/track, 1008 cylinders
Units = cylinders of 513 * 512 = 262656 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1        1008      258523+   b  W95 FAT32

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

The disk is fixed, but doesn't have a filesystem. You can either make one in Linux, or slap it on a Windows machine & format it.

To make a FAT32 filesystem on Linux, run:

mkfs.vfat -F 32 /dev/sdf1

RAID Expansion and The Beauty of XFS

Existing Hardware Setup

My workstation has 4 SATA2 hard disk drives installed: 3x 320GB & 1x 80GB. The first 3 are on RAID arrays, while the last is a standalone partition with Windows to play games once in a while. The RAID arrays is managed by the Linux kernel, which is known as software RAID.

Existing Storage Capacity

The total storage capacity of the array was 592GB, of which 50GB was free. As I haven't finished my work on the NAS box, I had no place to store my future stuff, so I needed storage urgently, and had to add another disk into the existing array. As I hope to get my NAS box soon, I bought only one 320GB HDD.

Stage 0: HDD Installation & Partitioning

Since I had no RAID controllers, adding the hard disk while the PC was running was out of the question. So, after turning off the box, I added the new hard disk, and booted the box. Then, I partitioned the new hard disk into 2 partitions: boot & data, where boot occupied 100MB only.
This is different from the previous 3 HDDs, since they had a swap partition of 2GB per disk. I didn't add a swap partition in the new one. (And yes, there's 2GB of unused space).
This is how the new disk looked after being partitioned:

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *           1          12       96358+  fd  Linux raid autodetect
/dev/sde2              13       38913   312472282+  fd  Linux raid autodetect

Another way to look at it would be:

Name    Flags    Part Type   FS Type                     Size (MB)
------------------------------------------------------------------
sde1    Boot      Primary    Linux raid autodetect           98.71
sde2              Primary    Linux raid autodetect       319971.62

* The first print was produced using fdisk, the second using cfdisk.

Stage 0.5: Remove Shadowed Mounts

My /boot is a RAID1 array that is mounted over the existing /boot, and it's mounted as read-only. So, sense I'll be expanding the root filesystem (/) I thought of removing all un-needed filesystems, just in case!

Stage 1: Closer Than Veins

Now it's time to add partitions to their RAID arrays. If you don't wish to suffer like I did, change the minimum speed limit as follows:

echo '150000' > /proc/sys/dev/raid/speed_limit_min

Then, I added the boot partition to the array:

mdadm /dev/md0 -a /dev/sde1
mdadm --grow /dev/md0 --raid-disks=4

The first line will add the disk as a spare one only, and not an active part of the array. The second line makes it part of the array and mirrors the data to the new member of the RAID1 array.

Then, I added the data partiton to the RAID5 array, in a similar matter to the previous partition:

mdadm /dev/md1 -a /dev/sde2
mdadm --grow /dev/md1 --raid-disks=4

Because the RAID5 array is big, rebuilding it takes a lot of time. Mine took about 8.5 hours! The rebuilding process starts as soon as you execute the 2^nd line. You can monitor the process of rebuilding the array by:

cat /proc/mdstat

You'll get something looking like these:

[=>...................]  reshape =  7.7% (24075264/310472064) finish=583.2min speed=8184K/sec
[===>.................]  reshape = 19.8% (61607172/310472064) finish=582.5min speed=7116K/sec

Neat, right? Indeed, but the work isn't done yet!

Stage 2: Command and Conquer

Occupying the newly available free space is the last thing to do. After roughly 8.5 hours of rebuilding the array, it's finally online and ready to be abused by yours truly.
As the title of this post suggests, I use XFS filesystem, and one great feature that it offers is the ability to grow the filesystem online: No need to unmount the filesystem.
Quoting the manual page:

The filesystem must be mounted to be grown (see mount(8)). The existing contents of the
filesystem are undisturbed, and the added space becomes available for additional file
storage.

This is how things went:

root@adrenalin:~# xfs_growfs /
/dev/root: No such file or directory
Usage: xfs_growfs [options] mountpoint

Options:
        -d          grow data/metadata section
        -l          grow log section
        -r          grow realtime section
        -n          don't change anything, just show geometry
        -I          allow inode numbers to exceed 32 significant bits
        -i          convert log from external to internal format
        -t          alternate location for mount table (/etc/mtab)
        -x          convert log from internal to external format
        -D size     grow data/metadata section to size blks
        -L size     grow/shrink log section to size blks
        -R size     grow realtime section to size blks
        -e size     set realtime extent size to size blks
        -m imaxpct  set inode max percent to imaxpct
        -V          print version information
root@adrenalin:~# xfs_growfs -t /etc/mtab /
meta-data=/dev/md/1              isize=256    agcount=45, agsize=3449690 blks
         =                       sectsz=4096  attr=0
data     =                       bsize=4096   blocks=155236032, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal               bsize=4096   blocks=16384, version=2
         =                       sectsz=4096  sunit=1 blks
realtime =none                   extsz=262144 blocks=0, rtextents=0
data blocks changed from 155236032 to 232854048
root@adrenalin:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md/1             889G  543G  347G  62% /
/dev/sdd1              67G   22G   46G  33% /crap
/dev/md/0              92M   16M   71M  19% /boot
root@adrenalin:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid5 sde2[3] sdc3[2] sdb3[1] sda3[0]
      931416192 blocks level 5, 128k chunk, algorithm 2 [4/4] [UUUU]

md0 : active raid1 sde1[3] sdc1[2] sdb1[1] sda1[0]
      96256 blocks [4/4] [UUUU]

unused devices: 
root@adrenalin:~# cat /etc/raidtab
raiddev                 /dev/md0
raid-level              1
nr-raid-disks           4
nr-spare-disks          0
persistent-superblock   1
chunk-size              32

device                  /dev/sda1
raid-disk               0
device                  /dev/sdb1
raid-disk               1
device                  /dev/sdc1
raid-disk               2
device                  /dev/sde1
raid-disk               3

raiddev                 /dev/md1
raid-level              5
nr-raid-disks           4
nr-spare-disks          0
persistent-superblock   1
parity-algorithm        left-symmetric
chunk-size              128

device                  /dev/sda3
raid-disk               0
device                  /dev/sdb3
raid-disk               1
device                  /dev/sdc3
raid-disk               2
device                  /dev/sde2
raid-disk               3