IT Dribble

Sometimes you just *need* to mount an image file under Linux (ie. forensics and/or data recovery).
This isn’t always easy if you DD the disk, then you need to work out the partition maths.

Easiest way is to ‘fisk -l’ the image file:

root@HackerBox:~/forensics# fdisk -l /mnt/temp/ewf1
Disk /mnt/temp/ewf1: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x39bf39be

Device                   Boot   Start      End           Sectors      Size Id  Type
/mnt/temp/ewf1p1 *         63         20948759 20948697  10G  7   HPFS/NTFS/exFAT

From the above you should see ‘Sectors’ are 512 bytes (pretty normal for NTFS) and the partition starts at 63 (sectors) in.

So finally all we need to do is mount the image with the command:

mount /mnt/temp/ewf1 /mnt/temp1 -o ro,loop,show_sys_files,streams_interace=windows,offset=$((63*512))

You can possibly leave out the ’show_sys_files,streams_interace=windows’ parameters if you aren’t doing forensics.

Previously I always had to fumble around using FDISK, do some dangerous task of deleting the partition table while in use, then add re-adding the table with the new extended settings!
Too difficult and too danger prone (although it never failed on me…)

Anyway I just stumbled on a way easier process! Introducing ‘growpart’.
The process for resizing EXT4 partition, ‘/dev/xvda1′ would be:

Backup the partition table (just in case)

sfdisk -d /dev/xvda > partition_backup

Resize the partition

growpart -v /dev/xvda 1

And finally resize the filesystem to make use of the larger partition

resize2fs /dev/xvda1

Happy resizing.

Note: Quick investigation it seems ‘growpart’ is from the RPM ‘cloud-disk-utils’ from Amazon AWS hosted systems. So not sure about regular avaliability.

Well before you power to replace the disk - can you be sure it will power back on?
Some distros have the bad habit of only install GRUB to one physical disk and if that disk dies…RAID wont save you (but a boot CD will…)

Firstly I like to confirm if /boot is configured in RAID 1. I then usually also install GRUB to all the physical disks via:

grub-install /dev/sda

and sdb and sdc etc. Then I power off the server and replace the disk.

After powering on the server, sometimes it won’t boot due to the new disk getting booted first, so make sure to select another disk in the BIOS boot-up menu.

After the operating system returns it’s a matter of recreating the partitions on the new disk, before trying to add it back into the RAID array. My servers all have the same disk sizes in the array and the same partition layouts, so to recreate them on the old disk I just perform:

sfdisk -d /dev/existing-disk | sfdisk /dev/new-disk

Confirm the new disks get the correct layout via:

cat /proc/partitions

and then add the partition back to the RAID array:

mdadm --add /dev/mdX /dev/sdXX

And then finally confirm its rebuilding via:

mdadm --detail /dev/mdX

OR

cat /proc/mdstat

For a raw filesystem try:

fdisk -l harddrive.img
mount -o ro,loop,offset=xxxxxxxxx harddrive.img /mnt/loop

or for filesystems with volume groups etc try:

losetup /dev/loop0 disk.img
kpartx -a /dev/loop0

Then to mount the first partition:

mount /dev/mapper/loop0p1 /mnt

Or to activate the volume group then mount the logical volume:

vgscan
vgchange -ay vg
mount /dev/vg/lv /mnt

Hope that helps.

So many times I need to measure the amount of IOPS on a Linux disk/storage system. While there are many tools for the jobs they just don’t seem to give you a ‘number’. For example Splunk indexers require 1200+ IOPS according to hardware recommendation guides but how do you find out if your any where close to that number? Use ‘bonnie++’, ‘iozone’ or perhaps ‘fio’? Well use any of those tools will create the type of read / write sequence you would like to replicate - but where the damn magic number???

Easiest two ways are:
Method #1:
run iozone -a (or bonnie++) in one screen then in another session / terminal use nmon, pressing D (capital D) to get disk stats and get the number from the Xfers column. This is your magic number (or IOPS reading)

┌nmon─14i─────────────────────Hostname=reddragon─────Refresh= 2secs ───19:51.57─
│ Disk I/O ──/proc/diskstats────mostly in KB/s─────Warning:contains duplicates─
│DiskName Busy    Read    Write       Xfers   Size  Peak%  Peak-RW    InFlight
│sda       99%    699.9     14.0KB/s  178.0   4.0KB  493%    3658.8KB/s   1   
│sda1       0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0   
│sda2      99%    699.9     14.0KB/s  178.0   4.0KB  493%    3658.8KB/s   1   
│dm-0       0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0   
│dm-1      99%    699.9     14.0KB/s  178.5   4.0KB  494%    3658.8KB/s   1   
│dm-2       0%      0.0      0.0KB/s    0.0   0.0KB   76%    2553.5KB/s   0   
│Totals Read-MB/s=2.1      Writes-MB/s=0.0      Transfers/sec=534.4 

In the above example I’m getting about 178 IOPS for my disk ’sda’

Method #2:
run fio with the correct workload (google how to use fio) and while it’s running it will actually tell you the IOPS.

[root@reddragon ~]# fio random-read-test.fio 
random-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.0.13
Starting 1 process
Jobs: 1 (f=1): [r] [85.1% done] [736K/0K/0K /s] [184 /0 /0  iops] [eta 00m:28s]

In this example I am getting 184 IOPS. Also if you wait until fio finishes it run - you can the IOPS reading from there. Eg.

random-read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.0.13
Starting 1 process
Jobs: 1 (f=1): [r] [98.9% done] [2224K/0K/0K /s] [556 /0 /0  iops] [eta 00m:02s]
random-read: (groupid=0, jobs=1): err= 0: pid=7239: Tue Feb 25 16:49:16 2014
  read : io=131072KB, bw=747406 B/s, iops=182 , runt=179578msec
    clat (usec): min=107 , max=117530 , avg=5473.62, stdev=4112.08
     lat (usec): min=107 , max=117531 , avg=5473.93, stdev=4112.08
    clat percentiles (usec):
     |  1.00th=[  245],  5.00th=[  302], 10.00th=[  370], 20.00th=[ 2480],
<SNIP>

As you can see: iops=182 - pretty consistent with the other results!