LZOP is my friend

I’ve been doing a lot of disk cloning lately, working up instructions for duplicating a “sysprep” style Fedora image for the LANforge product. Now that live CDs tend to allow live installation of packages, I can boot one, plugin in a stick with the archive of my raw image, and install pv and lzop and I’m off to the races.

First you need to destroy whatever was on /dev/sda first of course:

wipefs -af /dev/sda

Install your utilities:

dnf install -y pv lzop

Then throw on your image:

pv -pet -B $((1024*1024*1024)) /media/images/frozen-sda.raw.lzop \
| lzop -dc | dd iflag=fullblock oflag=direct bs=1M of=/dev/sda

I think this is super cool. After that gets thrown on the drive, you need to run parted on it to correct any partition glitches. Fsck the paritions after that.

 

Raw ZVOL access in ZFS on Linux not an option for VirtualBox?

I have gotten about as far as I can go…I can boot a Fedora netinstall cd, and I can attempt to fdisk /dev/sda, but this device will not open in fdisk. This might be consistent with no “rdsk” device in the dev tree. Funny though, I can fdisk /dev/zd0, and it lists partitions.

Could this be a problem with how I have my vmdk setup?

Two drives fail at once? Oh yeah…

Just as I thot I was all cool for having a sixteen drive NAS, today’s opening of it and trying a new network card (did not fit) left me with bad news on the next powerup.

 > dmesg | grep ata | grep error:
[   23.223221] ata13.00: error: { ABRT }
[   23.234448] ata13.00: error: { ABRT }
[   31.262674] ata13.00: error: { ABRT }
[   31.275241] ata13.00: error: { ABRT }
[   31.288012] ata13.00: error: { ABRT }
[   39.073802] ata13.00: error: { ABRT }
[   50.815339] ata13.00: error: { ABRT }
[   50.827082] ata13.00: error: { ABRT }
[   57.606645] ata13.00: error: { ABRT }
[   69.616356] ata7.00: error: { ABRT }
[   69.616451] ata13.00: error: { ABRT }

That’s failure of two drives. TWO at the same time! ….and look at this:

 > zpool status -v
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: scrub repaired 0 in 4h43m with 0 errors on Sat Sep  6 00:13:22 2014
config:

        NAME                                            STATE     READ WRITE CKSUM
        tank                                            ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G9PDPC  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G9SBBC  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G6GMGC  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G95REC  ONLINE       0     0     0
          raidz1-1                                      ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G9LH9C  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G95JPC  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G6LUDC  ONLINE       0     0     0
            ata-Hitachi_HTS547575A9E384_J2190059G5PXYC  ONLINE       0     0     0
          raidz1-2                                      ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_X3EJSVUOS            ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_X3EJSVUNS            ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_933PTT11T            ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_933PTT17T            ONLINE       0     0     0
          raidz1-3                                      ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_933PTT12T            ONLINE       0     0     0
            ata-TOSHIBA_MQ01ABD050_933PTT13T            ONLINE       0     0     2
            ata-TOSHIBA_MQ01ABD050_933PTT14T            ONLINE       0     0     2
            ata-TOSHIBA_MQ01ABD050_933PTT0ZT            ONLINE       0     0     0
        logs
          ata-OCZ-AGILITY4_OCZ-77Z13FI634825PNW-part5   ONLINE       0     0     0
        cache
          ata-OCZ-AGILITY4_OCZ-77Z13FI634825PNW-part6   ONLINE       0     0     0

errors: No known data errors

Two checksum errors in the same Raid 5 volume. That’s going to be a very tricky replacement. I think I’m going to either replace one disk at a time and hope for the best resilver possibilities, or maybe…add a PCI controller back in there and add another zvol and migrate data from one zvol to another? That’ll be a wild trick.

It will frack up my backups for a while, that’s for sure. Oh, and those Toshiba drives? That’s three Toshiba failures, zero Hitatchi failures.