Max out your IOPs with GNU Parallel

The GNU Parallel program is a clever way to simultaneously operate thing on multiple files (and other things). Tonight, we create the most IO intensive way to copy virtual machine images from one system to another: parallel rsync.

My source system is an 8-core E5 with 32G of ram. I’m trying to backup 400+GB of VM images.

parallel-100

And watch those eight 3-TB drives spin their darned hearts out, even in a RAID10 array.

parallel-101

Just shows you that you can have plenty of disk IO, but your processor is still faster.