Max out your IOPs with GNU Parallel

The GNU Parallel program is a clever way to simultaneously operate thing on multiple files (and other things). Tonight, we create the most IO intensive way to copy virtual machine images from one system to another: parallel rsync.

My source system is an 8-core E5 with 32G of ram. I’m trying to backup 400+GB of VM images.

parallel-100

And watch those eight 3-TB drives spin their darned hearts out, even in a RAID10 array.

parallel-101

Just shows you that you can have plenty of disk IO, but your processor is still faster.

Discover more from Bitratchet

Subscribe now to keep reading and get access to the full archive.

Continue reading