WildBill's Blogdom

Mongo only pawn, in game of life.

Quick Tip: Compression Is Useful.

| Comments

I was recently faced with having to make a copy of a bunch of data – nearly a terabyte – to backup a server before a major software upgrade on it. A test run of the copy looked like it’d take about 24 hours for the data to copy using “cp -Rp”. I wasn’t happy with that and wanted to cut it down, so I tried making a compressed tarball of the data instead, using “tar czvf”. Guess what? Creating the tarball took about half the time, just over 12 hours. If you think about it, this makes sense… you can’t change the amount of data you read in during a copy, but you can change the amount you write out through compression. By trading disk I/O for CPU, I was able to nearly cut the copy time in half. This also works for restoring the data too, just remember that you’re trading disk I/O for CPU. :)