Take advantage of multiple CPU cores during file compression
Posted by: Anonymous
on February 13, 2008 02:16 PM
I also wrote a multi-threaded zip utility called ZipMt (http://code.google.com/p/zipmt). ZipMT uses bzip2 but unlike pbzip2 it can also compress data from a pipeline using multiple CPUs. It does not support multi-threaded decompression but it is fully compatible with the standard bunzip2. In general though, decompression is not nearly as CPU intensive as compression so there is less benefit to using a parallel approach (you're mostly disk bound anyway).
Feel free to try out ZipMT since it's free and the source and binaries are available on code.google.com. There is a windows version too that was compiled with cigwin.