This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Take advantage of multiple CPU cores during file compression

Posted by: Anonymous [ip: 206.230.48.34] on February 13, 2008 02:16 PM
I also wrote a multi-threaded zip utility called ZipMt (http://code.google.com/p/zipmt). ZipMT uses bzip2 but unlike pbzip2 it can also compress data from a pipeline using multiple CPUs. It does not support multi-threaded decompression but it is fully compatible with the standard bunzip2. In general though, decompression is not nearly as CPU intensive as compression so there is less benefit to using a parallel approach (you're mostly disk bound anyway).

Feel free to try out ZipMT since it's free and the source and binaries are available on code.google.com. There is a windows version too that was compiled with cigwin.

-Drew

#

Return to Take advantage of multiple CPU cores during file compression