This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature: Desktop Software

CLI Magic: Manage all your archives with atool

By Ben Martin on February 05, 2008 (9:00:00 AM)

Share    Print    Comments   

The atool package is a collection of Perl scripts that allows you to handle many different archive formats and compression schemes using a single command-line interface. Atool uses other tools behind the scenes to perform the heavy lifting. With atool, you can handle any archive without having to remember what command-line tool to use to expand it and which options that particular tool expects.

Atool comes with a collection of commands, including aunpack, apack, als, acat, adiff, and arepack. The tools are all links to the main atool Perl script, but they perform the actions that their name suggests -- for example, unpacking or creating a new archive.

By default aunpack places all extracted files into a subdirectory, except that when an archive contains only a single file, that file will by default be extracted directly into the current directory. You can override this behavior with the -subdir (or -D) option, which will force any extracted files to be placed into a subdirectory even when the archive contains only a single file. This is great for extracting an archive to /tmp and not accidentally placing all the files directly in /tmp because the archive does not have the files in a subdirectory.

Atool can handle tar files compressed with gzip, bzip2, compress, lzop, and 7z, as well ass rar, lha, ace, arj, rpm, deb, jar, 7zip, and cpio and a few other formats. You must have the native tools installed for each format in order to use atool against that format. Some archive formats can only be listed or extracted, because the program to create such an archive is not available. For example, ace archives cannot be created or updated.

Let's see how it works. In the session below I first create a trivial little filesystem and then create a tar.bz2 file. Most folks know the command to make a tar compressed with either bzip2 or gzip by heart, but as the end of the example shows, you can use the same syntax to create an archive in any supported format.

$ cd /tmp $ mkdir atest $ cd atest $ mkdir testdir $ cd testdir $ date >df1.txt $ date >df2.txt $ mkdir subdir1 $ date >subdir1/subfileb.txt $ cd /tmp/atest $ date >rawfile.txt $ apack testdir.tar.bz2 testdir testdir/ testdir/df1.txt testdir/subdir1/ testdir/subdir1/subfileb.txt testdir/df2.txt $ file testdir.tar.bz2 testdir.tar.bz2: bzip2 compressed data, block size = 900k $ tar tjvf testdir.tar.bz2 drwxrwxr-x ben/ben 0 2008-01-17 18:11 testdir/ -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/df1.txt drwxrwxr-x ben/ben 0 2008-01-17 18:11 testdir/subdir1/ -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/subdir1/subfileb.txt -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/df2.txt $ als testdir.tar.bz2 drwxrwxr-x ben/ben 0 2008-01-17 18:11 testdir/ -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/df1.txt drwxrwxr-x ben/ben 0 2008-01-17 18:11 testdir/subdir1/ -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/subdir1/subfileb.txt -rw-rw-r-- ben/ben 29 2008-01-17 18:11 testdir/df2.txt $ apack testdir.cpio.7z testdir 7-Zip (A) 4.51 beta Copyright (c) 1999-2007 Igor Pavlov 2007-07-25 p7zip Version 4.51 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) Scanning Creating archive testdir.cpio.7z Compressing testdir/df1.txt Compressing testdir/df2.txt Compressing testdir/subdir1/subfileb.txt Everything is Ok $ als testdir.cpio.7z 7-Zip (A) 4.51 beta Copyright (c) 1999-2007 Igor Pavlov 2007-07-25 p7zip Version 4.51 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) Listing archive: testdir.cpio.7z Date Time Attr Size Compressed Name ------------------- ----- ------------ ------------ ------------------------ 2008-01-17 18:11:04 ....A 29 42 testdir/df1.txt 2008-01-17 18:11:06 ....A 29 testdir/df2.txt 2008-01-17 18:11:16 ....A 29 testdir/subdir1/subfileb.txt 2008-01-17 18:11:16 D.... 0 0 testdir/subdir1 2008-01-17 18:11:11 D.... 0 0 testdir ------------------- ----- ------------ ------------ ------------------------ 87 42 3 files, 2 folders

By default atool tries to handle the 7zip format using the 7z tool. You will likely have a p7zip package available for your distribution, and may have to tell atool to use its 7za executable instead of just 7z. Atool provides a simple key/value configuration file for all users in /etc/atool.conf and on a per-user basis using ~/.atoolrc. The below configuration makes atool use the 7za executable by default.

# cat /etc/atool.conf path_7z 7za

You can get at single files within archives by using acat. Once again the command line is simple, and you do not have to worry about switches to the decompression and expansion tools in order to pick off a single file in an archive. By default the selected file is written to stdout. As shown below, you can use the quiet command-line option to suppress output from the decompression process when viewing a file in a terminal window. Unfortunately, the tools used to handle 7zip cannot write directly to the console and must be sent to a pager. By default the command pager is used to display files, though this may be overridden if the PAGER environment variable is set.

$ acat testdir.tar.bz2 testdir/df1.txt testdir/df1.txt Thu Jan 17 18:11:04 EST 2008 $ acat -q testdir.tar.bz2 testdir/df1.txt Thu Jan 17 18:11:04 EST 2008 $ export PAGER=less $ acat -q -p testdir.cpio.7z testdir/df1.txt 7-Zip (A) 4.51 beta Copyright (c) 1999-2007 Igor Pavlov 2007-07-25 p7zip Version 4.51 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) Processing archive: testdir.cpio.7z Extracting testdir/df1.txt Everything is Ok Thu Jan 17 18:11:04 EST 2008 $ acat -p testdir.cpio.7z testdir/df1.txt 2>/dev/null Thu Jan 17 18:11:04 EST 2008

Extraction of archives is where atool really shines. As mentioned before atool lets you avoid having to always check if an archive is friendly enough to extract its multiple files into a single subdirectory. In the below example I first extract the cpio file that I created before. In this case the files are already in a subdirectory, so atool did not have to perform any cleanup. I then create a jar archive which contains two files directly in the root directory of the archive. Using aunpack to expand this jar file will result in the two files being placed into a subdirectory automatically.

$ cd /tmp/atest $ mkdir extraction $ cd extraction $ aunpack ../testdir.cpio.7z ... Processing archive: ../testdir.cpio.7z Extracting testdir/df1.txt Extracting testdir/df2.txt Extracting testdir/subdir1/subfileb.txt Extracting testdir/subdir1 Extracting testdir Everything is Ok testdir.cpio.7z: extracted to `testdir' [ben@alkid extraction]$ ls -lh total 8.0K drwxrwxr-x 3 ben ben 4.0K Thu.17.Jan 18:01 testdir/ $ cat ./testdir/df2.txt Thu Jan 17 18:11:06 EST 2008 $ cd /tmp/atest/testdir [ben@alkid testdir]$ ls -l total 24 -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df1.txt -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df2.txt drwxrwxr-x 2 ben ben 4096 2008-01-17 18:11 subdir1 $ apack ../nasty-archive.jar df1.txt df2.txt adding: df1.txt (stored 0%) adding: df2.txt (stored 0%) $ cd .. $ rm -rf extraction $ cd extraction $ aunpack ../nasty-archive.jar Archive: ../nasty-archive.jar extracting: Unpack-3078/df1.txt extracting: Unpack-3078/df2.txt nasty-archive.jar: extracted to `nasty-archive' (multiple files in root) $ ls -lR .: total 8 drwx------ 2 ben ben 4096 2008-01-17 18:40 nasty-archive ./nasty-archive: total 16 -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df1.txt -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df2.txt

To automatically change into the new subdirectory that atool creates when extracting an archive -- which is the default behaviour many people will want from the command -- you have to wrap aunpack in a shell function. The shell function below is based on one from the atool manual page, with some slight modifications to subprocess execution and the ability to work around an rm command that is aliased to include -i by default.

aunpack () { TMP=$(mktemp /tmp/aunpack.XXXXXXXXXX); atool -x --save-outdir=$TMP "$@" DIR="$(cat $TMP)" [ "$DIR" != "" -a -d "$DIR" ] && cd "$DIR" /bin/rm -f $TMP }

The commands arepack and adiff make archives a convenient unit of data to work with. With arepack you can convert an archive from one format into another. The adiff command shows the output of the diff command between two archives. The args_diff option defaults to -ru for diff generation. The example shown below starts with testdir as it was in a previous example. Toward the end of the example I modify the default diff command-line options and regenerate the diff showing only the additions.

$ ls -l testdir -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df1.txt -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 df2.txt drwxrwxr-x 2 ben ben 4096 2008-01-17 18:11 subdir1 $ ls -l testdir/subdir1/ -rw-rw-r-- 1 ben ben 29 2008-01-17 18:11 subfileb.txt $ apack testdir-first.tar.gz testdir testdir/ testdir/df1.txt testdir/subdir1/ testdir/subdir1/subfileb.txt testdir/df2.txt $ mv testdir/df2.txt testdir/subdir1/ $ date >>testdir/subdir1/subfileb.txt $ date >testdir/newfileA.txt $ apack testdir-second.tar.gz testdir testdir/ testdir/newfileA.txt testdir/df1.txt testdir/subdir1/ testdir/subdir1/subfileb.txt testdir/subdir1/df2.txt $ adiff testdir-first.tar.gz testdir-second.tar.gz Only in Unpack-1710/testdir: df2.txt Only in Unpack-7823/testdir: newfileA.txt Only in Unpack-7823/testdir/subdir1: df2.txt diff -ru Unpack-1710/testdir/subdir1/subfileb.txt Unpack-7823/testdir/subdir1/subfileb.txt --- Unpack-1710/testdir/subdir1/subfileb.txt 2008-01-17 18:11:16.000000000 +1000 +++ Unpack-7823/testdir/subdir1/subfileb.txt 2008-01-17 22:39:40.000000000 +1000 @@ -1 +1,2 @@ Thu Jan 17 18:11:16 EST 2008 +Thu Jan 17 22:39:40 EST 2008 $ vi ~/.atoolrc args_diff -Nuar $ adiff testdir-first.tar.gz testdir-second.tar.gz |grep ^+ +++ Unpack-9186/testdir/df2.txt 1970-01-01 10:00:00.000000000 +1000 +++ Unpack-9186/testdir/newfileA.txt 2008-01-17 22:39:48.000000000 +1000 +Thu Jan 17 22:39:48 EST 2008 +++ Unpack-9186/testdir/subdir1/df2.txt 2008-01-17 18:11:06.000000000 +1000 +Thu Jan 17 18:11:06 EST 2008 +++ Unpack-9186/testdir/subdir1/subfileb.txt 2008-01-17 22:39:40.000000000 +1000 +Thu Jan 17 22:39:40 EST 2008

Finally, with atool you can add support for new compression schemes and archive formats without having to change the commands you use to manage and compress archives.

Ben Martin has been working on filesystems for more than 10 years. He completed his Ph.D. and now offers consulting services focused on libferris, filesystems, and search solutions.

Share    Print    Comments   

Comments

on CLI Magic: Manage all your archives with atool

Note: Comments are owned by the poster. We are not responsible for their content.

CLI != Magic

Posted by: Anonymous [ip: 169.233.27.26] on February 05, 2008 12:42 PM
The CLI is not magic. Please stop propagating this type of thinking.

#

Re: CLI != Magic

Posted by: Anonymous [ip: 71.230.125.229] on February 05, 2008 04:39 PM
lol. Right on cue! I'm sure CLI seems like magic to people who aren't familiar with how to use it. :)

#

Re: CLI != Magic

Posted by: Anonymous [ip: 87.98.217.43] on February 06, 2008 08:35 AM
Get ya a dictionary and look up the word metaphor!

#

Re: CLI != Magic

Posted by: Anonymous [ip: 213.157.29.8] on February 07, 2008 11:32 AM
> The CLI is not magic. Please stop propagating this type of thinking.

"Any sufficiently advanced technology is indistinguishable from magic" (Arthur C. Clarke 3rd law)

#

This looks pretty useful

Posted by: Anonymous [ip: 169.233.27.26] on February 05, 2008 12:43 PM
I hate having to type out bzip2 or gzip and then *respecify* the file extension. Good idea.

#

CLI Magic: Manage all your archives with atool

Posted by: Anonymous [ip: 204.234.82.253] on February 05, 2008 04:41 PM
Looks like something similar to <a href="http://hartlich.com/deco/">deco</a>. Both seem pretty useful.

#

threaded view default?

Posted by: Anonymous [ip: 169.233.27.26] on February 05, 2008 09:12 PM
Could the Linux.com folks make threaded comment views the default instead of the the flat view we all have to put up with as of now? By the way, the CAPTCHA system has repeatedly given me the same CAPTCHA, so you all might want to look into that as well.

#

CLI Magic: Manage all your archives with atool

Posted by: Anonymous [ip: 66.92.218.124] on February 07, 2008 12:35 AM
This doesn't do as much, but it's only 20 lines of shell script:
http://www.noah.org/wiki/Ax_Archive_Extract

#

CLI Magic: Manage all your archives with atool

Posted by: Anonymous [ip: 82.44.206.119] on February 19, 2008 12:14 AM
The package in the Ubuntu repositories is very out of date. Download the Debian package at the official website to be able to use 7z files

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya