This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new!

Feature: Desktop Software

When files disappear, Magic Rescue saves the day

By Bruce Byfield on February 14, 2008 (9:00:00 AM)

Share    Print    Comments   

If you've ever had that sick realization that you made a mistake immediately after emptying your Trash or deleting a file with Shift-Del, then Magic Rescue may be the cure you're looking for. Magic Rescue searches block devices for particular file types, then restores them to a designated directory where you can sort through them.

Although subject to certain limitations, such as how recently a file was deleted and the availability of a definition for the file header of a given format, Magic Rescue is not difficult to use. It even features a man page with a few mini-tutorials. However, it does require organization and planning in order to use effectively.

Setting up

Before you start to use Magic Rescue, you need two things: A directory to hold recovered files, and a recipe for the file type you are trying to recover.

To prevent feedback loops that can trash the system and possibly overwrite the files you are trying to recover, the directory should not be on the block device you are searching. If your system only has one partition, consider mounting a flash drive or external hard drive to hold the directory. If you have multiple partitions, you need to make sure that the directory is on a partition that has as much free space as you need for the recovered files -- as the man page notes, some searches, especially for graphic and audio files, result in hundreds of large files, so you need to be ready for them. On my system, 3GB of free space was more than enough, but depending on your work and download habits, you might need more.

A recipe is a small script that recognizes the characteristic pattern of a file format's header. If you are familiar with different file types -- or willing to research them -- you can write your own recipe, using information in the man page as a guide. When you have finished writing a recipe, you can use /usr/share/magicrescue/tools/checkrecipe to test it.

Most people, though, will probably either use the recipes installed with Magic Rescue in /usr/share/magicrescue/recipes, or search the Internet for a specific recipe. The latest version comes with recipes for identifying avi, elf, gimp-xcf, gzip, jpeg, mp3, Microsoft Office, perl, png, and zip files, as well as files, and files with the GNU General Public License in the header. These recipes are also useful as examples if you need to write your own recipe.

Running Magic Rescue

The man page suggests that you run the command hdparm -d1 -c -u1 /dev/device to enable direct memory access before running Magic Rescue. The command is not strictly necessary, but it can significantly reduce the time that the program takes to run. However, you may prefer to tweak performances by limiting the operation in other ways provided by the command parameters (see below).

To run Magic Rescue, you must specify a minimum of a results directory and a recipe. The basic command is magicrescue -d directory -r recipe device or, to give an example, magicrescue -d /mnt/external -r /usr/share/magicrescue/recipes/zip /dev/sda1. You can enable searches for multiple formats by specifying a directory that holds all the recipes for those formats.

If you want a running record of results, you can add - M i0 to view each input and output file processed.

You can use the -b blocksize parameter to limit results to files that start at a multiple of the blocksize specified. The man pages suggest a blocksize of 512 for most purposes.

If you are comfortable with hexadecimal numbers, you can also specify a specific position on the partition to search with -O = position, or -O + position to start the search after a position, or -O - position to start before it. The -O parameter is especially useful if you have to use Ctrl-C to interrupt a long search. If you note the current position of the search, you can use O = to continue the search later from the position where it stopped.

Utilities for after the search

To further help you organize your search, Magic Rescue includes two utilities in /usr/share/magicrescue/tools. By using the command dupemap delete,report resultdirectory, you can eliminate all duplicate files in your result directory. If you first use dupemap report -dfile over multiple directories, you can create a database of files, then add -dfile to the command to eliminate files elsewhere on your system.

Alternatively, magicsort resultdirectory uses the file command to move each unique result in the directory to a separate file directory.

Other recovery methods

Magic Rescue's man page ends with the disclaimer, "Magic Rescue is not meant to be a universal application for file recovery. It will give good results when you are extracting known file types from an unusable file system, but for many other cases there are better tools available." Among the tools it recommends is gpart when you are searching for intact partitions, The Sleuth Kit for undamaged partitions (despite its limited support for different types of partitions), and Foremost for cases where Magic Rescue lacks a recipe.

Although it's not mentioned in the man pages, you might also want to investigate GRescue, a GNOME version of Magic Rescue now in the early stages of development.

All these are potentially useful programs, but you may find the man page disclaimer overly modest. While other programs have a larger set of options and utilities, whether you are working with damaged or intact filesystems, once you have the recipe you need, you may find that Magic Rescue suffices for file recovery.

Bruce Byfield is a computer journalist who writes regularly for

Share    Print    Comments   


on When files disappear, Magic Rescue saves the day

Note: Comments are owned by the poster. We are not responsible for their content.

nice article

Posted by: Michael Shigorin on February 14, 2008 07:41 PM
Thanks Bruce, this kind of article is what comes like water upon googling in dire situation!
(I am not, at least today)


When files disappear, Magic Rescue saves the day

Posted by: Anonymous [ip:] on February 14, 2008 07:50 PM
This could be very hard to use. Say I have a text file I accidentally deleted. I guess for the rules I have to know the beginning text. I am trying this program out it just doesn't seem that useful to me.

I created a file like so
echo Deletetest with more stuff so I can see it working> Deletetest.txt

removed it like so
rm -f Deletetest.txt

used this as the ruleset
echo -e "0 string Deletetest\nextention txt\ncommand safecat \"\$1\" > \$1">rules.txt

created a place to put saved files like so
mkdir /tmp/rescued

then used magicrescue as follows
magicrescue -r rules.txt /dev/VolGroup00/LogVol00 -d /tmp/resuced

Maybe I just don't understand something but it just wasn't working too well.
one of the files it rescues should be right no?


Re: When files disappear, Magic Rescue saves the day

Posted by: Anonymous [ip:] on February 15, 2008 11:52 AM
Did your test file actually get written to disk?? maybe the
echo '...' > Deletetext.txt ; rm -f Deletetext.txt
command was performed entirely in cache without syncing to disk. In that case there would nothing to rescue.

Btw, in case of text file, it is IMO better to use grep.


Re(1): When files disappear, Magic Rescue saves the day

Posted by: Anonymous [ip:] on February 16, 2008 09:33 AM
Grep? For deleted files? Please enlighted me, noob that I am. Thanks.


Re(2): When files disappear, Magic Rescue saves the day

Posted by: Anonymous [ip:] on February 19, 2008 10:10 PM
grep. as in: grep -aC1000 some_fragment_of_the_deleted_file /dev/sda1

Saved my ass a few times.


Others rescue utilities

Posted by: Anonymous [ip:] on February 16, 2008 08:25 PM
Here are link for others programs that can rescue or protect your data: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Watch out for ext3

Posted by: Anonymous [ip:] on February 18, 2008 06:18 AM
I learnt the hard way that there is no 'simple' undelete on ext3 filesystems.

I'm posting this to make others aware. For example, if you accidentially delete a directory, don't assume that you will be able to recover most of it - or any of it at all.

The behaviour is different than ext2, and other filesystems you may have encountered like FAT and NTFS where you had a fighting chance of getting most or all of it back as long as you didn't write to the partition.

In my case, I deleted a user in the Webmin admin interface. For some reason which I haven't figured out, the /home/ directory was deleted. Talk about a sick feeling...

The tools in the article allow you to recover files by detecting their content, which is a lifesaver in some cases, but didn't help me.


When files disappear, Magic Rescue saves the day

Posted by: Anonymous [ip:] on February 26, 2008 07:20 PM
like the previous poster mentioned, ext3 is pretty nasty when it comes to file recovery. As stated in the wikipedia article
"Unlike ext2, ext3 zeroes out block pointers in the inodes of deleted files. It does this to simplify read-write access to the filesystem when the journal is being replayed after an unclean mount. This, however, effectively prevents files from being undeleted."

Which is also part of why ext3 is so slow on large delete operations. Some view this as a good thing (added security) but it's pretty annoying when you are the one trying to restore a file. Btrfs can't come fast enough in my opinion.

In any event, these utilities still come in quite handy on corrupted flash disks like camera storage or usb flash drives, which are often formatted to fat16/fat32


This story has been archived. Comments can no longer be posted.

Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya