How to recover lost data on an ext3/4 partition

I recently had a late-night oopsie with tar, overwriting the file I wanted to package (source code I had worked on for a week). After a few seconds of creepy feelings, I remembered that I use a journaling filesystem and thus, there would be a good chance that there would be several copies of the data still physically lying around on the storage medium.

So, what to do when you lost a file? For your first action, you have two options.

First option: Power off the system without shutting it down, then boot from a live CD. The foolproof option.

Second option: If you can ensure that no new files will be created on the filesystem where the loss occured while you are trying to recover the data, you can work from the running system and avoid further potential data loss from uncleanly powering down the system.

Next, become root and grep the filesystem where the data loss took place.

grep -a -B300 -A300 "searchString" /dev/sdb1 > dumpfile

searchString is a piece of text that preferably only occurs in your lost file. Try to be as specific as possible so you ideally only get results from the file's contents. The -B and -A options specify the number of context lines to output before and after each match. Choose values large enough to include the entire lost file, if possible. Replace /dev/sdb1 with the device node of the filesystem where the lost data is. You can find out the node by just running mount without parameters. dumpfile is the target file containing the found text. Of course the dump file should not be saved on the partition where you lost the data! Save it on another filesystem or a tmpfs if you have one set up. Mounting a USB stick or something for the dump file may be risky because a device node will be created on the root partition, which stinks if your lost data sits exactly there.

After grep is done, examine the dump file with a text editor. If you are certain that it contains your entire lost file in a usable or recoverable state, you can start copying and pasting into empty files, restoring various versions of your file of which a few will probably be current.

Store volatile data in RAM using tmpfs

There are several reasons why you might want to hold certain data in non-permanent memory. Applications that create a lot of temporary data, causing system slowdown and strain on the storage medium, are an example, or you want the data to be gone after every power cycle because it is of no use anymore at that point, or the data would simply eat up too much storage over time.

Unix-like platforms support a file system named tmpfs, which is a RAM filesystem growing and shrinking dynamically with its contents. Setup is straightforward. To mount /tmp as tmpfs, you add the following line to /etc/fstab, after the physical partition mounts:

none /tmp tmpfs defaults 0 0

Be sure to clear out the contents of the old /tmp directory before remounting, because the mount will "overlay" what was previously there and the data will not be accessible but still occupy space.

Other locations that are good candidates for a tmpfs: /var/tmp, /home/username/tmp, /var/cache/apt/archives, web browser and other caches in your home directory. Don't be tempted to put /var/log on a tmpfs because the logs are important information sources for troubleshooting and forensics. Use logrotate instead for limiting the amount of log storage.

When you are done setting up the temporary file systems, issue a mount -a to remount everything from fstab.