Files management

Backup

Everyone has personal data that nothing could recreate (pictures, emails, creations, …), or global data and configuration that it would take a lot of time to recreate. However you can lose some of them or all of them in several situations: hard drive crash, hard drive corruption, computer theft, computer destruction (fire…).

My advice:

  • partition your hard drive to have a separate partition for system and data
  • put important application data on the data partition (configuration, emails, …)
  • do a full mirror backup of the data partition regularly (eg with rsync or a deduplicate software such as Attic) on an external hard drive or a network drive. Try to keep at least one copy somewhere else from your home (network drive, or one at home and one at work).
  • take precautions to put the odds on your side in case of problem: make copies of your disks MBR (output of command p of fdisk), of your encrypted partitions headers, etc.

File Systems

ext3, ext4

Reserved blocks

By default ext3 reserves 5% of disk space to super-user. The intent is to let to critical applications the ability to write to the disk when it is full, but it has no use for a data partition, you just waste 5% of your partition.

You can check and remove these reserved blocks with the following commands:

tune2fs -l /dev/sda1 | grep Reserved
tune2fs -r 0 /dev/sda1

Duplicates

  • geekie for images (fork of gqview)
  • fdupes
  • fslint

Secure delete

  • secure-delete (srm, sfill, sswap, smem)
    • -l option to be a lot faster: 2 passes instead of 38 (or -ll for only 1 pass), enough to prevent the use of consumer tools like photorec, but not for specialized companies and governments ;-)
  • shred (less advanced but more common)

Data recovery

First unmount your partition and remount it read-only.

  • extundelete –restore-file Documents/file.dat /dev/sda4 : the easiest solution if there are only a few files and you know their name. Accepts not unmounting the partition, works generally ok if you do it immediately after removing the files.
  • testdisk (photorec) is great to recover files on a mobile storage device because it works with any filesystem (finds signatures in data so no need of journal), and find all deleted files on the partition.
  • ext3grep
    ext3grep <partition> --restore-file <filename> # filename => file ; works great, but only for one file at a time...
    ext3grep <partition> --restore-all --deleted --after=1270639550 # dates -> files
    ext3grep <partition> --histogram=dtime --deleted --after=1270639000 --before=1270640000 # => dates
    ext3grep <partition> --ls --inode 2 # filenames => inodes (navigating in directories with inodes)
    ext3grep <partition> --search Libs/jafar/modules/ # filename,dates -> blocks
    ext3grep <partition> --restore-inode <inode> # inodes => files

    Notes: “restore-all” failed while building stage2 cache with error “ext3grep: init_directories.cc:535: void init_directories(): Assertion `lost_plus_found_directory_iter != all_directories.end()' failed.”. However doing a “ls inode” created this stage2 cache, and afterwards “restore-all” worked… but just restored everything on the disk even not deleted files/dirs, not taking into account the “after” option… But manually editing the stage2 cache to only keep files/dirs you want to restore then “restore-all” worked perfectly!

  • others: debugfs, foremost, unrm, ext3undel

Disk Recovery

In case the MBR/partition table of you disk is damaged.

Make a backup before

You should always keep a backup of your partition table !

The first way is to store the output of p command of fdisk.

You can also do a dump of the MBR and EBR:

dd if=/dev/sda of=sda.dd bs=512 count=1 # full MBR dump
sfdisk -d /dev/sda > sda.sfdisk         # MBR and EBR partition tables

Out of curiosity, the file command is able to interpret the content of your MBR dump:

file sda.dd

Restore with a backup

If you have the output of the p command of fdisk, then you can manually recreate the partition table with fdisk with the same information. As long as you don't mount or format, modifying the partition table with fdisk doesn't modify the partitions data.

If you have a full dump of MBR and EBR, you can automatically restore it:

dd if=sda.dd of=/dev/sda
sfdisk /dev/sda < sda.sfdisk

To restore the MBR without the partition table:

dd if=sda.dd of=/dev/sda bs=446 count=1

To restore only the partition table:

dd if=sda.dd of=/dev/sda bs=1 skip=446 count=66

Restore without a backup

If you don't have a copy of your partition info, don't panic, some software can recover them by searching for the partitions in the disk content (but it has to be formatted as a standard filesystem, ie not encrypted):

  • testdisk (very good) howto
  • gpart (didn't work very well for me, only found the NTFS partition) howto

Performance optimization

  • verynice
software/files.txt · Last modified: 2015/07/30 21:02 by cyril
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki