Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
software:files [2011/02/07 12:50] cyril |
software:files [2024/04/13 13:11] cyril [Creation] |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Files management ====== | ====== Files management ====== | ||
+ | |||
+ | ===== Partitioning ===== | ||
+ | |||
+ | Cheatsheet for preparing and a new disk and using it with: | ||
+ | * LVM (Logical Volume Manager) | ||
+ | * Principle: **physical volumes** (hardware) are merged into **volume groups**, and then split back into **logical volumes**, which can be used as a partition (or as a disk with a partition table, but not very useful?). | ||
+ | * Advantages: flexible/ | ||
+ | * Cons: only supported in Linux, additional layer of complexity | ||
+ | * Cons for removable media: volume groups are automatically activated when the device is plugged, and need to be manually deactivated before removing it, even if we haven' | ||
+ | * LUKS | ||
+ | * Filesystems (ext4, btrfs, ...) | ||
+ | * Subvolumes | ||
+ | |||
+ | |||
+ | ==== Creation ==== | ||
+ | |||
+ | This section presents how to use each tool, in a given order, but depending on the use case you may want to skip some layers, or apply them in a different order. In particular: | ||
+ | * You may not use LVM | ||
+ | * You may want to perform LVM over LUKS instead of LUKS over LVM, if you prefer having a single password and unlock for all volumes. | ||
+ | * But you DO want to use LUKS (seriously, always encrypt your disks). | ||
+ | |||
+ | === Partitions === | ||
+ | |||
+ | * If you want to use LVM and don't need to boot on the disk, not much to do in this section, just remove all the existing partitions with '' | ||
+ | * Otherwise, create a '' | ||
+ | * If you need to boot on the disk, create the required '' | ||
+ | * If you want to use LVM, create a single large partition with the remaining space with '' | ||
+ | * Otherwise create the required system and data partitions with '' | ||
+ | |||
+ | === LVM === | ||
+ | |||
+ | * create physical volume: '' | ||
+ | * Check with '' | ||
+ | * if it complains with the error '' | ||
+ | * create volume group: '' | ||
+ | * Check with '' | ||
+ | * create logical volume: '' | ||
+ | * Check with '' | ||
+ | * ''< | ||
+ | * ''< | ||
+ | |||
+ | === LUKS === | ||
+ | |||
+ | * Encrypt the volume/ | ||
+ | * ''< | ||
+ | * Choose a strong passphrase as it can be brute-forced (at least 80 bits of entropy) | ||
+ | * By default it will configure the key derivation take 2 seconds | ||
+ | * Open (decrypt) the volume: '' | ||
+ | |||
+ | === Filesystem === | ||
+ | |||
+ | * Choose the filesystem: | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * Create the filesystem: '' | ||
+ | * Tune the filesystem: | ||
+ | * With '' | ||
+ | * Mount the filesystem: '' | ||
+ | * Some filesystem tuning must be done after mount: | ||
+ | * With '' | ||
+ | |||
+ | |||
+ | === Sub-volumes === | ||
+ | |||
+ | Some filesystems such as BTRFS and ZFS allow to create subvolumes. | ||
+ | |||
+ | * BTRFS: | ||
+ | * The filesystem root is a subvolume | ||
+ | * You can create other subvolumes: '' | ||
+ | * Check with '' | ||
+ | |||
+ | |||
+ | ==== Usage ==== | ||
+ | |||
+ | === Open === | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Close === | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Backup === | ||
+ | |||
+ | * BTRFS snapshots: | ||
+ | * A snapshot is deduplicated copy of a subvolume, using CoW (Copy-on-Write) mechanism. | ||
+ | * They are useful for storing a history with deduplication, | ||
+ | * They can be stored inside the subvolume (because they are a subvolume themselves, and snapshots are not recursive) | ||
+ | * Create a read-only snapshot: '' | ||
+ | * Delete a snapshot: '' | ||
+ | * Analyze snapshot disk usage: '' | ||
+ | * ZFS snapshots: | ||
+ | * They can be recursive. | ||
+ | |||
+ | ==== Maintenance ==== | ||
+ | |||
+ | === Renaming / Updating === | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Checks === | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Backup === | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Disk usage === | ||
+ | |||
+ | * '' | ||
+ | |||
+ | === Defragmentation === | ||
+ | |||
+ | * With '' | ||
+ | * '' | ||
+ | * can also be used to change compression of existing files (but breaks deduplication) with option '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Compression === | ||
+ | |||
+ | * With '' | ||
+ | * '' | ||
+ | |||
+ | === Resizing === | ||
+ | |||
+ | TODO | ||
+ | |||
===== Backup ===== | ===== Backup ===== | ||
Line 8: | Line 142: | ||
* partition your hard drive to have a separate partition for system and data | * partition your hard drive to have a separate partition for system and data | ||
* put important application data on the data partition (configuration, | * put important application data on the data partition (configuration, | ||
- | * do a full mirror backup of the data partition regularly (eg with rsync) on an external hard drive or a network drive. Try to keep one copy somewhere else from your home (network drive, or one at home and one at work). | + | * do a full mirror backup of the data partition regularly (eg with rsync or a deduplicate software such as Attic) on an external hard drive or a network drive. Try to keep at least one copy somewhere else from your home (network drive, or one at home and one at work). |
- | * take precautions to put the odds on your side in case of problem: make copies of your disks MBR (output of command p of fdisk), of your encrypted partitions headers... | + | * take precautions to put the odds on your side in case of problem: make copies of your disks MBR (output of command p of fdisk), of your encrypted partitions headers, etc. |
+ | |||
+ | ==== Tools ==== | ||
+ | === rsync === | ||
+ | |||
+ | === Borg Backup === | ||
+ | |||
+ | |||
+ | === Restic === | ||
+ | |||
+ | === BTRFS snapshots === | ||
+ | |||
+ | The BTRFS filesystem allows to perform some sorts of backups: | ||
+ | * On the work disk, regularly creating snapshots allows to keep an history, for recovery in case of bad manual operation | ||
+ | * It is also useful in order to " | ||
+ | * On a backup disk, snapshots can also be used to keep an history. | ||
+ | * If you update the backup with '' | ||
+ | * However if you moved or modified files on a btrfs filesystem, you can send the increment between two snapshots: '' | ||
+ | * You can also deduplicate afterwards using offline tools for out-of-band deduplication (cf [[https:// | ||
+ | |||
+ | |||
+ | |||
+ | ===== Digital Will ===== | ||
+ | |||
+ | The goal is threefold: | ||
+ | * No one non-authorized can ever access to your data | ||
+ | * The trusted persons can only access to your data in some conditions (deceased, coma, ...) | ||
+ | * You are alerted when your data is accessed by the trusted persons (in case the access was not legitimate) | ||
+ | |||
+ | Different approaches: | ||
+ | - A service that gives your data to designated persons when they provide a death certificate for you (Wishbook, ...) | ||
+ | * Cons: sending a fake document, does not work for coma, service needs to remain available, price, incomplete control. | ||
+ | * Variants: store it directly in a vault at the bank, or at the notary. | ||
+ | - A service that regularly checks that you are alive by requesting a connection with your private credentials (period can be adapted to the situations), | ||
+ | * Cons: there will be some delay between when you stop pinging and when your data becomes available, service needs to remain available, and if you host it yourself there is still a risk that your server crashes at the wrong time. | ||
+ | - A service that waits for a request to reveal your data with a personal password, sends you one or several emails to warn you that this request has been made, and in the absence of opposition from you in some delay (that can be adapted to the situation) sends your data (can be self-hosted). | ||
+ | * Cons: there will be some delay between when you stop pinging and when your data becomes available, service needs to remain available, or if you host it yourself there is still a risk that your server crashes at the wrong time (but if the service is associated to your password manager for instance, then the availability is not a problem anymore...) | ||
+ | - Split the secret between several people (cf [[https:// | ||
+ | * Cons: people need to remain accessible (and not loose the information), | ||
+ | - Store your key on a piece of opaque paper (eg visit card), with you (eg in your smartphone), | ||
+ | * Cons: need to actively monitor the integrity of your artifact, doesn' | ||
+ | * Variant: it seems that it is possible to make "paint to scratch" | ||
+ | |||
+ | Different methods could be combined, for instance 2 or 3 plus 4. But 3 managed by the password manager is probably unbeatable. | ||
+ | |||
+ | Ideally, for increased safety, the data to be obtained is always encrypted with a key that the designated persons possess. | ||
+ | |||
+ | What to transmit? | ||
+ | * Passwords (master password of your password manager, computer, encrypted data partitions, phone, ...) | ||
+ | * Instructions about what data you have | ||
+ | |||
+ | Notes: | ||
+ | * Different levels of amount of information for your spouse, children, other family, friends? | ||
+ | * How to transmit data (such as pictures) to a child? Probably has to go through a tutor. | ||
+ | |||
+ | |||
===== File Systems ===== | ===== File Systems ===== | ||
- | ==== ext3 ==== | + | ==== ext3, ext4 ==== |
=== Reserved blocks === | === Reserved blocks === | ||
- | By default ext3 reserved | + | By default ext3 reserves |
You can check and remove these reserved blocks with the following commands: | You can check and remove these reserved blocks with the following commands: | ||
Line 39: | Line 229: | ||
First unmount your partition and remount it read-only. | First unmount your partition and remount it read-only. | ||
- | * testdisk (photorec) | + | * '' |
- | * ext3grep < | + | * '' |
+ | * '' | ||
ext3grep < | ext3grep < | ||
ext3grep < | ext3grep < | ||
Line 48: | Line 239: | ||
ext3grep < | ext3grep < | ||
</ | </ | ||
+ | * '' | ||
* others: debugfs, foremost, [[http:// | * others: debugfs, foremost, [[http:// | ||
Line 95: | Line 287: | ||
* testdisk (very good) [[http:// | * testdisk (very good) [[http:// | ||
* gpart (didn' | * gpart (didn' | ||
+ | |||
+ | ==== Performance optimization ==== | ||
+ | |||
+ | * e4rat ([[http:// | ||
+ | * preload ([[http:// | ||
+ | * prelink | ||
+ | |||
+ | * verynice | ||
+ | |||