Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
software:files [2022/12/24 00:07] cyril [Digital Will] |
software:files [2024/04/19 14:27] cyril [Maintenance] |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Files management ====== | ====== Files management ====== | ||
+ | |||
+ | ===== Partitioning ===== | ||
+ | |||
+ | Cheatsheet for preparing and a new disk and using it with: | ||
+ | * LVM (Logical Volume Manager) | ||
+ | * Principle: **physical volumes** (hardware) are merged into **volume groups**, and then split back into **logical volumes**, which can be used as a partition (or as a disk with a partition table, but not very useful?). | ||
+ | * Advantages: flexible/ | ||
+ | * Cons: only supported in Linux, additional layer of complexity | ||
+ | * Cons for removable media: volume groups are automatically activated when the device is plugged, and need to be manually deactivated before removing it, even if we haven' | ||
+ | * LUKS | ||
+ | * Filesystems (ext4, btrfs, ...) | ||
+ | * Subvolumes | ||
+ | |||
+ | |||
+ | ==== Creation ==== | ||
+ | |||
+ | This section presents how to use each tool, in a given order, but depending on the use case you may want to skip some layers, or apply them in a different order. In particular: | ||
+ | * You may not use LVM | ||
+ | * You may want to perform LVM over LUKS instead of LUKS over LVM, if you prefer having a single password and unlock for all volumes. | ||
+ | * But you DO want to use LUKS (seriously, always encrypt your disks). | ||
+ | |||
+ | === Partitions === | ||
+ | |||
+ | * If you want to use LVM and don't need to boot on the disk, not much to do in this section, just remove all the existing partitions with '' | ||
+ | * Otherwise, create a '' | ||
+ | * If you need to boot on the disk, create the required '' | ||
+ | * If you want to use LVM, create a single large partition with the remaining space with '' | ||
+ | * Otherwise create the required system and data partitions with '' | ||
+ | |||
+ | === LVM === | ||
+ | |||
+ | * create physical volume: '' | ||
+ | * Check with '' | ||
+ | * if it complains with the error '' | ||
+ | * create volume group: '' | ||
+ | * Check with '' | ||
+ | * create logical volume: '' | ||
+ | * Check with '' | ||
+ | * ''< | ||
+ | * ''< | ||
+ | |||
+ | === LUKS === | ||
+ | |||
+ | * Encrypt the volume/ | ||
+ | * ''< | ||
+ | * Choose a strong passphrase as it can be brute-forced (at least 80 bits of entropy) | ||
+ | * By default it will configure the key derivation take 2 seconds | ||
+ | * Open (decrypt) the volume: '' | ||
+ | |||
+ | === Filesystem === | ||
+ | |||
+ | * Choose the filesystem: | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * Create the filesystem: '' | ||
+ | * Tune the filesystem: | ||
+ | * With '' | ||
+ | * Mount the filesystem: '' | ||
+ | * Some filesystem tuning must be done after mount: | ||
+ | * With '' | ||
+ | |||
+ | |||
+ | === Sub-volumes === | ||
+ | |||
+ | Some filesystems such as BTRFS and ZFS allow to create subvolumes. | ||
+ | |||
+ | * BTRFS: | ||
+ | * The filesystem root is a subvolume | ||
+ | * You can create other subvolumes: '' | ||
+ | * Check with '' | ||
+ | |||
+ | |||
+ | ==== Usage ==== | ||
+ | |||
+ | === Open === | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Close === | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Backup === | ||
+ | |||
+ | * BTRFS snapshots: | ||
+ | * A snapshot is deduplicated copy of a subvolume, using CoW (Copy-on-Write) mechanism. | ||
+ | * They are useful for storing a history with deduplication, | ||
+ | * They can be stored inside the subvolume (because they are a subvolume themselves, and snapshots are not recursive) | ||
+ | * Create a read-only snapshot: '' | ||
+ | * Delete a snapshot: '' | ||
+ | * Analyze snapshot disk usage: '' | ||
+ | * ZFS snapshots: | ||
+ | * They can be recursive. | ||
+ | |||
+ | ==== Maintenance ==== | ||
+ | |||
+ | === Renaming / Updating === | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Checks === | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Backup === | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Disk usage === | ||
+ | |||
+ | * '' | ||
+ | |||
+ | === Defragmentation === | ||
+ | |||
+ | * With '' | ||
+ | * '' | ||
+ | * can also be used to change compression of existing files (but breaks deduplication) with option '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Compression === | ||
+ | |||
+ | * With '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === SSD TRIM === | ||
+ | |||
+ | * TRIM (or discard) operation means informing the SSD drive about the unused memory, so that it can perform efficiently wear leveling. | ||
+ | * Checking TRIM support: run '' | ||
+ | * Each layer must forward the TRIM commands to the layer above, until it reaches the drive: | ||
+ | * dm-crypt. Depending on how the encrypted volume is mounted, different possibilities: | ||
+ | * '' | ||
+ | * set option '' | ||
+ | * set option '' | ||
+ | * LVM | ||
+ | * Two options are available: | ||
+ | * Continuous TRIM, i.e. configuring the filesystem to notify instantly each block that is freed. | ||
+ | * It is not advised because can cause issues. | ||
+ | * Periodic TRIM, i.e. explicitly notifying the free blocks periodically. | ||
+ | * Using the '' | ||
+ | |||
+ | |||
+ | Source : https:// | ||
+ | |||
+ | === Resizing === | ||
+ | |||
+ | TODO | ||
+ | |||
===== Backup ===== | ===== Backup ===== | ||
Line 10: | Line 164: | ||
* do a full mirror backup of the data partition regularly (eg with rsync or a deduplicate software such as Attic) on an external hard drive or a network drive. Try to keep at least one copy somewhere else from your home (network drive, or one at home and one at work). | * do a full mirror backup of the data partition regularly (eg with rsync or a deduplicate software such as Attic) on an external hard drive or a network drive. Try to keep at least one copy somewhere else from your home (network drive, or one at home and one at work). | ||
* take precautions to put the odds on your side in case of problem: make copies of your disks MBR (output of command p of fdisk), of your encrypted partitions headers, etc. | * take precautions to put the odds on your side in case of problem: make copies of your disks MBR (output of command p of fdisk), of your encrypted partitions headers, etc. | ||
+ | |||
+ | ==== Tools ==== | ||
+ | === rsync === | ||
+ | |||
+ | === Borg Backup === | ||
+ | |||
+ | * Create a Borg repository in the current folder: < | ||
+ | * ''< | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * Create archives: < | ||
+ | --compression auto, | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | === Restic === | ||
+ | |||
+ | === BTRFS snapshots === | ||
+ | |||
+ | The BTRFS filesystem allows to perform some sorts of backups: | ||
+ | * On the work disk, regularly creating snapshots allows to keep an history, for recovery in case of bad manual operation | ||
+ | * It is also useful in order to " | ||
+ | * On a backup disk, snapshots can also be used to keep an history. | ||
+ | * If you update the backup with '' | ||
+ | * However if you moved or modified files on a btrfs filesystem, you can send the increment between two snapshots: '' | ||
+ | * You can also deduplicate afterwards using offline tools for out-of-band deduplication (cf [[https:// | ||
+ | |||
+ | |||
+ | |||
+ | ===== Container files ===== | ||
+ | |||
+ | Sources: | ||
+ | * [[https:// | ||
===== Digital Will ===== | ===== Digital Will ===== |