How to change the filesystem in your partition disk

This can be a painful and slow experience so I wanted to share some tips about this topic :slight_smile:

First of all, using tools that automatically convert your filesystem is dangerous and not reliable, you can lose your data! So the best option is always to just transfer the data, this also ensures their consistency and health of your data located in your disks

The only tool that I recommend for that is using rsync, this is the only tool that provides everything needed, secured, fast, integrity, correct copy of file types, reliable, and you can even stop a copy and continue another day

Concept

Let's say that I want to move from reiserfs to btrfs, as:

  • My disk uses reiserfs, and even if is good, seems like no more well supported in new kernels (it crash your entire system by using it)
  • I want to convert it to the new BTRFS filesystem which supports transparent compression, because I want to keep my files use the less space possible on disk and it also has good performances and features

Since you cannot convert the filesystem on the fly, the best option is to make a backup and copy it back to your disk with the new filesystem

Problem

But now I'm in a difficult situation:

  • Computer A uses Elive 3.0, on which the version of BTRFS is too old to consider using it (btrfs was in heavy development with tons of bugs, but now seems like to be pretty usable in newer kernels)
  • Computer B uses Elive 3.7.x which is 64 bit and much updated kernel, it supports well btrfs but it doesn't supports well reiserfs, it can crash the system even if looks like to read it more or less correctly (computer feels bottlenecked), so i don't put my confidence on that

So I can't use both disks in the same computer, I'm forced to use 2 different computers. And this is not a problem at all using rsync :slight_smile:

Using rsync to transfer data

To replicate the data from dir1 to dir2 you should use this rsync call:

sudo rsync -avn --delete /home/thana/dir1/ /home/thana/dir2/

Notes:

  1. sudo is needed unless you are sure that all the files are owned by your user (which should be the case for transfer files that belongs to your user / home files)
  2. always use the "n" option (which is for "dry-run") before to run the real command, just to make sure that you are not going to make a disaster
  3. always use absolute paths, if you repeat the same command in the future using the history of your terminal you should be located in the same directory, its just a security measure
  4. always (very important) end the name of the directories with a slash, never run it like /home/thana/dir1 , you need to slash the end, otherwise your entire /home/thana can be overwritten / deleted
  5. rsync includes many options and features, read the man to know all the options

This is the best and optimal way to transfer files from a directory to another, ensuring consistency, management of links, etc.. in other words, to replicate data from a place to another

But what if the data is from another computer (called zeus)? you just need to call in one of these possible ways:

sudo rsync -avn --delete zeus:/home/thana/dir1/ /home/thana/dir2/
sudo rsync -avn --delete root@zeus:/home/thana/dir1/ /home/thana/dir2/
sudo rsync -avn --delete zeus.local:/home/thana/dir1/ /home/thana/dir2/
sudo rsync -avn --delete root@ipaddress:/home/thana/dir1/ /home/thana/dir2/

Any of them are valid, note that the "zeus.local" one requires that feature provided in the Elive installer, which basically allows you to ping / access to any remote computer in your network by using its name.local value

Verify data

If the disks includes only this data you can run "df" to compare the same sizes, otherwise use "du -hs" to the directories

If you want to make sure-sure that the data has been transfered correctly, run again the same rsync command, if you add the option "c" it will checksum every data to make sure that their contents are the same (this is as slow as a full new transfer)

Reformat disk and Mount it

Now, I wanted to switch to btrfs with transparent compression, I need to do all of this from computer B since is the one that supports it correctly, you can do it from gparted

Unplug / plug the disk. Open your filemanager and mount your new partition

Now run the command "mount" to ensure that all the mount options are the wanted ones (since you did not defined them!), and there's is a problem: our mounted btrfs is not mounted with compression, in fact, you cannot set this parameter when formatting the partition (so bad!), and to use compression you need to always mount the filesystem with this option included (auto if is set in your fstab), but you can remount the partition to add this option:

mount -o remount -o compress=zstd /mnt/something/

Another option is to run this command to recompress the data:

sudo btrfs defragment -r -v -zstd /mnt/something/

Now you are ready to copy back the data to your new mounted disk, and start using it when finishes :slight_smile:

Note: by not being able to "always mount by default" the partition with compression, all the new data will be not compressed, you should remount it before to use the disk or to run the defrag command later (much slower)

:furrydance:


Conclusion steps:

  • transfer data of your disk to a backuped place, using rsync
  • verify transfered data
  • reformat your original disk
  • mount it with the correct options
  • transfer data back to your disk

what about clonezilla-live-2.3.2-22-i686-pae.iso via network

(don't beat me, just an idea ...)

Never tried it :slight_smile: , they probably uses rsync (because is the best tool) in the backend, looking at their website looks like to be very dedicated for that purpose, with rare filesystems too

Btw i just found hilarious the first entry in their "limitations" section :rofl2:

The destination partition must be equal or larger than the source one.

Why hilarious?
Actually I saw a disc cloning station that is able to write a clone to a smaller disc
(of course the source partition hast to fit into the target disc)
I have often been at this point, wishing I would have that cloning station.
Spended hours with the dam 'dd' options,
but never succeed.

I guess
it will become very tricky for you anyway
because you want to change the partition and the partitiontable...
If I would be in your shoes,
I would probably try with creating an image from the source and then writng the DATA with a dd option (?) or so - to the clean target with the desired format options...
But for being honest:
Am happy not be in your shoes this time

Clonezilla uses "dd" extensively which has a downside I lately discovered: It also copies the backups of the partition table referring to the UUID of the original partition.
See: Existing GPT partition table seems to get corrupted by installer - #13 by triantares

So in case of restoring a messed (cloned) partition it really starts to get horrific.

.... that's why it's get called cloning. LOL

Because it sounds like saying "you cannot transfer the contents of a bottle of water into a glass of water"

dd doesn't look / uses files, only treats raw disk contents, it simply creates a "same" image in the target (this includes the filesystem), so nothing will be changed, to change the filesystem you should simply transfer the files from one disk (with filesystem A) to the other disk (with filesystem B)

not hard, it's already done :slight_smile: , the biggest issue (if we can call it issue) is that you need to ask yourself "im going to proceed with this, in this way" many times and verify again and again that you have a backup, because any error will cost the lose of your valuable data

yes, it's a pure snapshot, an identical clone

in the same way, if you are backuping a messed system, you will restore a messed system, just the same way it was before lol

so I don't consider it much useful for most of the cases, you only really want to transfer data from one place to another, or even using rdiff-backup which is like rsync but including incremental backups (versions of your data)

Well, for files and data I always was happy with
"Aptik"
but it is more relevant for the user-space, imo.