Question: About backups , Incremental or Full ?

[jalsco]

I am using Dietpi-Backup with mostly the defaults... so it is using rsynch.

As such I expected the first backup to be Full and the subsequent ones to be Incremental - and so I was expecting the later ones would be much smaller and much faster, than the "Full".

I have taken 4, all within last 24 hours (mostly for testing purposes).. so not alot has changed on the system in that time. The last one (4th) was "snapshot" before something was reinstalled this morning. In settings I have it set to "keep 3" backups. so I guess the oldest one was overwritten (?) ,thus keeping only the 3 newest ones (which includes the "snapshot" )

[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup$ ls -al
total 184
drwxr-xr-x  5 root   root   4096 Dec  2 01:25 .
drwxr-xr-x  5 dietpi root   4096 Dec  1 02:52 ..
-rw-r--r--  1 root   root    375 Dec  2 01:25 .dietpi-backup_stats
drwxr-xr-x 11 root   root   4096 Dec  2 01:25 data
drwxr-xr-x 11 root   root   4096 Dec  1 22:10 data_2
drwxr-xr-x 11 root   root   4096 Dec  1 01:26 data_3
-rw-r--r--  1 root   root 162812 Dec  2 01:25 dietpi-backup.log
[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup$ du -sh *
4.1G    data
4.1G    data_2
4.0G    data_3
160K    dietpi-backup.log

I do not understand why all 3 backups are about the same size about 4.1GB. Is there an obvious explanation for that ???
I expected an "Incremental" would be less than 1gb (given it is created only a couple hours later) - but they are not. They are same size. Am I am using the right command to get a sense of the size ( see above) ?? , I am using du -sh

To me - it indicates backups being made are always Full and never Incremental. Is that how it is supposed to be operating ? And, if that is default behaviour, is it possible (with settings and options) to add some "incremental" backups into the mix - for time and diskspace savings ?

[jalsco]

[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup$ du -shl *
4.7G    data
4.7G    data_2
4.6G    data_3
160K    dietpi-backup.log

How are those differences explained ? 4.7GB to 4.1GB Does it mean 0.6GB is not actually backed up ?

[MichaIng]

It's 4.7G to 4.6G, while it is likely a much smaller difference than 0.1G, as it crossed a rounding up vs rounding down border. Check via du -sml * to see size in MiB, giving a more accurate picture.

[jalsco]

@MichaIng I mean the difference when using the -l option on the du command. As foloows, for the data folder:

[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup$ cd data
[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup/data$ du -sh
4.1G    .
[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup/data$ du -shl
4.7G    .
[Fri Dec 02] root@batopi: /mnt/pibax/dietpi-backup/data$

[MichaIng]

Ah sorry, I've not compared with your first post. -l counts multiple hardlinks (representation of data in the filesystem tree) to the same inode (reference to actual data blocks on disk) multiple times, hence does not represent thr real disk usage. Since we run rsync with -H switch, hardlinks are preserved as such and not each transferred as individual file.

You should see the same difference in:

du -sm / 2> /dev/null
du -sml / 2> /dev/null

Isn't it?

[Joulinar]

we always do full backups if you use a new empty target folder. Therefore size should be same.

[jalsco]

ok - but your "if" statement is confusing.
The 2nd and 3rd and 4th backups to that target folder ( /mnt/pibax) would not have been to a "new empty folder".
That means those cases do not match the "if" statement you wrote - yet they also are "Full" backup.

[MichaIng]

Not sure where you got the info from that those are incremental backups, probably we need to rephrase something somewhere to avoid this misunderstanding? Every iteration is a full backup, so you can remove all other dirs and recover from any single iteration the whole system at this stage.

The only case where something is done incrementally is when all iterations up to the amount you configured do already exist. In this case, the oldest backup isn't removed, but it is used as basis for the newest. So rsync does not need to write all files, but only the difference between the oldest backup and the current system state. The result is still a full backup, only the rsync writes to generate it are reduced.

[jalsco]

I think got it now - thanks for taking the time to explain !

So in that last case - "when all iterations up to the amount you configured do already exist." ... when dietpi-backup is going to remove "an oldest backup" - it uses the unchanged files in it to avoid Writing that file again - to save time ( but not space). That was the basis of my misunderstanding - I had misunderstood that case before, thinking it was describing an "Incremental" . But it still makes a "Full" only much much faster.

I got it now. Thank you 🙏🏻😊