Use rsync to compare large directory trees

I had to rewrite some large back-ups recently in order for restic to properly de-duplicate them. I checked that the re-written back-ups are identical to the original ones by comparing the entire directory trees of the mounted back-ups. I had some doubts on whether diff -qr or rsync -acni --delete would be the best approach and did some benchmarks and tests using /usr (total: 3.2GB) which has a large mix of files and permissions. I first tried a bind mount, but diff skips comparing anything entirely since it determines that both directories have the same device/inode. I used a copy of /usr instead. Timing results are: tool time rsync 17.41 busybox diff 31.19s GNU diff 22.94s A quirk of using diff is that is follows symlinks, which is problematic for symlinks which point out of the tree being compared. This fails for broken symlinks, where diff exits with an error. Busybox diff ignores all permissions errors, which doesn’t sound right at all. Finally, a key factor is that rsync can will…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town