I discovered a critical bug in Apple’s implementation of Time Machine in OS X Mavericks 10.9.4 that causes the dissapearance of some data from backup snapshots. I highly suggest that you check the integrity of your backups to see if you are affected by this bug. If you are affected then I suggest that you raise your own bug report with Apple, and switch to a homegrown rsync solution until Apple fixes this problem.
This bug still exists in OS X Yosemite (10.10) to some extent! I opened yet another bug #22164368.
You may notice that some arbitrary files are missing from snapshots. These files are not excluded by any exclusion rules. Additionally, the files will appear in the very first snapshot that Time Machine creates, but are then omitted in subsequent snapshots. Time Machine gives no indication that anything is wrong which falsely leads the user to believe that all is well.
I wrote a small ruby script that will compare your current working data to what is stored on your Time Machine drive. You can grab the script from GitHub and check the integrity of your latest TM snapshot.
Alternatively, you can perform a differential using the diff tool with a command similar to the following.
diff -rq /Users/jon/ /Volumes/Time\ Machine\ 2/Backups.backupdb/Jon's\ iMac/Latest/Fusion/Users/jon/
Apple bug report state
I do not know what version of OS X this bug was introduced, but I caught it in 10.9.4. Prior to this discovery I have never encountered such data loss with Time Machine, as far as I’m aware of. I opened Apple bug report #17819383 for this which was then marked as a duplicate of #17224229 and is still open.
Here is a fantastic bash script on Github that replicates Time Machine’s behavior and it works reliably: https://github.com/eaut/rsync-time-backup
I discovered this bug when one of my sparse bundles became corrupted and I needed to restore it to a previous version. The sparse bundle, and several other important documents in the same folder were completely missing from from the most recent snapshots leaving me with a copy more than 30 days old [most likely from OS X 10.9.3 or older]. Fortunately, I have multiple backup systems in place so I recovered the file from an alternate source, but this was a rude awakening in a system that I have trusted for several years since its introduction.