
Once again today, #FreeBSD and #ZFS saved a setup. Suddenly, a colleague realized that a database was acting up - probably some massive operation had deleted something. The machine takes snapshots every 15 minutes and keeps them for a few hours, then one a day and keeps those for days. To make a long story short, the July 4th dump still had the correct data. To get there, we just had to clone all the snapshots (going back day by day) and test them.
Snapshots are one of the best inventions since sliced bread.