Your answer is too definitive. Applications and databases can indeed suffer power loss while losing nothing of value. An ACID database used by a correctly designed application will lose nothing on power loss; state "in memory" is uncommitted in such a system effectively doesn't exist. It never happened and, furthermore, no one cares.
The term of art is "crash consistent," and any ACID database must preserve all committed state across events such as power loss. Such a database is correctly backed up when copying a simultaneous point in time snapshot across all involved volumes.
Not all databases are truly ACID. Lots of software relies on uncommitted database state. But we're talking about a solved problem here; if you require ACID behavior the means to achieve exactly that are available. Any exception to that statement, including hardware misfeatures or lack of two phase commit across databases, is equivalent to "incorrectly designed."
In a correctly designed system quiescing the database isn't necessary, but might still be used as a precaution or a performance optimization.
Your figure is too high. Commercial relational databases, many open source databases and widely used file systems are all sufficient. Oracle, SQL Server, DB/2, PostgreSQL and MySql/MariaDB (with InnoDB) are all ACID systems and recover from crash consistent backups fine. Ext4, XFS, NTFS, etc. are all sufficient file systems, unusually in default form. Reliable storage devices are widely available.
Many applications designed around these platforms use COMMIT properly. When they're don't quiescing the database probably won't help with backups anyhow; all you're doing is deferring writes and if the application is not ACID then you can still lose. The database itself, however, will recover and since we're tossing probabilities around, that's 99% of the battle.
I'd bet quite a large amount of money that if you could see every process running on every server on earth: less than 1% of the total would be database processes
meanwhile most of the rest seek randomly in files, buffer and write data without calling ever fsync, or append without a pointer and appropriate write barriers (I'm looking at you journald)
I don't know what your basis of measurement is, so I'll continue to ignore your ratios. Tolerance of crash consistent state isn't exclusive to databases. Filesystems themselves are often designed to tolerate, for example, sudden power loss without corruption. Applications that manage large collections of files can also achieve this; maildir is an example of such a scheme. All of these systems are using related principles of consistency, and all of them can be correctly backed up by copying volume snapshots.
are you ignoring the main point too, or simply missing it entirely (repeatedly)?
the fact there exist some applications (databases, applications using maildir) that correctly use write barriers doesn't mean all applications are always consistent on disk
Comments
Your answer is too definitive. Applications and databases can indeed suffer power loss while losing nothing of value. An ACID database used by a correctly designed application will lose nothing on power loss; state "in memory" is uncommitted in such a system effectively doesn't exist. It never happened and, furthermore, no one cares.
The term of art is "crash consistent," and any ACID database must preserve all committed state across events such as power loss. Such a database is correctly backed up when copying a simultaneous point in time snapshot across all involved volumes.
Not all databases are truly ACID. Lots of software relies on uncommitted database state. But we're talking about a solved problem here; if you require ACID behavior the means to achieve exactly that are available. Any exception to that statement, including hardware misfeatures or lack of two phase commit across databases, is equivalent to "incorrectly designed."
In a correctly designed system quiescing the database isn't necessary, but might still be used as a precaution or a performance optimization.
this is fantastic and all for carefully written databases (none of which have ever had any bugs)
unfortuntely most applications aren't like that and assume the filesystem is both atomic and durable
Your figure is too high. Commercial relational databases, many open source databases and widely used file systems are all sufficient. Oracle, SQL Server, DB/2, PostgreSQL and MySql/MariaDB (with InnoDB) are all ACID systems and recover from crash consistent backups fine. Ext4, XFS, NTFS, etc. are all sufficient file systems, unusually in default form. Reliable storage devices are widely available.
Many applications designed around these platforms use COMMIT properly. When they're don't quiescing the database probably won't help with backups anyhow; all you're doing is deferring writes and if the application is not ACID then you can still lose. The database itself, however, will recover and since we're tossing probabilities around, that's 99% of the battle.
I'd bet quite a large amount of money that if you could see every process running on every server on earth: less than 1% of the total would be database processes
meanwhile most of the rest seek randomly in files, buffer and write data without calling ever fsync, or append without a pointer and appropriate write barriers (I'm looking at you journald)
I don't know what your basis of measurement is, so I'll continue to ignore your ratios. Tolerance of crash consistent state isn't exclusive to databases. Filesystems themselves are often designed to tolerate, for example, sudden power loss without corruption. Applications that manage large collections of files can also achieve this; maildir is an example of such a scheme. All of these systems are using related principles of consistency, and all of them can be correctly backed up by copying volume snapshots.
are you ignoring the main point too, or simply missing it entirely (repeatedly)?
the fact there exist some applications (databases, applications using maildir) that correctly use write barriers doesn't mean all applications are always consistent on disk