When designing our product it became quickly apparent that immutability isn't practical, we opted for MVCC ACID transactions, but the most difficult part of MVCC database is getting the purge right.
You can cheat purges with some clever optimization but at some point, you need to clean up old versions and when you do that, you are using precious I/O.
Getting purge/compaction right is hard. Update intensive scenarii are always problematic for databases.
I used some tricks to reduce the amount of data that must be cleaned up at any given point in time, but it was not possible to evade it completely. I still have to do concurrent compaction, and there's some really nasty corner cases I don't have any good solutions for, it's a hard problem.
Comments
Agreed.
When designing our product it became quickly apparent that immutability isn't practical, we opted for MVCC ACID transactions, but the most difficult part of MVCC database is getting the purge right.
You can cheat purges with some clever optimization but at some point, you need to clean up old versions and when you do that, you are using precious I/O.
Getting purge/compaction right is hard. Update intensive scenarii are always problematic for databases.
As a matter of interest, what product is that?
I used some tricks to reduce the amount of data that must be cleaned up at any given point in time, but it was not possible to evade it completely. I still have to do concurrent compaction, and there's some really nasty corner cases I don't have any good solutions for, it's a hard problem.