Skip to content

Comment on It’s Okay to Store Data in Apache Kafka (2017)parent

Comments

If you write "read it to the latest version" code for every version, you need to write that for every past version - potentially a high burden. Writing "convert it to the next version" instead is much less work for humans, and probably usually the best call, but less efficient when you actually need to read that old data. There are hybrid approaches to consider, too.

Wherever you are on that spectrum, there is code. Exercising it may slow down your test suite. It might need to be touched when you update lint rules (at least to add annotations to shut off the new rules).

Code nearly always represents some amount of debt. Code to deal with history, more so.

We have a slight disagreement on what it means to accrue technical debt.

When a schema changes for an all, someone has to migrate it and the data, unless the design is that all incompatible data is dropped. Either that’s the codebase itself or the dev team is punting to the user. Punting to the user just shifts the burden around.

If the R&D team shipped a shitty / incomplete schema to get the software out the door that they need to change later, then yes, that is technical debt - something they’ll eventually need to repay.

If requirements evolve over time and thus the schema needs to, that is not necessarily technical debt in the usual sense, which usually implies a temporary technical compromise for the purposes of expediency / getting something out the door.

I suppose I agree there is a trade off here and a penalty - after many years the migrations get slow to apply etc, and you could say that checkpointing the schema every few versions and preventing upgrades from any prior point is a way of cleaning up the “migration debt”.

But people like to suggest that there’s some other way , ie. the OP saying “ If you get your domain model wrong at the start, it will haunt you forever.”.... I have never seen any system get the domain model perfectly right at the start!

That may be. I was mostly trying to address the claim that "[i]t's not technical debt [because] there is nothing to [...] maintain."

Financially speaking, it's more like an annuity than a credit card.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.