Skip to content

Ask HN: How does your team handle data backfilling?

1 pointk1nsdiscuss
On HN

My team of SWE's is at an inflection point with how we handle database backfills. Think new fields on existing records across environments being populated by some conditional logic. I am curious how other development teams handle their own backfill operations and if there is a "standard" or rule-of-thumb.

In past teams, repos comprised of common functionality executed by Jupyter Notebooks, or even loose scripts, would be used for us to review incoming backfills before the engineer would shell in and execute the script. This is better than no reviews of backfill operations but I would like to think there is a more streamlined way.

Comments

No comments yet.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.