Skip to content

Ask HN: Best tools for real-time API proxying/transformation?

2 pointsC-Loftusdiscuss
On HN

I am a data engineer and do a lot of open source work writing API proxies over scientific APIs to present them as standardized formats [0] with better discoverability. I don't have access to the underlying db so I can't just create a new API directly.

When writing data pipelines for generating data assets, I have really benefited from using a scheduler framework like Dagster. It gives my projects a quick way to get a nice UI and common helpers (cron schedules, asset graphs, partitioned data, concurrency) with relatively little code.

However, it doesn't seem like an analogous framework exists for real time proxying. (i.e. something with config as code, a nice UI for tracing requests, concurrency/caching helpers for dealing with messy transformations). Am I missing anything?

[0] https://ogcapi.ogc.org/

Comments

No comments yet.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.