Skip to content

Comment on Elixir and Machine Learning in 2024 so far: MLIR, Arrow, structured LLM, etc.parent

Comments

People have to realise these are mostly for hobby. It is really hard to get these working with other libraries.

I have been dealing with this type of feedback for almost 20 years at this point: "This library doesn't work for my needs, so the conclusion is that it is hobby/not useful/etc for everyone". In your experience, we are not there yet (and we may never be), but the generalization does not hold.

Take Explorer, it was the first library of the bunch to be used in production because it was created by Chris Grainger to write data pipelines at his company. And the blog post does include the link to a talk on production usage.

Other projects, like Nx, started as an experiment, but nowadays it is used in production quite a bit and we spend a lot of the time refining it based on feedback we receive.

No one is claiming those libraries are complete. It is still pre-1.0. But we claim to be evolving and improving based on feedback. So if anyone wants to try something different than Python, and you can afford to engage, give feedback to maintainers, and even contribute, then you will hopefully have a good time and be closer to running it in production too.

Thank you Jose, for your 2 decades of relentless work. I have not disparaged any maintainers or their work. I get it from your comments that these projects are in early days of adoption and development.

From my perspective, just because Chris Grainger does data pipelines in production does not prove the project is for everyone. It means it worked out for him.

May be seeing more updates, papers, interviews, podcasts about their usage in production will change things.

For now, my assessment is everything is wonderful till one hits a limitation of these libraries. If it works out, its wonderful, if not its like neither here nor there.

That's precisely my point. Because Chris runs it in production, it doesn't mean the project is production-ready for everyone. And the fact you ran into roadblocks, doesn't mean they are "mostly for hobby" or are "half-baked". Your initial comment lacked this nuance and rushed into generalizations, which I did my best to clarify.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.