Skip to content

Comment on Ask HN: What tools do you use for data munging and merging?parent

Comments

There are examples of this online.

Can you link to any of them? I wasn't able to find anything that actually works when I needed to do this a while ago. (I needed to repeatedly hit a REST API until it returned no results.)

Absolutely.

https://community.alteryx.com/t5/Alteryx-Designer-Discussion...

Let me know if that doesn't cover your use case.

BTW, if you're hitting a REST API, you might want to try using the Python tool (it's an embedded Jupyter interface that will default to just running the pure python code when the workflow is executed without a user configuring it), just because I prefer using requests library over using the Alteryx download/curl tool myself.

Yep, I've seen the link. Never managed to get it to run "until the iteration output stream does not receive any more records" though, nor did I see that documented anywhere.

you might want to try using the Python tool

Yep, I just end up using Python outside of Alteryx altogether.

Yeah, it all comes down to your needs and the use case. If there is a lot of value in allowing non-coders to leverage your work, and more importantly, modify pieces of it, then a tool like Alteryx is worth it. However, if you can just write a Python script, and aren't constantly being hassled to update it for whatever data source or new need, then just go that route. Otherwise, if you don't empower the non-tech users, eventually you'll produce enough stuff that you'll end up spending an increasing percentage of your time maintaining and dealing with your own products. For me, that's been the biggest benefit. Nothing better than telling somebody to just switch the input on a workflow to a db now that the file isn't being sent directly, instead of having to do it for them.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.