It depends a lot on how the application is set up.

So, my ideal async task thingie would be to start an async worker on the fly when there are tasks; the worker will be killed when the task has finished (or stay alive some time to pick any new tasks, whatever). So no plumbing of running a worker indenepdently.

If you are running a single process in a single thread, then you have one async event loop scheduling all async tasks. In that context, background jobs can run and stay running as long as the event loop is running, and you won't have any problems unless you accidentally call some blocking routine in the background task (then you'll block up the whole event loop because it's all still single-threaded).

You could even have multiple subthreads with one event loop in each, or multiple subprocesses, each with one or more subthreads, each with its own event loop. There is nothing about the design of the Python language that prevents you from doing this. The GIL is annoying but not a dealbreaker.

Things get trickier when you are dealing with a pool of workers being managed by some external process. I can't speak for other languages, but this is the industry standard setup in Python web development. You could of course disregard the fact that anything is different about your setup and go about the DIY process of spawning threads or async background tasks, building machinery to manage their liftetimes, etc. But then you're circumventing not only your web framework but also the master server process that manages the worker pool, and things could get really weird and messy.

Exactly how weird it gets, and how much DIY framework hacking you need to do, depends on the implementation details of your particular framework and the "master server" system that you are using.

Thus we ended up with standalone task queueing/running tools like Celery and Dramatiq (and now I guess Chard), which let you avoid fighting your framework and writing your own background task background task runner, at the expense of running a separate daemon and pool of workers. From the perspective of a developer, this is a great tradeoff because it lets me write web framework code that's focused on serving web stuff, and it lets me write background task code that doesn't have web framework details cooked into it. You could even go a step further and use something like Google Cloud Tasks or Pub/Sub if you want to avoid managing and monitoring another thing.

But I think it's important to be clear that these systems exist not because of anything particularly weird or bad about Python itself, but because of how Python web frameworks generally are expected to work.