In general, you are right to complain, audio in the web is always a problem; the only debate is about just how much and in how many ways it is broken.

In practice, you probably want to take a look at Howler.js - it's pretty comprehensive library that tries its best to overcome the various limitations and problems.