I expect that self-driving will become a safety mechanism before it becomes a full driver replacement. It will probably keep track of cars around you as you're driving manually, and if you get close to a collision, it will take over and move you away.

As the data improves, the road information becomes better curated, and so on, I expect that it will become a driving aid, as you describe.

And I expect that within a couple of decades, fully driverless cars (with nobody behind the wheel) will become commonplace.