I will never in my life understand why people go through all the trouble of booting up a headless browser, only then to slurp the HTML back across the WebDriver interface so they can _re-parse_ it using some rando library. Not only is that inefficient, it almost guarantees questions on r/webscraping or SO about "but I see some element in the browser, why is $random_library not parsing it the same as the browser?!11"
Comments
I will never in my life understand why people go through all the trouble of booting up a headless browser, only then to slurp the HTML back across the WebDriver interface so they can _re-parse_ it using some rando library. Not only is that inefficient, it almost guarantees questions on r/webscraping or SO about "but I see some element in the browser, why is $random_library not parsing it the same as the browser?!11"