You are right, i'd need both actually. I can't find any service which provides weekly/monthly data download for a flat subscription price, so that i can perform local fast lookups.
How granular do you need this data? Domains are ultimately not publicly known, especially when it comes to subdomains. Do you just need domains that get a certain amount of public traffic or are you trying to get as much domains as possible? In the latter case scraping this data would require the logs of public DNS servers to even know of the existence of many domains (although certificate transparency logs could get one half the way).
Comments
You are right, i'd need both actually. I can't find any service which provides weekly/monthly data download for a flat subscription price, so that i can perform local fast lookups.
How granular do you need this data? Domains are ultimately not publicly known, especially when it comes to subdomains. Do you just need domains that get a certain amount of public traffic or are you trying to get as much domains as possible? In the latter case scraping this data would require the logs of public DNS servers to even know of the existence of many domains (although certificate transparency logs could get one half the way).