Comment on Robots.txt for the NYT has a specific exclusion for an 1996 news articleparentComments−MattGaiser5yLike many people who get exceptions. They asked.
Comments
Like many people who get exceptions. They asked.