It's not about robots.txt but yes, the owners of 538 can just send a cease and desist letter to get them all immediately removed. Many sites that don't want to preserve history have done this already.