It's a gray area, but you usually have to show a pretty big difference that couldn't have been to due to anything but de facto discrimination of a kind that would've been illegal if it were official policy.

The main reason the idea was introduced was because, during the civil-rights era, a bunch of companies and even state/local governments abolished their official "no blacks" policies, but nonetheless unofficially still didn't hire blacks (or, if they had low-level black workers, had a de facto no-blacks-in-management policy). So the 1964 Civil Rights Act is intended to ban that sort of unwritten-rules discrimination. The background of Jim Crow laws was a motivator as well, since many didn't officially impose any restrictions on the basis of race, but used proxies that happened to correlate well with race; and the Civil Rights Act was intended to do away with all those roundabout methods, too.