Ah ha, thanks! So turns out gray cards use "middle gray" which has both a physical definition as well as a specific RGB value:

https://en.wikipedia.org/wiki/Middle_gray

Fascinating. I never knew there was an "standard" mapping between real-world colors and RGB.

Now if only online clothing store photos would map to that, so that the olive green coat I order doesn't turn out to be closer to medium brown or vice-versa... :P