This reminds me of the worry of the potential A/B tester: won't my users get confused if they see site A from one web browser/computer, but site B from another?

I think this is just a self-defeating worry, though. Most people won't know or care about your site or brand, or remember you from one session to another. If they don't like your page, they'll just click the back button. If they do get upset, well, the internet is effectively a pool of infinitely more people who can click your adwords.

So exercise caution when changing big features used by paying customers, but don't be afraid to experiment and show things to random visitors.