I just reread the first chapter of Penrose's _The Emperor's New Mind_ so this is based off of it.

Let's simplify the problem to just conversation, for the sake of simplicity. Suppose we have a program that can pass the Turing test... But the way it is constructing it's responses is only a more capable version of the emacs psychiatrist. Does it have consciousness?

I posted a link to ENM in another post. Maybe look at the first chapter. It hits the nail on the head, IMHO.

EDIT: Or even just read the first few pages... It seems to introduce it decently, if your short on time.