It's not actually thinking, though.

That may well be, I genuinely don't know. However, consider the following thought experiment:

Ask a random stranger on the street[*] to "generate a random password" and observe their behaviour. Are they whipping out their Python interpreter or just giving you a string of characters?

Now ask yourself whether this random stranger is capable of thought.

I think it's pretty clear that the former is a poor test for the latter.

[*] someplace other than Silicon Valley :)