According to both Amazon and Google, it matters immensely. I can't find the link right now, but Amazon said they found a noticeable step-up of conversion loss in their sales funnel for each 200 milliseconds of 'slow' they added.

Meaning, faster is better.