I suggest hiding the points from submissions, comments and other people's summary page.

Seeing the points doesn't add value and it distorts the opinions and upvotes (people tend to like more what other people recommend).