There is certainly a tipping point between sets and bitsets. Bitsets are far more efficient for storage and for performing unions or intersections for millions of users. For example, for a typical 10% of user base active daily for mobile apps (http://www.avc.com/a_vc/2011/07/301010.html), bit sets will pay off both in storage and performance.

Obviously, each use case is different and for very sparse case, sets would be a better choice.