don't have anything in common with the rest of the employees outside of the project they're working on.

I fail to grasp this part. How is this negative? I often have nothing in common with the rest of the guys in my team but it doesn't affect my work in any way.