Difficult people can potentially be far worse than having no one in their position. They may cast a cloud over every team meeting with a bad attitude. They can be a troll under the bridge, standing in the way of movement with excuses and a million reasons why it can't be done. Everyone on the team may end up walking on eggshells around that person, and end up doing everything they can to avoid interacting with them. It makes everyone else's job twice as hard.