The problem is that, anecdotally, no one seems to use Lists

That anecdote is not really worth much. Especially because those lists are probably very important for those who do use them.

Most people won't care about filtering their social relationships, until they do care. At that point, you want them to have the option.