so as a customer, I need to keep track of all public keys of my customers?

Yes. You probably want a reasonable limit. A customer might have ten phones, but if they have a hundred phones they already know they're a weirdo and you won't be the first one to tell them that. You will probably also want to have a way for customers to distinguish one from another, so that they can say "Oh I gave my old iPhone to my brother" and delete the unneeded credentials.

The good news is that as their name might suggest, they're not secrets, so now you don't need to worry about anybody stealing them. If Keyri did even a halfway competent job of this they're useless to anybody except you.