Correct! Remote members of the team get access to shared team keys, and the team's data, even though they don't have accounts on that server. Knowledge of the team key suffices to allow a remote user to authenticate and transfer (encrypted) data to and from the server.

There is very little server-to-server communication, which simplifies the design and software upgrades.