Go modules are built around git

Not really. Modules are specced based on zip files and metadata in text files. There's just support for extracting that data from git repos transparently.

Here's a slightly out of date write-up: https://research.swtch.com/vgo-module