It is quite easy to cross-compile cgo program to most architectures - just use Zig for this:

  CC="zig cc -target aarch64-linux"