I'd also recommend still requiring a sudo password on the other end and sending auth events to a auditing server.

You should be auditing everything already, and adding in sudo now adds another attack vector. You also now have two different accounts that can be manipulated to compromise a system.