Even working on the copy text would decrease accidental unsubscribes.

Something as simple as "This email was sent to somebody@example.org, if you are this person click here to unsubscribe somebody@example.org" then display the email address again prominently on the unsub page