Try changing the SMTP host. Kick off a build, it uses the old host (even if that throws errors) to try and send the mail. A server restart is required before it picks up the change.