[Loki123]

Good ways to make SSH skid-proof other than key auth are -

• Pluggable authentication modules
  
• Making SSH listen on a port other than 22
  
• Disabling remote root logins (keep in mind that this doesn't restrict you/other people from using 'su root')
  
• Google's two-factor SSH authentication
  

About that second item, it is perfectly pointless to run SSH on another port if you use SSH keys as the only authentication mechanism as there is no way in hell anyone will be able to break that SSH key. That kind of "security" is called security through obscurity which can sometimes open you up to real vulnerabilities and security risks.

Lets say you change the port to 2222. All ports above 1024 are privileged ports that means only root can use them. Change the port to 2222 and another user has the ability to mimic the SSH server. Not to mention, changing the SSH port does not provide increased security, it is a mere flawed attempt at hiding a service that shouldn't and doesn't need to be hidden.

About that Google's Two-Factor SSH authentication. It is a nice approach, but in the long run, is it really worth it to secure it with two factor authentication along with SSH keys with a decent enough passphrase ?
I'd probably prefer allowing only specific IP addresses to connect then if I'd really think I would require Two Factor Authentication, making sure my computers are secure enough for the SSH key (Whom should always have a decent passphrase) doesn't get stolen.