This is part of the mini-series OpenSSH for Devs.
SSH agent forwarding let’s you lock down remote hosts while making them easier to access and use in automated ways. One co-worker succinctly describes agent forwarding as “the shit”.
Securely connect to a remote host from a remote host without a password.
laptop:~$ ssh -A host1.example.com
Linux host1 2.6.35-25-server #44-Ubuntu SMP Fri Jan 21 19:09:14 UTC 2011 x86_64 GNU/Linux
host1:~$ scp host2.example.com:some.config .
some.config 100% 1612 1.6KB/s 00:00
host1:~$ logout
Connection to host1.example.com closed.
The SSH agent has become so integrated into our local systems many people don’t realize it’s being used. Devs use it daily to avoid having to retype their SSH key1 password every time they connect to a remote host. The typical workflow is:
The agent serves us by holding onto our private key and transparently authenticating to remote hosts when we connect instead of making us type a password.2
Simply put, agent forwarding allows you to access a remote machine from a remote machine.
Let’s look at the scenario above: connect to host1 and download a file from host2. Without agent forwarding, you’re lucky if you just get to type your password again. If host2 has password authentication disabled or your account has no password set, there’s two options. Option 1: download the file from host2 to your local machine and then upload it to host1. Option 2: upload your SSH private key to host1 and authenticate to host2 using your key password. Compare these to agent forwarding where you run scp and the file is downloaded without question.
If you’ve run into this problem more than a few times, learning about agent forwarding may feel like this:
The SSH agent provides a rare pairing of increased security and better user experience.
From a per-host perspective, you can disable password authentication on all your remote machines and rely on SSH keys for superior auth. Leaked passwords are no longer a vector for unauthorized access since you can’t login with them. Forget about generating random passwords for every user on every new server. If sudo access isn’t needed, don’t set a password at all. If sudo access is required you can get away with reusing passwords, keeping your devops team lean3.
From a network perspective, you ideally want your private servers only accessible via a bastion host or other intermediary. With agent forwarding, instead of this setup being a pain to get into, it’s a single command:
$ ssh -At public.example.com ssh private1.internal
Linux private1 2.6.35-25-server #44-Ubuntu SMP Fri Jan 21 19:09:14 UTC 2011 x86_64 GNU/Linux
private1:~$ logout
Connection to private1.internal closed.
Connection to public.example.com closed.
Agent forwarding can be turned on via the command-line by passing -A or via your SSH config by setting ForwardAgent yes.
I’d be negligent if I didn’t recommend setting this only for hosts you trust. While it’s not possible to steal a private key through an agent, it’s trivial for a malicious root user to login to remote hosts with your public key.
Is there another way you use SSH agent forwarding? You should post a comment or send me a message.
This article assumes you already use an SSH key to access remote hosts. If you don’t, send me a note. If I get enough questions about SSH keys, I’ll do a writeup on them. ↩
Some systems aren’t setup with an askpass program and the agent running in the background. In those cases, some devs will generate their SSH private key without a password to get the effect of not needing to type in their password for every SSH connection they make. Regardless of the security implications, that setup loses a beneficial feature of SSH: agent forwarding! ↩
Buzzwords aside, having to search for a password randomly generated 2 months ago before getting on with your task is sure way to wipe stored state and kill a task’s momentum. ↩
This is part of the mini-series OpenSSH for Devs.
An SSH config let’s you set options you use often (e.g., the user to login as or the port to connect to) globally or per-host. It can save a lot of typing and helps make SSH Just Work.
Instead of typing:
ssh -p734 teamaster@sencha.example.com
You can type:
ssh sencha
By having this in your ~/.ssh/config:
Host sencha
HostName sencha.example.com
Port 734
User teamaster
In your home is a .ssh directory. This is where your SSH keypair and known_hosts1 files are. This directory is not made of unicorns. Create a file named config and your SSH tools2 will use it’s settings.
If your laptop username is different than the one on your remote hosts, create it with:
User jon.postel
If you use a non-standard SSH port to avoid the bots, create it with:
Port 22022
Have different settings for different hosts? No problem. Just keep in mind the first match wins and put specific settings before generic ones:
# ancient box we never upgraded
Host host1.example.com
User oldusername
# still on port 22
Host *.example.com
Port 22022
ForwardAgent yes
# defaults for all hosts
Host *
User bofh
You’re probably familiar with the dance you do when connecting to a host for the first time:
$ ssh host1.example.com
The authenticity of host 'host1.example.com (192.0.2.101)' can't be established.
RSA key fingerprint is 39:9b:de:ad:9e:be:ef:95:ca:fe:1b:53:b0:00:00:b5.
Are you sure you want to continue connecting (yes/no)?
It looks impressive, but it’s worthless. If you’re worried about man-in-the-middle attacks, there are much better things to do. Start by disabling password authentication3 and require people to have an SSH key on the server. Expecting people to check these hashes means you’ve already failed.
To get rid of the dance add something like:
Host *.compute-1.amazonaws.com
StrictHostKeyChecking no
First time connections will give you a warning, but you’ll make it in.
You can create aliases4 by using Host to match a name and HostName to say where to connect.
Host web1
HostName ec2-192-0-2-42.compute-1.amazonaws.com
The advantages over modifying /etc/hosts are you don’t need to be root and SSH will use the same host key for web1 and ec2-192-0-2-42.compute-1.amazonaws.com. The disadvantage is that only SSH tools see this. For example, your browser has no idea web1 is an alias for that EC2 host. Because of this, I sometimes create both the SSH alias and hosts entry for the best of both worlds.
There are a lot of options, but these are the ones I’ve seen used most:
yes to turn on SSH agent forwarding.no to skip the “authenticity of host” dance.Do you have a favorite option not mentioned here? You should post a comment or send me a message.
The known_hosts file contains the keys for all the remote hosts you’ve connected to. The stored key is compared to the remote key when you connect to warn of a man-in-the-middle attack. ↩
ssh, scp, sftp, sshfs, well-written paramiko based Python tools, and probably more. ↩
In the server’s sshd_config set PasswordAuthentication no. Contact me if you are interested in a post on securing the SSH server. ↩
I made this term up. There may be a better one. ↩