OpenSSH/Cookbook/Proxies and Jump Hosts
A proxy is an intermediary that forwards requests from clients to other servers. Performance improvement, load balancing, security or access control are some reasons they are used. Some proxies include caching to save bandwidth and increase speed by avoiding going out to the net to keep retrieving the same, unchanged documents.
- 1 SOCKS Proxy
- 2 Jump Hosts -- Passing through a gateway or two
- 3 Passing through a gateway with an ad hoc VPN
- 4 References
It's possible to connect via an intermediate machine using a SOCKS proxy. SOCKS4 and SOCKS5 proxies are currently supported by OpenSSH. SOCKS5 allows transparent traversal, by an application, of a firewall or other barrier and can use strong authentication with help of GSS-API.Dynamic application-level port forwarding allows the outgoing port to be allocated on the fly thus creating a proxy at the tcp session level.
Here the web browser can connect to the SOCKS proxy on port 3555 on the local host:
$ ssh -D 3555 server.example.org
Using ssh as a socks5 proxy, or in any other capacity where forwarding is used, you can specify multiple ports in one action:
$ ssh -D 80 -D 8080 -f -C -q -N email@example.com
For example in Firefox, you'll also want the DNS requests to go via your proxy, so changing about:config needs network.proxy.socks_remote_dns set to true
It'll be similar for other programs that support SOCKS proxies.
You can tunnel samba over ssh, too.
Tunneling SSH Over Tor with Netcat
Instead of using ssh as a SOCKS proxy, it is possible to tunnel the SSH protocol itself over a SOCKS proxy such as Tor. Tor is anonymity software and a corresponding network that uses relay hosts to conceal a user's location and network activity. Its architecture is intended to prevent surveillance and traffic analysis. Tor can be used in cases where it is important to conceal the point of origin of the SSH client.
On the end point that the client sees, Tor is a regular SOCKS 5 proxy and can be used like any other SOCKS 5 proxy. So this is tunneling SSH over a SOCKS proxy. For example, if Tor is installed locally and listening on a port, then SSH can run over Tor  using netcat:
$ ssh -o ProxyCommand="nc -X 5 -x localhost:9150 %h %p" server.example.org
If the user name on the remote system is different from that on the local system, it is possible to pass along a different user name.
$ ssh -o User=fred -o ProxyCommand="nc -X 5 -x localhost:9150 %h %p" server.example.org
When attempting a connection like this, it is very important that it does not leak information. In particular, the DNS lookup should occur over Tor and not be done by the client itself. Make sure that if VerifyHostKeyDNS is used that it be set to 'no'. The default is 'no' but check to be sure. It can be passed as a run-time argument to remove any doubt or uncertainty.
$ ssh -o "VerifyHostKeyDNS=no" -o ProxyCommand="nc -X 5 -x localhost:9150 %h %p" server.example.org
Using the netcat-openbsd nc package, this seems not to leak any DNS information. Other netcat packages might or might not be the same. It's also not clear if there are other ways in which this method might leak information. YMMV.
Jump Hosts -- Passing through a gateway or two
It is possible to connect to another host via an intermediary or two so that the client can act as if the connection were direct. ProxyCommand works and the utility Netcat fits here, too.
A useful trick is "ssh -tt" which forces tty allocation, so instead of the above you can do the following, connecting to server2 via firewall as the jump host:
$ ssh -tt firewall.example.com ssh -tt server2.example.org
This opens an ssh terminal to the remote machine. You can also pass commands. For example, to reattach to a remote screen session using screen you can do the following:
$ ssh -tt firewall.example.com ssh -tt server2.example.org screen -dR
The chain can be arbitrarily long and is not limited to just two hosts.
Port Forwarding via an Intermediate Host
Tunneling, also called port forwarding, is when a port on one machine mapped to a connection to a port on another machine. In that way remote services can be accessed as if they were local. Or in the case of reverse port forwarding, vice verse. Forwarding can be done directly from one machine to another or via a machine in the middle.
Below we are setting up a tunnel from the localhost to machine2, which is behind a firewall, machine1. The tunnel will be via machine1 which is publicly accessible and also has access to machine2.
$ ssh -L 2222:machine2.example.org:22 machine1.example.org
Next connecting to the tunnel will actually connect to the second host, machine2.
$ ssh -p 2222 remoteuser@localhost
It is possible to use all the options in this way, such as -X for X11 forwarding. Here is an example of running rsync between the two hosts using machine1 as an intermediary with the above setup.
$ rsync -av -e "ssh -p 2222" /path/to/some/dir/ localhost:/path/to/some/dir/
SOCKS proxy via an Intermediate Host
If you want to open a SOCKS proxy via an intermediate host, it is possible:
$ ssh -L 8001:localhost:8002 firstname.lastname@example.org -t ssh -D 8002 email@example.com
The client will see a SOCKS proxy on port 8001 on the local host, which is actually a connection to machine1 and traffic will ultimately enter and leave the net through machine2. Port 8001 on the local host connects to port 8002 on machine1 which is a SOCKS proxy to machine2. The port numbers can be chosen to be whatever you needed, but forwarding privileged ports still requires root privileges.
ProxyCommand with Netcat
Another way is to use the ProxyCommand configuration directive and netcat. The utility netcat is for reading and writing network connections directly. It can be used to pass connections onward to a second machine. In this case, login is the final destination reached via the intermediary jumphost.
$ ssh -o 'ProxyCommand ssh %h nc login.example.edu 22' \ -o 'HostKeyAlias=login.example.edu' \ jumphost.example.org
Keys and different login names can also be used. Using ProxyCommand, ssh will first connect to jumphost and then from there to login.example.edu. The HostKeyAlias directive is needed to look up the right key for login.example.edu, without it the key for jumphost will be tried and that will, of course, fail unless both have the same keys. The account user2 exists on jumphost.
$ ssh -o 'ProxyCommand ssh -i key-rsa -l user2 %h nc login.example.edu 22' \ -o 'HostKeyAlias=login.example.edu' \ jumphost.example.org
It's also possible to make this arrangement more permanent and reduce typing by editing ssh_config Here a connection is made to host2 via host1:
Host host2.example.org ProxyCommand ssh firstname.lastname@example.org nc %h %p
Here a connection is made to server2 via server1 using the shortcut name 'jump'.
Host jump ProxyCommand ssh %h nc server2.example.org 22 HostKeyAlias server2.example.org Hostname server1.example.org User fred
It can be made more general:
Host gateway.example.org ProxyCommand none Host *.example.org my-private-host ProxyCommand ssh email@example.com nc %h %p
The same can be done with sftp by passing parameters on to ssh. Here is a simple example with sftp where machine1 is the jump host to connect to machine2. The user name is the same for both hosts.
$ sftp -o 'ProxyCommand=ssh %h nc machine2.example.edu 22' \ -o 'HostKeyAlias=machine2.example.edu' \ firstname.lastname@example.org
Here is a more complex example using a key for server1 but regular password-based login for the SFTP server.
$ sftp -o 'ProxyCommand ssh -i /Volumes/Home/fred/.ssh/server1_rsa \ -l user2 server1.example.edu nc sftp.example.edu 22' \ -o 'HostKeyAlias=sftp.example.edu' sftp.example.edu
If the user accounts names are different on the two machines, that works, too. Here, 'user2' is the account on the second machine which is the final target. The user 'fred' is the account on the intermediary or jump host.
$ ssh -l user2 \ -o 'ProxyCommand ssh -l fred %h nc machine2.example.org 22' \ -o 'HostKeyAlias machine2.example.org' \ machine1.example.org
Passing through a gateway using netcat mode
As of OpenSSH 5.4, a 'netcat mode' can connect stdio on the client to a single port forwarded on the server. This can also be used to connect using ssh(1), but it needs the ProxyCommand option either as a run time parameter or as part of ~/.ssh/config. However, it no longer needs netcat to be installed on the intermediary machine(s). Here is an example of using it in a run time parameter.
$ ssh -o ProxyCommand="ssh -W %h:%p jumphost.example.org" server.example.org
In that example, authentication will happen twice, first on the jump host and then on the final host where it will bring up a shell.
The syntax is the same if the gateway is identified in the configuration file. ssh(1) expands the full name of the gateway and the desintation from the configuration file. The following allows the destination host to be reached by entering ssh server in the terminal.
Host server Hostname server.example.org ProxyCommand ssh jumphost.example.org -W %h:%p
The same can be done for SFTP. Here the destination SFTP server can be reached by entering sftp sftpserver and the configuration file takes care of the rest. If there is a mix up with the final host key, then it is necessary to add in HostKeyAlias to explicitly name which key will be used to identify the destination system.
Host sftpserver HostName sftpserver.example.org HostKeyAlias sftpserver.example.org ProxyCommand ssh jumphost.example.org -W %h:%p
It is possible to add the key for the gateway to the ssh-agent which you have running or else specify it in the configuration file. The option User refers to the user name on the destination. If the user is the same on both the destination and the originating machine, then it does not need to be used. If the user name is different on the gateway, then the -l option can be used in the ProxyCommand option. Here, the user fred on the local machine, logs into the gateway as fred2 and into the destination server as fred3.
Host server HostName server.example.org User fred3 ProxyCommand ssh -l fred2 -i /home/fred/.ssh/rsa_key jumphost.example.org -W %h:%p
If both the gateway and destination are using keys, then the option IdentityFile in the config is used to point to the gateway's private key and the option IdentityFile specified on the commandline points at the destination's private key.
Host jump HostName server.example.org IdentityFile /home/fred/.ssh/rsa_key_2 ProxyCommand ssh -i /home/fred/.ssh/rsa_key jumphost.example.org -W %h:%p
The old way prior to OpenSSH 5.4 used netcat, nc(1).
Host server Hostname server.example.org ProxyCommand ssh jumphost.example.org nc %h %p
But that should not be used anymore and the netcat mode, provided by -W, should be used instead. The new way does not require netcat at all on any of the machines.
Recursively chaining gateways
It is possible to make the configuration more abstract and allow passing through an arbitrary number of gateways. This particular configuration only works if the user name is the same across all hosts involved. There are limitations resulting from using the slash as a separator, as there would be with other symbols. However, it allows use of dirname(1) and hostname(1) to process the host names.
Host */* ProxyCommand ssh $(dirname %h) -W $(basename %h):%p
In this way hosts are separated with a slash (/) and can be arbitrary in number.
$ ssh host1/host2/host3/host4
If keys are to be used, then agent forwarding can be specified in the command given in the ProxyCommand option using -A and first loading the keys into the agent.
The following configuration uses sed(1) to allow different port numbers and user names using the plus sign (+) as the delimiter for hosts, a colon (:) for ports, and an equal sign (=) for user names. The basic structure is
ssh $() -W $():$() and where %h is substituted for the target host name.
Host *+* ProxyCommand
ssh -v $(echo %h | sed -e'
s/+[^+]*$//; s/\([^+=]*\)=\([^+]*\)$/\2 -l \1/; s/^\([^+:]*\):\([0-9]*\)+/-p \2 \1+/'
) -W $(echo %h | sed -e'
):$(echo %h | sed -e'
s/^.*+//; /:/!s/^.*/22/; s/^.*://' ;)
The port can be left off for the default of 22 or delimited with a colon (:) for non-standard values.
$ ssh host1+host2:2022+host3:2224
As-is, the colons confound sftp(1), so the above configuration will only work with it using standard ports. If sftp(1) is needed on non-standard ports then another delimiter, such as an underscore (_), can be configured.
Any user name except the final one can be specified for a given host using the designated delimiter, in the above it is an equal sign (=). The destination host's user name is specified with -l and all others can be joined to their corresponding host name with the delimiter.
$ ssh -l user3 user1=host1+user2=host2+host3
If user names are specified, depending on the delimiter, ssh(1) can be unable to match the final host to an IP number and the key fingerprint in known_hosts. In such cases, it will ask for verification each time the connection is established, but this should not be a problem if the equal sign (=) is used.
Passing through a gateway with an ad hoc VPN
Two subnets can be connected over ssh by configuring the network routing on the end points to use the tunnel. The result is a VPN. A drawback is that root access is needed on both hosts, or at least sudo access to ifconfig and route. Note, there are very few instances where use of a VPN is legitimately called for, not because VPNs are illegal (quite they opposite, indeed data protection laws in many countries make them absolutely compulsory to protect content in transit) but simply because OpenSSH is usually flexible enough to complete most routine sysadmin and operational tasks using normal SSH methods as and when required. This SSH ad-hoc VPN method is therefore needed only very rarely.
Take this example with two networks. One network has the address range 10.0.50.1 through 10.0.50.254. The other has the address range 172.16.99.1 through 172.16.99.254. Each has a machine, 10.0.50.1 and 172.16.99.1 respectively, that will function as a gateway. Local machine numbering starts with 3 because 2 will be used for the tunnel interfaces on each LAN.
+----10.0.50.1 172.16.99.1----+ + 10.0.50.2 ===== 172.16.99.2 + | | 10.0.50.3-----+ +---172.16.99.3 | | 10.0.50.4-----+ +---172.16.99.4 | | 10.0.50.5-----+ +---172.16.99.5 | | 10.0.50.etc---+ +---172.16.99.etc | | 10.0.50.254---+ +---172.16.99.254
First a tun device is created on each machine, a virtual network device for point-to-point IP tunneling. Then the tun interfaces on these two gateways are then connected by an ssh tunnel. Each tun interface is assigned an IP address.
The tunnel connects machines 10.0.50.1 and 172.16.99.1 to each other, and each are already connected to their own local area network (LAN). Here is a VPN with the client as 10.0.50.0/24, remote as 172.16.99.0/24. First, set on the client:
$ ssh -f -w 0:1 192.0.2.15 true $ ifconfig tun0 10.1.1.1 10.1.1.2 netmask 255.255.255.252 $ route add 172.16.99.0/24 10.1.1.2
On the server:
$ ifconfig tun1 10.1.1.2 10.1.1.1 netmask 255.255.255.252 $ route add 10.0.50.0/24 10.1.1.1
- "SOCKS Protocol version 5". IETF. http://tools.ietf.org/html/rfc1928. Retrieved 2011-02-17.
- "SSH Over Tor". The Tor Project. 2012-08-28. https://trac.torproject.org/projects/tor/wiki/doc/TorifyHOWTO/ssh. Retrieved 2013-05-04.
- "OpenSSH 5.4 Release Notes". OpenSSH. 2010-04-07. http://www.openssh.com/txt/release-5.4. Retrieved 2013-04-19.
- Josh Hoblitt (2011-09-03). "Recursively chaining SSH ProxyCommand". [Read This Fine Material] from Joshua Hoblitt. https://joshua.hoblitt.com/rtfm/2011/09/recursively_chaining_ssh_proxycommand/. Retrieved 2014-02-14.