The current set of Secure Shell protocols is SSH2. It is a rewrite of the old, deprecated SSH1 protocol. It contains significant improvements in security, performance, and portability. The default is now SSH2. OpenSSH uses the SSH protocol, connecting over TCP. Normally, one SSH session per TCP connection is made, but multiple sessions can be multiplexed over a single TCP connection if planned that way.
The Secure Shell protocol is an open standard. As such, it is vendor-neutral and maintained by the Internet Engineering Task Force (IETF). The current protocol is described in RFC 4250 through RFC 4256 and standardized by the IETF secsh working group. The overall structure of SSH2 is described in RFC 4251, The Secure Shell (SSH) Protocol Architecture.
The SSH protocol is composed of three layers: the transport layer, the authentication layer, and the connection layer.
- SSH-CONNECT – The connection layer runs over the user authentication protocol. It multiplexes many different concurrent encrypted channels into logical channels over the authenticated connection. It allows for tunneling of login sessions and TCP-forwarding. It provides a flow control service for these channels. Additionally, various channel-specific options can be negotiated. This layer manages the SSH session, session multiplexing, X11 forwarding, TCP forwarding, shell, remote program execution, invoking SFTP subsystem.
- SSH-USERAUTH – The user authentication layer authenticates the client-side to the server. It uses the established connection and runs on top of the transport layer. It provides several mechanisms for user authentication. These include password authentication, public-key or host-based authentication mechanisms, challenge-response, pluggable authentication modules (PAM), Generic Security Services API (GSSAPI) and even dongles.
- SSH-TRANS – The transport layer provides server authentication, confidentiality and data integrity over TCP. It does this through algorithm negotiation and a key exchange. The key exchange includes server authentication and results in a cryptographically secured connection: it provides integrity, confidentiality and optional compression. 
Among the differences between the current protocol, SSH2, and the deprecated protocol, SSH1, are that SSH2 uses host keys for authentication. Whereas SSH1 used both server and host keys to authenticate. There's not much which can be added about the protocols which is not already covered with more detail and authority in RFC 4251 .
SSH File Transfer Protocol (SFTP)
The SSH File Transfer Protocol (SFTP) is a binary protocol to provide secure file transfer, access and management.
SFTP on the server side was added by Markus Friedl in time for the 2.3.0 release of OpenSSH in November 2000. Damien Miller added support for SFTP to the client side in time for 2.5.0. Since then, many have added to both the client and the server.
SFTP is not FTPS
For basic file transfer, nothing more is needed than an account on the machine with the OpenSSH server. SFTP support is built into the OpenSSH server package. The SFTP protocol, in contrast to old FTP, has been designed from the ground up to be as secure as possible for both login and data transfer.
Unless the use-case calls for publicly available, read-only, downloads, don't worry about trying to fiddle with FTP. It is the protocol FTP itself that is inherently insecure. It's great for read-only, public data transfer. The programs vsftpd and proftpd, for example, are secure insofar as the server software itself goes, although the protocol is insecure. In other words the program itself is more or less fine and if you need to provide read-only, publicly available downloads then FTP maybe the right tool. Otherwise forget about FTP. Nearly always when users ask for "FTP" they don't mean specifically the old file transfer protocol from 1971 as described in RFC 114, but a generic means of file transfer and there are many ways to solve that problem. This is especially true since the next part of their request is usually how to make it secure. The name "FTP" is frequently mis-used generically to mean any file transfer utility, much the same way as the term "Coke" is used in some of the Southern States to mean any carbonated soft drink, not just Coca-Cola. Consider SFTP or, for larger groups, even SSHFS, Samba or AFS. While old FTP succeeded very well in achieving its main goal to promote use of networked computers by allowing users at any host on the network to use the file system of any cooperating host, it cannot be made secure. There's nothing to be done about that, so it is time to get over it.
Again, it is the protocol itself, FTP, which is the problem. With FTP, the data, passwords and user name are all sent back and forth unencrypted. Anyone on the client's subnet, the server's subnet or any subnet in between can 'sniff' the passwords and data when FTP is used. With extra effort it is possible to wrap FTP inside SSL or TLS, thus creating FTPS. However, tunneling FTP over SSL/TLS is complex to do and far from an optimum solution.
Unfortunately name confusion combined with the large number of posts and discussions created by complex, nit-picky tasks like wrapping FTP in SSL to provide FTPS, the wrong way still turns up commonly in web searches regarding file transfer. In contrast, easy, relatively painless solutions vanish because it is rarely necessary to post how to do those. Also, an easy solution can be summed up in very few lines and maybe a single answer. Thus, there is still a lot of talk online about 'securing' FTP and very little mention of using SFTP. It's a vicious cycle that this book hopes to help break: Difficult task means lots of discussion and noise, lots of discussion and noise means strong web presence, strong web presence means high Google ranking.
SFTP tools are very common, but might be taken for granted and thus overlooked. SFTP tools are just as easy to use and more functional than old FTP clients. In fact a lot of improvements have been made in usability. There is no shortage of common, GUI-based SFTP clients to transfer files: Filezilla, Konqueror, Dolphin, Nautilus, Cyberduck, Fugu, and Fetch top the list but there are many more. Most are Free Software. These SFTP clients are very easy to use. For example, in Konqueror, just type in the URL to the sftp server, where the server name or address is xx.yy.zz.aa.
If it is desirable to start with a specific directory, then that too can be specified.
One special client worth knowing about is sshfs. With sshfs as an SFTP client the other machine is accessible as an open folder on your machine's local file system. In that way any program you normally have to work with files, such as LibreOffice, Inkscape or Gimp can access the remote machine via that folder.
Background of FTP
FTP is from the 1970s. It's a well proven workhorse, but from an era when if you were on the net you were supposed to be there and if there was trouble it could usually be cleared up with a short phone call or an e-mail or two. It sends the login name, password and all of the data unencrypted for anyone to intercept. FTP clients can connect to the FTP server in either passive or active modes. Both active and passive modes for FTP use two ports, one for control and one for data. In FTP Active mode, after the client makes a connection to the FTP server it then allows an incoming connection to be initiated from the server to for data transfer. In FTP Passive mode, after the client makes a connection to the FTP server, the server then responds with information about a second port for data transfer and the client initiates the second connection. Thus FTP is most relevant now as Anonymous FTP, which is still excellent for read-only downloads without login. To transfer read-only data, FTP is still one way to go, as would be using the web (HTTP or HTTPS) or a P2P protocol like bittorrent are other options for offering read-only downloads. Using tcpdump to show FTP activity
An illustration of how the old protocol, FTP, is insecure can be had from the utility tcpdump. It can show what is going over the network during an Anonymous FTP session, or for that matter any FTP session. Look at the manual page for tcpdump for an explanation of the individual arguments, but the overall result of the usage below is that it displays the first FTP or FTP-Data packets going from the client to the server and vice versa.
The output below hows an excerpt from the output of tcpdump which captured packets between an FTP client and the FTP server, one line per packet.
$ sudo tcpdump -q -s 0 -c 10 -A -i eth0 \ "tcp and ( port ftp or port ftp-data) " tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes … :18:36.010820 IP desk.55227 > server.ftp: tcp 16 E..D..@.@.....1.[.X.....G.r. ........l..... ."......USER anonymous :18:36.073192 IP server.ftp > desk.55227: tcp 0 E..4jX@.7.3.[.X...1..... ...G.r#........... .....".. :18:36.074019 IP server.ftp > desk.55227: tcp 34 E..VjY@.7.3.[.X...1..... ...G.r#....Y...... ....."..331 Please specify the password. :18:36.074042 IP desk.55227 > server.ftp: tcp 0 E..4..@.@..+..1.[.X.....G.r# ..)........... ."...... :18:42.098941 IP desk.55227 > server.ftp: tcp 23 E..K..@.@.....1.[.X.....G.r# ..)....gv..... .".w....PASS email@example.com :18:42.162692 IP server.ftp > desk.55227: tcp 23 E..KjZ@.7.3.[.X...1..... ..)G.r:........... .....".w230 Login successful. … :18:43.431827 IP server.ftp > desk.55227: tcp 14 E..Bj\@.7.3.[.X...1..... ..SG.rF.....j..... ....."..221 Goodbye. …
As can be seen in lines 3 and 7, data, such as text, from the server is visible. In lines 1 and 5, text entered by the user is visible, in this case including the user name and password used to log in. Fortunately the session is Anonymous FTP, which is read-only and used for downloading. Anonymous FTP is a rather efficient way to publish material for download. For Anonymous FTP, the user name is always "anonymous" and the password the user’s e-mail address and the server's data always read-only.
If you have the package openssh-server already installed, no further configuration of the server is needed to start using SFTP for file transfers. Though comparatively speaking, FTPS is significantly more secure than FTP. If you want remote remote login access, then both FTP and FTPS should be avoided. A very large reason to avoid both is to save work.
FTPS is FTP tunneled over SSL or TLS. A goal of FTP was to encourage the use of remote computers. It, along with the web, has succeeded. A goal of FTPS was to secure logins and transfers, and was a necessary step in securing file transfers with the legacy protocol.
Since SFTP is so much easier to deploy and most systems now include both graphical and text-based SFTP clients, FTPS can really be considered deprecated for most occasions. Some good background material can be found in the Request for Comments (RFCs) for FTP and FTPS. There, SFTP and even HTTPS are better matches and largely supercede FTPS. See the section on Client Applications for an idea of the SFTP clients available.
Privilege separation is when a process is divided into sub-processes, each of which have just enough access to just the right services to do their part of the job. An underlying principle is that of least privilege, which is where each process has exactly enough privileges to accomplish a task, neither more nor less. The goal of privilege separation is to compartmentalize any corruption and prevent a corrupt process from accessing the entire system. Privilege separation is applied in OpenSSH by using several levels of access, some higher some lower, to run sshd(8) and its subsystems and components. The server ➊ starts out with a privileged process ➋ which then creates an unprivileged process ➌ to work with the network traffic. Once the user has authenticated, another unprivileged process is created ➍ with the privileges of that authenticated user. See the "Sequence Diagram for OpenSSH Privilege Separation". So as seen in the diagram, a total of four processes get run to create an SSH session. One, the server, remains and listens for new connections and spawn child processes. Here it is seen waiting for a connection.
$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' USER PID PPID S STARTED COMMAND root 1473 1 S 05:44:12 /usr/sbin/sshd
It is this privileged process that listens for the initial connection from clients.
$ netstat -ntlp | awk '/sshd/ || NR<=2' Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1473/sshd tcp6 0 0 :::22 :::* LISTEN 1473/sshd
After the initial connection while waiting for password authentication from user 'fred', a privileged monitor process supervises an unprivileged process by user 'sshd' which handles contact with the remote user's client.
$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' USER PID PPID S STARTED COMMAND root 1473 1 S 05:44:12 /usr/sbin/sshd root 9481 1473 S 14:40:37 sshd: fred [priv] sshd 9482 9481 S 14:40:37 sshd: fred [net]
Then after authentication is completed, an established session for user 'fred', a new privieged monitor process supervises a process running as user 'fred'.
$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' USER PID PPID S STARTED COMMAND root 1473 1 S 05:44:12 /usr/sbin/sshd root 9481 1473 S 14:40:37 sshd: fred [priv] fred 9579 9481 S 14:42:02 sshd: fred@pts/30
Privilege separation has been the default in OpenSSH since version 3.3 Since version 5.9, privilege separation further applies mandatory restrictions on the system calls the privilege separated child can perform. The intent is to prevent a compromised privilege separated child from being used to attack other hosts either by opening sockets and proxying or by probing local kernel attack surface.  Since version 6.1, this sandboxing is the default.
- "OpenSSH Manual Pages". OpenSSH. http://www.openssh.com/manual.html. Retrieved 2011-02-17.
- "RFC 4251: The Secure Shell (SSH) Protocol Architecture". 2006-01. http://tools.ietf.org/html/rfc4251. Retrieved 2013-10-31.
- "Why You Need To Stop Using FTP". JDPFu.com. 2011-07-10. http://blog.jdpfu.com/2011/07/10/why-you-need-to-stop-using-ftp. Retrieved 2012-01-09.
- Manolis Tzanidakis (2011-09-09). "Stop Using FTP! How to Transfer Files Securely". Wazi. http://olex.openlogic.com/wazi/2011/stop-using-ftp-how-to-transfer-files-securely/. Retrieved 2012-01-09.
- Nils Provos. "Privilege Separated OpenSSH". University of Michigan. http://www.citi.umich.edu/u/provos/ssh/privsep.html. Retrieved 2011-02-17.
- "OpenSSH 5.9 Release Notes". OpenSSH. 2011-09-06. http://www.openssh.com/txt/release-5.9. Retrieved 2012-11-17.
Overview • Why Encryption • Protocols • Implementations • Clients • Client Configuration • Server • Patterns • Utilities • Third Party • Logging • Development