SSH: What happens behind the scenes

Every time you type ssh user@host, you trigger a cryptographic ceremony that involves at least 6 distinct protocol phases, 3 layers of encryption negotiation and a minimum of 8 TCP round trips i.e., before a single byte of your shell session is transmitted. Let's tear it apart.

What SSH actually is?

Not just secure telnet. SSH is three protocols stacked on top of each other, all defined in RFC 4251–4254

Layer	RFC	Purpose
SSH Transport Layer	RFC 4253	Encryption, key exchange, integrity
SSH Authentication Layer	RFC 4252	User identity verification
SSH Connection Layer	RFC 4254	Channels, multiplexing, port forwarding

All of this rides on TCP port 22 (one persistent TCP connection)

The binary packet format (RFC 4253)

Every SSH message after key exchange looks like this

┌──────────────────────────────
│  uint32   packet_length  (4 bytes)            │
│  byte     padding_length (1 byte)             │
│  byte[]   payload        (variable)           │
│  byte[]   random_padding (padding_length)     │
│  byte[]   MAC            (variable, optional) │
└──────────────────────────────

• packet_length = length of everything AFTER the length field itself

• Padding is added to make packet_length + padding_length + 1 a multiple of cipher block size (or 8)

• MAC (Message Authentication Code) covers: sequence number + packet content — not encrypted, computed over plaintext then appended to ciphertext

NOTE: SSH is NOT TLS. It has its own key exchange, its own record format, its own MAC scheme.

THE FULL CONNECTION LIFECYCLE

Phase 0: TCP Handshake

Before SSH does anything, TCP happens:

Client → SYN        → Server
Client ← SYN-ACK   ← Server
Client → ACK        → Server

3 packets. Connection established on port 22.

Observable

tcpdump -i eth0 'tcp port 22' -nn

Phase 1: Version Exchange (RFC 4253)

First thing both sides do — send a plaintext identification string

Server → "SSH-2.0-OpenSSH_9.3\r\n"
Client → "SSH-2.0-OpenSSH_9.3\r\n"

• This is cleartext — fully visible in Wireshark

• Format: SSH-protoversion-softwareversion [comments]

• Client and server both record each other's version — used later in key exchange hash

• If versions are incompatible, connection drops immediately

Observable

# Capture the banner
nc -v 192.168.1.100 22
# or
ssh -vvv user@host 2>&1 | head -20

Phase 2: Algorithm Negotiation — KEXINIT (RFC 4253)

Both sides send a SSH_MSG_KEXINIT packet simultaneously (no waiting). This is the "menu"

byte         SSH_MSG_KEXINIT (20)
byte[16]     cookie (random bytes)
name-list    kex_algorithms
name-list    server_host_key_algorithms
name-list    encryption_algorithms_client_to_server
name-list    encryption_algorithms_server_to_client
name-list    mac_algorithms_client_to_server
name-list    mac_algorithms_server_to_client
name-list    compression_algorithms_client_to_server
name-list    compression_algorithms_server_to_client
...

Negotiation rule: client's preference list wins — first algorithm in the client's list that the server also supports is chosen.

Typical modern negotiation result:

What	Chosen Algorithm
Key Exchange	curve25519-sha256
Host Key	ssh-ed25519
Encryption	[email protected] or aes256-gcm
MAC	(implicit with AEAD cipher)
Compression	none (usually)

Observable

ssh -vvv user@host 2>&1 | grep -E 'kex|cipher|mac|compress'
# Shows:
# kex: algorithm: curve25519-sha256
# kex: host key algorithm: ssh-ed25519
# kex: server->client cipher: chacha20-poly1305...

Phase 3: Key Exchange — The Cryptographic Heart (RFC 4253)

This is the most important phase. Using Diffie-Hellman or ECDH to establish a shared secret without transmitting it.

With curve25519-sha256 (most modern):

Step 1 — Client generates ephemeral keypair:

client_private_key = random scalar (32 bytes)
client_public_key  = scalar × G  (Curve25519 base point)

Step 2 — Client sends SSH_MSG_KEX_ECDH_INIT:

byte      SSH_MSG_KEX_ECDH_INIT (30)
string    Q_C  ← client's ephemeral public key

Step 3 — Server computes shared secret + signs everything:

Server has:

• Its own ephemeral keypair (server_priv, server_pub)

• Client's public key Q_C

Server computes:

K (shared_secret) = server_priv × Q_C
                  = client_priv × Q_S  (ECDH magic — same result both sides)

Server builds exchange hash H:

H = SHA-256(
  V_C || V_S ||         ← version strings from Phase 1
  I_C || I_S ||         ← KEXINIT payloads from Phase 2
  K_S  ||               ← server's HOST key (public, long-term)
  Q_C  ||               ← client's ephemeral pubkey
  Q_S  ||               ← server's ephemeral pubkey
  K                     ← shared secret
)

Server sends SSH_MSG_KEX_ECDH_REPLY:

string    K_S     ← server host public key
string    Q_S     ← server ephemeral public key
string    sig_H   ← signature of H using K_S (host key)

Step 4 — Client verifies:

1. Compute K = client_priv × Q_S  (same shared secret!)
2. Recompute H
3. Verify sig_H using K_S
4. Check K_S against known_hosts (TOFU or CA)

Critical insight: The shared secret K is never transmitted. Both sides compute it independently. Even if someone recorded all packets, they cannot compute K without either private key. This is Forward Secrecy.

Step 5 — Key Derivation (RFC 4253)

From K and H, six keys are derived using the hash:

IV  client→server  = HASH(K || H || "A" || session_id)
IV  server→client  = HASH(K || H || "B" || session_id)
Key client→server  = HASH(K || H || "C" || session_id)
Key server→client  = HASH(K || H || "D" || session_id)
MAC client→server  = HASH(K || H || "E" || session_id)
MAC server→client  = HASH(K || H || "F" || session_id)

From this point forward: all packets are encrypted.

Step 6 — SSH_MSG_NEWKEYS: Both sides send this to say "switching to new keys now." One packet each.

Phase 4: SSH Authentication Layer (RFC 4252)

Now inside encrypted tunnel. Client must prove identity.

Step 1 — Service request:

SSH_MSG_SERVICE_REQUEST → "ssh-userauth"
SSH_MSG_SERVICE_ACCEPT  ← server confirms

Step 2 — Auth methods (tried in order):

Method A: publickey (most common)

1. Client sends probe:
   SSH_MSG_USERAUTH_REQUEST {
     username, "ssh-connection", "publickey",
     FALSE,  ← just checking if key is acceptable
     "ssh-ed25519", public_key_blob
   }

2. Server responds SSH_MSG_USERAUTH_PK_OK if key is in authorized_keys

3. Client sends actual auth:
   SSH_MSG_USERAUTH_REQUEST {
     ..same..,
     TRUE,   ← now actually signing
     signature of (session_id || request_data) using private key
   }

4. Server verifies signature → SSH_MSG_USERAUTH_SUCCESS

Where is authorized_keys checked?

~/.ssh/authorized_keys  (default)
# or configured via AuthorizedKeysFile in sshd_config
# or AuthorizedKeysCommand for LDAP/Vault lookups

Method B: password

SSH_MSG_USERAUTH_REQUEST {
  username, "ssh-connection", "password",
  FALSE,
  plaintext_password  ← encrypted by transport layer!
}

Method C: keyboard-interactive (used for MFA/PAM)

Multi-round challenge-response. Used by Google Authenticator PAM, Duo, etc.

Observable

ssh -vvv user@host 2>&1 | grep -E 'auth|Authentications'
# debug1: Authentications that can continue: publickey,password
# debug1: Next authentication method: publickey
# debug1: Offering public key: /home/user/.ssh/id_ed25519

Phase 5: SSH Connection Layer — Channel Multiplexing (RFC 4254)

Authentication passed. Now the connection protocol opens logical channels over the single TCP connection.

Opening a channel

Client → SSH_MSG_CHANNEL_OPEN {
  channel_type: "session"
  sender_channel: 0
  initial_window_size: 2097152  (2MB)
  maximum_packet_size: 32768
}

Server → SSH_MSG_CHANNEL_OPEN_CONFIRMATION {
  recipient_channel: 0
  sender_channel: 0
  initial_window_size: ...
  maximum_packet_size: ...
}

Requesting a PTY (pseudo-terminal):

SSH_MSG_CHANNEL_REQUEST {
  channel: 0
  request_type: "pty-req"
  want_reply: TRUE
  TERM: "xterm-256color"
  width_chars, height_chars, width_px, height_px
  terminal_modes: (speed, echo, etc.)
}

Requesting a shell:

SSH_MSG_CHANNEL_REQUEST {
  channel: 0
  request_type: "shell"
  want_reply: TRUE
}

Data flow:

Client → SSH_MSG_CHANNEL_DATA { channel: 0, data: "ls -la\n" }
Server → SSH_MSG_CHANNEL_DATA { channel: 0, data: "total 48\n..." }

Flow control — window adjust:

When receiver's buffer fills:
SSH_MSG_CHANNEL_WINDOW_ADJUST { channel: 0, bytes_to_add: 2097152 }

Key insight: Multiple channels can run simultaneously — shell + port forwards + X11 — all multiplexed over one TCP connection with independent flow control windows

Phase 6: Session Teardown

User types exit → shell sends EOF
SSH_MSG_CHANNEL_EOF    (no more data from this side)
SSH_MSG_CHANNEL_CLOSE  (I'm done with this channel)
SSH_MSG_DISCONNECT {
  reason_code: SSH_DISCONNECT_BY_APPLICATION
  description: "logout"
}
TCP FIN/ACK

OBSERVABILITY DEEP DIVE

Tool 1: ssh -vvv — Built-in debug

ssh -vvv -o 'LogLevel=DEBUG3' user@host 2>&1 | less

Shows every protocol message, algorithm chosen, key used.

Tool 2: Wireshark / tshark — Packet-level

# Capture SSH handshake (cleartext phase visible)
tshark -i eth0 -f 'tcp port 22' -w /tmp/ssh.pcap

# Analyze
tshark -r /tmp/ssh.pcap -Y 'ssh' -T fields \
  -e frame.number -e ssh.message_code -e ssh.kex.algorithms

In Wireshark:

• Filter: ssh

• You'll see: Protocol: SSHv2, message types like Key Exchange Init, Elliptic Curve Diffie-Hellman Key Exchange Init/Reply, New Keys

• After New Keys: everything shows as Encrypted packet

Tool 3: strace — System call level

# Trace the ssh client
strace -e trace=network,read,write -f ssh user@host 2>&1 | head -100

# On server side, trace sshd for a specific connection
strace -p $(pgrep -f "sshd: user") -e trace=read,write

Tool 4: eBPF / bpftrace — Zero overhead production observability

# Trace all connect() calls to port 22
bpftrace -e '
  tracepoint:syscalls:sys_enter_connect /
    ((struct sockaddr_in *)args->uservaddr)->sin_port == 5632
  / {
    printf("SSH connect: pid=%d comm=%s\n", pid, comm);
  }'

# Watch SSH data writes (channel data)
bpftrace -e '
  kprobe:tcp_sendmsg /comm == "ssh"/ {
    printf("ssh write: pid=%d bytes=%d\n", pid, arg2);
  }'

Tool 5: /proc and kernel crypto — See what's happening inside

# See active SSH connections
ss -tnp | grep :22

# Check which crypto is loaded
cat /proc/crypto | grep -A5 'curve25519\|chacha20'

# OpenSSL timing of key exchange (benchmark)
openssl speed curve25519 ed25519

Tool 6: sshd debug mode — Server-side visibility

# Run sshd in debug mode on alternate port (non-disruptive)
/usr/sbin/sshd -d -p 2222

# Full verbose
/usr/sbin/sshd -ddd -p 2222

THE FULL PICTURE

Complete timeline of one SSH connection

t=0ms    TCP SYN
t=1ms    TCP SYN-ACK
t=2ms    TCP ACK ──────────────────── TCP established

t=3ms    Server → SSH banner (cleartext)
t=4ms    Client → SSH banner (cleartext)

t=5ms    Client → SSH_MSG_KEXINIT
t=5ms    Server → SSH_MSG_KEXINIT (simultaneous)

t=6ms    Client → SSH_MSG_KEX_ECDH_INIT  (client ephemeral pubkey)
t=8ms    Server → SSH_MSG_KEX_ECDH_REPLY (host key + sig + server pubkey)

t=8ms    [Both sides derive 6 symmetric keys]

t=9ms    Client → SSH_MSG_NEWKEYS
t=9ms    Server → SSH_MSG_NEWKEYS
         ──────────────────────── EVERYTHING ENCRYPTED FROM HERE

t=10ms   Client → SSH_MSG_SERVICE_REQUEST ("ssh-userauth")
t=11ms   Server → SSH_MSG_SERVICE_ACCEPT

t=12ms   Client → SSH_MSG_USERAUTH_REQUEST (publickey probe)
t=13ms   Server → SSH_MSG_USERAUTH_PK_OK
t=14ms   Client → SSH_MSG_USERAUTH_REQUEST (signed)
t=15ms   Server → SSH_MSG_USERAUTH_SUCCESS

t=16ms   Client → SSH_MSG_CHANNEL_OPEN ("session")
t=17ms   Server → SSH_MSG_CHANNEL_OPEN_CONFIRMATION
t=18ms   Client → SSH_MSG_CHANNEL_REQUEST ("pty-req")
t=19ms   Server → SSH_MSG_CHANNEL_SUCCESS
t=20ms   Client → SSH_MSG_CHANNEL_REQUEST ("shell")
t=21ms   Server → SSH_MSG_CHANNEL_SUCCESS

t=22ms   ──────────────────────── SHELL PROMPT APPEARS

~22ms, ~20+ TCP segments, 3 protocol layers, 1 shared secret never transmitted.