Brownbag Talk: SSH ninja sauce
Notes from a brownbag talk I gave at Dimagi
This talk delves into power-usage of
Local port forwarding
ssh user@remoteserver -L localport:targetserver:targetport
Any network traffic hitting
localport on the local machine (the machine you’re ssh’ing from) will be forwarded over the encrypted
ssh channel to
remoteserver (the machine you’re ssh’ing to), and then onward to
The traffic coming into the local machine, and going from
targetserver is unencrypted.
Only the segment between the local machine and
remoteserver is encrypted.
Usually the tunneled traffic originates from your local machine and
targetserver is the same as
remoteserver, so in practice the entire communication would be secure.
targetserver is resolved relative to
targetserver could be an internal IP on
remoteserver’s LAN that you could not access directly from the local machine.
This is also what lets you specify “
targetserver, since it’s “
localhost” as resolved with respect to
Local forwards are useful for the following scenarios:
access a service on a remote machine that is only available locally (either because only local connections are allowed, or the port is blocked via firewall)
ssh remoteserver -L 4000:localhost:5984
access couchdb running on a remote server; we can now access it using the address
access a firewalled/NAT’ted machine that you cannot access directly
ssh remoteserver -L 8080:192.168.1.15:80
192.168.1.15is only visible on
remoteserver’s LAN. We access its webserver via
localhost:8080; traffic is forwarded through
encrypting a channel that would otherwise be unencrypted
ssh remoteserver -L 5900:localhost:5900
vncis an insecure protocol; passwords are sent in the clear and desktop contents are visible to anyone snooping the traffic. Instead of vnc’ing to
remoteserver, we securely
masking the origin of traffic
ssh stoogeserver -L 8080:dupedserver:80
Forwarding a connection in this manner is not always transparent. For example, HTTP includes a
Hostparameter that specifies the hostname the browser wants to connect to. In this case that will be
dupedserverwill see that.
By default, only local traffic can connect to
To enable external hosts to use the port forward through your machine, invoke as
ssh remoteserver -L
localport:targetserver:targetport (note asterisk).
Remote port forwarding
ssh user@remoteserver -R remoteport:targetserver:targetport
Any network traffic hitting
remoteserver will go to your local machine (via encrypted channel), and then onward to
targetserver from the local machine.
targetserver is now resolved relative to the local machine, but in the remote forwarding scenario
targetserver is nearly always “
making a public server (visible to the internet at large) that forwards to your local machine
ssh publicserver -R 8053:localhost:8053
You’re testing an android app over GPRS against your dev server. The app can only hit public IPs. Now it can access your dev server via
There is a catch with remote forwards.
For the above scenario (which is really the only useful scenario) to work, external traffic must be allowed to connect to
This is only allowed if the
GatewayPorts setting in
This is not the case by default on most installs.
Therefore, you must have root on
remoteserver to enable this.
The setting is in the config of the
ssh server, so no client settings you do can override it.
As for the “
*:” to enable external connections, it seems usually implicit for remote forwards.
But… sometimes not; I don’t understand the circumstances when it is necessary vs. not – when in doubt, add it; it can’t hurt anything.
It will not override
If you can’t change
sshd_congig, there is a workaround.
We can daisy-chain a local forward to the remote forward, so all traffic hitting
remoteport originates from
ssh remoteserver -R dummyport:localhost:targetport
- then, on
ssh localhost -L *:remoteport:localhost:dummyport(note the ‘*’)
Needless to say, this is ridiculous.
Running a command remotely (in lieu of a shell)
ssh user@server command
ssh user@server ls -l ~
~ typed after a newline is the
ssh escape character.
It allows you to perform out-of-band actions with the
~?– print list of available commands
~.– terminate the session (useful if network dropped or session is hung)
~[ctrl-z]– suspend session (instead of suspending the currently running program inside the session)
~C– enter command line to add additional port forwards on-the-fly (type
~~– type a literal
ssh sessions can piggyback on an originating session’s connection.
No authentication is needed for the piggybacking sessions (and as a side-benefit the connection will establish very quickly).
To start the first session (the “master session”):
ssh -M -S /tmp/sshsocket user@server
For piggybacking sessions:
ssh -S /tmp/sshsocket user@server
This is particularly useful when the piggybacking sessions run a command on the remote server instead of a shell.
I use connection multiplexing to control my media server. I’ll open the media player on the server and have the UI running locally (see next section). This is the master connection. Then I set keybindings to run control commands (volume up/down, prev/next track, etc.) as piggybacking sessions. Each keypress/command runs in a new, short-lived piggybacking session – now you can see why quick connection time is so important.
/tmp/sshsocket can be any file, unique to the master session.
Any user/process with access to this file can piggyback on the session.
You can also use shorthand like
ssh will auto-expand to help maintain uniqueness among master sessions.
Run GUI programs on a remote server!
ssh -X user@server callofduty
Any configuration option available in
ssh_config can be invoked on a per-session basis.
This is very useful when running
ssh from scripts:
ssh -o BatchMode=yes– fail immediately if any interactive prompt is displayed (e.g., password prompt), since these would hang your script forever
ssh -o ExitOnForwardFailure=yes– abort if the desired port forwards could not be set up
ssh -o ServerAliveInterval=60– ‘ping’ the server every 60 seconds and terminate the session if some consecutive number of pings go unanswered (usually 3)
The combination of these three options can ensure robust
ssh tunneling from non-interactive scripts.
In fact we have written just such a script.
Quickly set up key-based auth
copies your public keys to the remote server
Quick and dirty, but not robust.
There are better options.
Usually the easiest bet for quick and dirty, with the added benefit of a GUI. Your file manager probably has support built-in, otherwise, you need a dedicated client.
sshfs user@server:path localpath
pathon the remote server as a filesystem under
localpath. You have to create
localpathyourself, sadly. Useful for both command-line and GUI browsing, without need for any special client support. Not robust against dropped connections. Unmount with
fusermount -u localpath
rsyncis fantastic tool, and can use
sshas a transport layer.
rsync -ravz -e ssh user@server:path localpath
user@server:pathis interpreted according to
To robustly transfer a 10GB database file from Africa while you sleep:
while [ true ] ; do rsync -ravz --progress --partial -e ssh user@server:path localpath ; sleep 5 ; done
- the loop will resume after dropped connections
--partialallows resuming the transfer
--progressdisplays a progress bar
sleep 5avoids flooding the server.
Make sure you’re set up to log into
user@serverusing password-less authentication, otherwise resume attempts will hang with a password prompt!
You may want to use a one-off keypair for the transfer. Specify an arbitrary private key to use (don’t forget to set up the corresponding public key on the remote server) by replacing
-e "ssh -i /path/to/privatekey.key".
There’s a long delay when logging in…
This is usually either:
sshserver is trying to reverse DNS lookup the client IP, and it has to wait for the lookup to time out
sshserver is trying to integrate with an authentication library that is not configured properly, and timing out
Fix by disabling the guilty library in
I’m sure you all know these:
C-a c– new window
C-a n– change windows
C-a k– kill window
C-a d– detach from session
screen -ls– display sessions
screen -r– resume session
How the f#$% do I scroll
Enter ‘copy’ mode:
In copy mode, you can navigate around the history using arrows and page up/down.
The size of the buffer is pretty limited by default. To make it bigger, do any of:
screen -h ####(affects all windows in this session)
C-a :scrollback ####(affects current window only)
- set a bigger default in your
In ‘copy’ mode, you can also search with
C-a s (forward) or
Copy text using mark and set points.
space to start/end at the current char;
y to start/end at the current line.
C-a ]– paste into current window
C-a >– dump copy buffer to file
C-a h– write current window (visible portion only) to file
C-a :hardcopy -h /must/specify/a/file– write the entire scrollback history to
C-a H– start/stop logging of current terminal to file
Monitoring a window for activity
C-a M– alert if this window has activity
C-a _– alert if this window has no activity for a while
C-a S– split current region horizontally
C-a |– split current region vertically
C-a [tab]– move to next region
C-a X– close this region
C-a Q– close all regions except this one
screen -x [session name]
allows multiple terminals to connect to the same session (must be same user)
You can actually set up multi-user screen sharing, but it’s kind of a bitch, and not that useful, for all of:
screenexecutable must be setuid root, along with other permissions changes
- no one can sudo; they must be running as their logged-in user
- there’s a shitload of setup commands you have to run
- you can’t really guarantee everyone is seeing the same thing; split-screens, window changes, etc., are all local to each connection
- a user closing a window or the session closes it for everybody
Communicate with network sockets using stdin/stdout:
nc host port
connect to a server
nc -l port
listen and accept a single connection from a client (does not create a server; only good for one connection)
Once the connection is open, communicate with it by typing/piping.
comments powered by Disqus