·
Practical approaches that scale from 5 servers to 500.
When you manage a handful of servers, remembering hostnames works. When you hit 20, 50, or more, you need a system. The right approach depends on your scale, your team, and whether you work alone or share access.
SSH connections accumulate. A staging server here, a production database there, a jump host, a few developer VMs. Before long you are juggling hostnames, ports, usernames, and key files. The knowledge lives in your head, in scattered notes, or in a config file that has grown unwieldy.
The real cost is not just finding the right connection - it is the context around it. Which server runs which service? What is the database URL on staging vs production? What was that one-liner you ran last month to check disk space on the app servers?
The simplest approach. Define host aliases in your SSH config file:
Host staging-app
HostName 10.0.1.42
User deploy
IdentityFile ~/.ssh/staging_key
Port 22
Host prod-db
HostName 10.0.2.15
User deploy
IdentityFile ~/.ssh/prod_key
Then connect with ssh staging-app. This works well for small setups. You can version-control the config file (minus sensitive values) and share it with teammates. And because tab completion, scp, rsync, and anything else that shells out to SSH all read the same file, an alias defined once works everywhere.
Directives cascade: for each option, the first matching value wins. Put specific hosts at the top and defaults at the bottom, and you stop repeating yourself:
Host staging-*
User deploy
IdentityFile ~/.ssh/staging_key
Host *
ServerAliveInterval 60
ServerAliveCountMax 3
AddKeysToAgent yes
The ServerAliveInterval pair keeps idle sessions from being dropped by NAT gateways and load balancers - the single most common "my SSH connection keeps dying" fix - and AddKeysToAgent means you type each key passphrase once per login session, not once per connection.
Most fleets keep production hosts off the public internet behind a jump host. Instead of SSHing twice, or copy-pasting the old ProxyCommand netcat incantation, use ProxyJump:
Host bastion
HostName bastion.example.com
User jump
Host prod-*
ProxyJump bastion
Now ssh prod-db tunnels through the bastion transparently - and port forwarding, scp, and rsync ride the same path with zero extra flags. Chains work too: ProxyJump bastion1,bastion2 for the truly locked-down environments.
One giant config file is where this approach starts to hurt. Include splits it into pieces:
# ~/.ssh/config
Include config.d/work
Include config.d/personal
Include config.d/clients/*
Each file holds one project's hosts. This is also the clean path to team sharing: check config.d/work into the team repo, keep config.d/personal out of it. (Include lines go at the top of the file, before any Host block, or they end up scoped to the last Host above them.)
If you run many short commands against the same host - deploy scripts, remote greps, editor plugins - each one pays the full SSH handshake. Connection multiplexing makes the first connection open a socket and every later one reuse it:
Host *
ControlMaster auto
ControlPath ~/.ssh/sockets/%r@%h-%p
ControlPersist 10m
Subsequent connections to the same host are near-instant, and an MFA prompt only fires once per ControlPersist window. Create the ~/.ssh/sockets directory first - ssh will not make it for you.
Match applies settings by condition rather than by hostname pattern alone - by user, by network, even by the output of a command:
# A different key when logging in as root, anywhere
Match user root
IdentityFile ~/.ssh/breakglass_key
# Everything in the prod subnet goes through the bastion
Match host 10.0.2.*
User deploy
ProxyJump bastion
Scales to: ~30-50 hosts in one flat file, considerably further with Include and ProxyJump discipline. What it never gives you: search, tagging, credential storage, or any memory of what actually runs on each host.
Tools like Termius, Royal TS, or MobaXterm provide a GUI for managing connections. You organize them into folders or groups, store credentials, and connect with a click. Termius adds cloud sync and mobile apps.
The trade-off is running a separate application alongside your terminal. You manage connections in one tool and work in another. Some people prefer this separation; others find the context switching costly. Two things worth checking before you commit: where the tool stores credentials (a local vault vs the vendor's cloud - your security team will care about the difference), and whether it can import your existing ~/.ssh/config so you are not maintaining two sources of truth.
Scales to: Hundreds of connections. Team sharing via the tool's built-in sync.
This is the approach yaw takes. Your server list lives inside the terminal itself - no external app, no browser tab, no separate credentials vault. Save each connection with a name and tags, pull it up from the command palette, and stored credentials stay on your machine behind AES-256-GCM encryption.
For fleet work, a few features matter: broadcast mode types into every open pane at once (rolling restarts, log checks); saved commands with {{variable}} placeholders reuse workflows across environments; color-coded profiles separate prod from staging visually. Tailscale nodes are auto-detected so you connect by hostname. SSH and five database engines (Postgres, MySQL, SQL Server, Mongo, Redis) share the same connection manager, and yaw connect <name> gives you CLI access without opening the GUI.
For large-scale infrastructure, tools like Ansible, Terraform, or AWS SSM Session Manager handle SSH access programmatically. Connections are defined in inventories or infrastructure-as-code, and access is managed through IAM roles or bastion hosts.
This is the right approach for large teams with dedicated DevOps. But it does not replace the need for quick, interactive SSH access when debugging or exploring.
| Approach | Sweet spot | Credentials | Team sharing | Overhead |
|---|---|---|---|---|
| ~/.ssh/config | 5-50 hosts (more with Include) | Keys on disk, agent-managed | Config in a repo; keys handled separately | None - ships with OpenSSH |
| Standalone manager (Termius, Royal TS, MobaXterm) | 50-500+ hosts | Stored in the app, often synced to vendor cloud | Built-in sync, usually per-seat pricing | A second app to run and learn |
| Terminal-integrated (yaw) | 5-500 hosts | Encrypted locally (AES-256-GCM), never leaves disk | Share definitions; each person keeps own keys | None beyond the terminal you already use |
| Config management (Ansible, SSM) | Hundreds+, audited orgs | IAM roles / vault-backed | Inventory lives in the IaC repo | Dedicated DevOps investment |
These are not mutually exclusive. A common combination is ssh_config for the muscle-memory hosts, a manager for the long tail, and SSM or Ansible for the fleet-wide operations - the point is to pick each layer deliberately instead of letting the flat file grow forever.
Individual SSH management is one problem. Team SSH management is another. How do you share connection definitions without sharing credentials?
Published by Yaw Labs.