How Do You Manage SSH Keys Securely in a DevOps Team?
Learn how DevOps teams manage SSH keys securely — per-engineer keys, centralized distribution, and short-lived certificates for instant revocation.
Expected Interview Answer
Secure SSH key management means generating a unique asymmetric key pair per engineer (never sharing private keys), distributing only public keys to servers via a centralized identity or configuration-management system, enforcing passphrases plus hardware-backed storage, and rotating or revoking keys immediately when access changes.
Each engineer keeps a private key locally, protected by a passphrase and ideally a hardware token such as a YubiKey, while the corresponding public key is pushed to every authorized host through configuration management, a bastion host, or an identity provider like AWS IAM/SSO rather than copied by hand. Centralizing authorized_keys distribution means revoking one engineer touches one source of truth instead of hunting across dozens of servers. Short-lived certificate-based SSH (via an SSH CA, e.g. HashiCorp Vault or step-ca) is the modern replacement for static keys, since certificates expire automatically and never need manual revocation. Auditing who can SSH where, disabling password authentication, and forcing key rotation on offboarding are the operational guardrails that turn “we use SSH keys” into an actually secure system.
- Eliminates shared/static credentials and reduces blast radius
- Enables instant, centralized revocation on offboarding
- Short-lived certificates remove manual rotation overhead
- Provides an auditable trail of who can access which hosts
AI Mentor Explanation
Managing SSH keys is like a franchise issuing each player their own personalized locker key rather than one master key shared by the whole squad. If a player is traded away mid-season, the franchise only needs to deactivate that one locker key instead of rekeying every locker in the building. A rolling “guest pass” system for touring net bowlers, valid for one practice session only, mirrors how short-lived SSH certificates expire automatically. The equipment manager keeps a signed record of every key ever issued, so any lost key can be traced back to exactly who held it.
Step-by-Step Explanation
Step 1
Generate a unique key pair per engineer
Each person creates their own private/public key pair locally, protected by a passphrase, never shared.
Step 2
Distribute public keys centrally
Push authorized public keys to hosts via configuration management, IAM, or an SSH certificate authority — never manual copy-paste.
Step 3
Prefer short-lived certificates over static keys
Use an SSH CA (e.g. HashiCorp Vault, step-ca) to issue certificates that expire automatically.
Step 4
Audit and revoke on offboarding
Maintain a single source of truth for who can access what, and revoke immediately when someone leaves or changes roles.
What Interviewer Expects
- Understanding that private keys must never be shared or transmitted
- Knowledge of centralized distribution vs manual authorized_keys editing
- Awareness of short-lived SSH certificates as the modern best practice
- Ability to explain the offboarding/revocation workflow
Common Mistakes
- Sharing one SSH key pair across an entire team
- Manually editing authorized_keys on each server instead of centralizing it
- Never rotating or expiring keys after someone leaves
- Leaving password authentication enabled alongside key-based auth
Best Answer (HR Friendly)
“We give every engineer their own SSH key pair, keep the private key on their machine protected by a passphrase, and only ever distribute public keys through our configuration management or identity system — never by copying files around manually. Where possible we use short-lived SSH certificates that expire automatically, and the moment someone leaves the team we revoke their access centrally instead of having to hunt across every server.”
Code Example
# Generate a strong Ed25519 key pair, protected by a passphrase
ssh-keygen -t ed25519 -C "jordan@company.com" -f ~/.ssh/id_ed25519
# Sign the public key into a short-lived certificate (valid 8 hours)
step ssh certificate jordan@company.com ~/.ssh/id_ed25519.pub \
--host --not-after 8h
# Revoke instantly by removing the principal from the CA policy,
# no need to touch authorized_keys on any server
step ssh revoke <cert-serial>Follow-up Questions
- What is the difference between an SSH key and an SSH certificate?
- How would you rotate SSH keys across 200 servers without downtime?
- Why is disabling password authentication important for SSH hardening?
- How does an SSH certificate authority simplify offboarding?
MCQ Practice
1. What is the recommended practice for SSH private keys?
Private keys must remain exclusively with their owner and be passphrase-protected; only the public key is ever distributed.
2. Why are short-lived SSH certificates preferred over static keys in modern DevOps?
Certificates issued by an SSH CA carry an expiry, so access is naturally revoked without editing authorized_keys on every host.
3. What should happen immediately when an engineer leaves the team?
A centralized identity or CA-based system lets you revoke exactly one person's access immediately, without touching every server manually.
Flash Cards
What should never be shared between engineers? — The private SSH key — only the public key is distributed.
What is the modern alternative to static SSH keys? — Short-lived SSH certificates issued by an SSH certificate authority.
How should authorized_keys be managed at scale? — Centrally, via configuration management or IAM — never by manual per-server edits.
What must happen on offboarding? — Immediate, centralized revocation of that person's SSH access.