To enable SSH key-based authentication, generate ssh public and private key pairs using the below command. Ssh-keygen -t rsa -b 4096 ssh-copy-id -i /.ssh/idrsa.pub email protected Now login to the host manually to see if key-based authentication works fine. Open your VScode remote SSH configuration file and add the below parameter.

The goal of this document is to help operational teams with the configuration of OpenSSH server and client.All Mozilla sites and deployment should follow the recommendations below.The Security Assurance and Security Operations teams maintain this document as a reference guide.

  • If you see No such file or directory or no matches found it means that you do not have an SSH key and you can proceed with the next step and generate a new one. Generate a new SSH key pair. The following command will generate a new 4096 bits SSH key pair with your email address as a comment: ssh-keygen -t rsa -b 4096 -C 'youremail@domain.com'.
  • Ssh-keygen -t rsa -b 4096 During the process you will be prompted for a password. Simply hit Enter when prompted to create the key. By default the public key is saved in the file /.ssh/idrsa.pub, while /.ssh/idrsa is the private key. Now copy the idrsa.pub file to the remote host and append it to /.ssh/authorizedkeys by entering.
  • $ ssh-keygen -t rsa -b 4096 -C 'youremail@example.com' This creates a new ssh key, using the provided email as a label. Generating public/private ed25519 key pair.

Most default OpenSSH settings that are security-related already provide good security, thus changing them is at your own risk and is not documented here. For example, these guidelines assume only SSH protocol 2 is configured in the server, and SSH protocol 1 is disabled. This also assumes that you are keeping OpenSSH up-to-date with security patches.See man sshd_config, man ssh_config for more information on specific settings if you nevertheless need to change them. |

Configuration

Different versions of OpenSSH support different options which are not always compatible. This guide shows settings for the most commonly deployed OpenSSH versions at Mozilla - however, using the latest version of OpenSSH is recommended.

Modern (OpenSSH 6.7+)

File: /etc/ssh/sshd_config

File: /etc/ssh/moduli

All Diffie-Hellman moduli in use should be at least 3072-bit-long (they are used for diffie-hellman-group-exchange-sha256) as per our Key management Guidelines recommendations. See also man moduli.

To deactivate short moduli in two commands: awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.tmp && mv /etc/ssh/moduli.tmp /etc/ssh/moduli

Intermediate (OpenSSH 5.3)

This is mainly for use by RHEL6, CentOS6, etc. which run older versions of OpenSSH.

File: /etc/ssh/sshd_config

File: /etc/ssh/moduli

All Diffie-Hellman moduli in use should be at least 2048-bit-long. From the structure of moduli files, this means the fifth field of all lines in this file should be greater than or equal to 2047.

To deactivate weak moduli in two commands: awk '$5 >= 2047' /etc/ssh/moduli > /etc/ssh/moduli.tmp; mv /etc/ssh/moduli.tmp /etc/ssh/moduli

Multi-Factor Authentication (OpenSSH 6.3+)

Recent versions of OpenSSH support MFA (Multi-Factor Authentication). Using MFA is recommended where possible.

It requires additional setup, such as using the OATH Toolkit or DuoSecurity.

ATTENTION In order to allow using one time passwords (OTPs) and any other text input, Keyboard-interactive is enabled in OpenSSH. This MAY allow for password authentication to work. It is therefore very important to check your PAM configuration so that PAM disallow password authentication for OpenSSH.

OpenSSH 6.3+ (default)

File: /etc/ssh/sshd_config

OpenSSH 5.3+ w/ RedHat/CentOS patch (old)

File: /etc/ssh/sshd_config

PAM configuration for use with the OATH Toolkit or DuoSecurity as second authentication factor.

File: /etc/pam.d/sshd

The PAM stack in this scenario executes the following logic (in our example wefollow the flow with pam_duo.so in use)

  • The pam_sepermit.so module is called which checks if the user attempting tolog in via SSH is present in the /etc/security/sepermit.conf.If the user is present in the config file, and the config asserts that the usercan only log in if SELinux is enforcing, and SELinux is not enforcing, thenthe PAM control of required prevents the user from logging in (though PAMwould continue down the stack).
  • The password-auth include is commented out and skipped
  • The /lib64/security/pam_duo.so module is called and the user is prompted fora duo MFA code.
    • If the code provided is correct PAM immediately permits the user access anddoesn’t continue executing.
    • If the code provided is incorrect, PAM continues down the stack
  • The pam_nologin.so checks if the file /etc/nologin exists and if so blocksaccess to the user.
  • If at the end of the stack, the single sufficient control of pam_duo.sodid not return a success, PAM defaults to deny and denies the login.

Ciphers and algorithms choice

  • When CHACHA20 (OpenSSH 6.5+) is not available, AES-GCM (OpenSSH 6.1+) and any other algorithm using EtM (Encrypt then MAC) disclose the packet length - giving some information to the attacker. Only recent OpenSSH servers and client support CHACHA20.
  • NIST curves (ecdh-sha2-nistp512,ecdh-sha2-nistp384,ecdh-sha2-nistp256) are listed for compatibility, but the use of curve25519 is generally preferred.
  • SSH protocol 2 supports DH and ECDH key-exchange as well as forward secrecy. Regarding group sizes, please refer to Key management Guidelines.

The various algorithms supported by a particular OpenSSH version can be listed with the following commands:

Configuration

If you have a file containing known_hosts using RSA or ECDSA host key algorithm and the server now supports ed25519 for example, you will get a warning that the host key has changed and will be unable to connect. This means you will have to verify the new host key.

The following configurations expect a recent OpenSSH client, as updating OpenSSH on the client side is generally not an issue.

Modern

This configuration is less compatible and you may not be able to connect to some servers which use insecure, deprecated algorithms. Nevertheless, modern servers will work just fine.

File: ~/.ssh/config

Intermediate (connects to older servers)

This configuration can connect to older OpenSSH servers which run old or intermediate configurations.

File: ~/.ssh/config

Ssh 4096

Key generation

Large key sizes are used as SSH keys are not renewed very often (see also Key management Guidelines).

Don’t hesitate to create multiple different keys for different usages. In particular, never mix your personal and Mozilla keys.

You may then want to add the new key to your SSH agent or your configuration file (or both).

Protection of user keys

  • Protected by strong passphrase.
  • Never copied to another system than your own workstation/personal physical disks/tokens.
  • Use SSH forwarding or SSH tunneling if you need to jump between hosts. DO NOT maintain unnecessary agent forwarding when unused.

Protection of machine keys

When SSH keys are necessary for automation between systems, it is reasonable to use passphrase-less keys.

  • The recommended settings are identical to the user keys.
  • The keys must be accessible only by the admin user (root) and/or the system user requiring access.
  • Usage of machine keys should be registered in an inventory (a wiki page, ldap, an inventory database), to allow for rapid auditing of key usage across an infrastructure.
  • The machine keys should be unique per usage. Each new usage (different service, different script called, etc.) should use a new, different key.
  • Only used when strictly necessary.
  • Restrict privileges of the account (i.e. no root or “sudoer” machine account).
  • Using a ForceCommand returning only the needed results, or only allowing the machine to perform SSH-related tasks such as tunneling is preferred.
  • Disable sftp if not needed as it exposes more surface and different logging mechanisms than SSH (and thus scp) itself.

Multi-factor bypass setup for machine keys

Machine keys do not play well with multi-factor authentication as there is no human interaction.

  • All logins from machine accounts should be protected by an additional authentication layer (VPN, another machine, etc.).
  • All logins from machine accounts are only allowed within the private IP-space, and if possible, only the relevant machine source IPs should be accessible.

File: /etc/ssh/sshd_config (OpenSSH 6.3+)

File: /etc/ssh/sshd_config (OpenSSH 5.3+ w/ RedHat/CentOS patch)

Auditing your existing SSH keys

Existing keys are generally stored in ~/.ssh/ (Linux/OSX) or %APPDATA% (Windows). Look for id_{rsa,ed25519,ecdsa,dsa}, identity, IdentityFile, *.pem, and other identity files.

Display SSH keys information

SSH agent forwarding

ATTENTION SSH Agent forwarding exposes your authentication to the server you’re connecting to. By default, an attacker with control of the server (i.e. root access) can communicate with your agent and use your key to authenticate to other servers without any notification (i.e. impersonate you).For this reason, one must be careful when using SSH agent forwarding. Defaulting to always forwarding the agent is strongly discouraged.Note also that while the attacker can use your key as long as the agent is running and forwarded, he cannot steal/download the key for offline/later use.

SSH forwarding allows you to jump between hosts while keeping your private key on your local computer. This is accomplished by telling SSH to forward the authentication requests back to the ssh-agent of your local computer. SSH forwarding works between as many hosts as needed, each host forwarding new authentication request to the previous host, until the ssh-agent that holds the private key is reached.

On each host, two environment variables are declared for the user enabling ssh-agent:

  • $SSH_AUTH_SOCK declares the location of the unix socket that can be used to forward an authentication request back to the previous host.(ex: /tmp/ssh-NjPxtt8779/agent.8779). Only present if using SSH agent forwarding.
  • $SSH_CONNECTION shows the source IP and port of the previous host, as well as the local IP and port. (ex: 10.22.248.74 44727 10.8.75.110 22).

To use ssh-agent, add the flag -A to your ssh commands:

You can set the following configuration parameter in your local ssh configuration at ~/.ssh/config.

Hardening the Agent forwarder

It is possible to require confirmation every time the agent is used (i.e. when you connect to a server through the SSH agent) by using the -c flag:

It is also possible to lock the key in the agent after a configurable amount of time, this can be done either for all keys when starting the agent, or per key when adding the keys to the agent with the -t flag:

For MacOSX in particular it’s possible to save the passphrase in the Keychain. If you do so it is strongly recommended to also change the keychain setting to lock itself when the computer is locked, and/or to timeout and lock the keychain. These settings are not controlled by OpenSSH.

Recommended, safer alternatives to SSH agent forwarding

OpenSSH >=7.3

OpenSSH 7.3 onwards allow users to jump through several hosts in a rather automated fashion. It has full support for scp and sftp commands as well as regular ssh.

For example to reach a host behind a bastion/jumphost:

You can also add these lines to your ~/.ssh/config

Older versions of OpenSSH

It is possible to directly forward ports for single jumps instead of forwarding the agent. This has the advantage of never exposing your agent to the servers you’re connecting to.

For example, you can add these lines to your ~/.ssh/config

This will automatically forward the SSH connection over ssh.mozilla.com when you connect to a mozilla.com SSH server.

Key material handling

Key material identifies the cryptographic secrets that compose a key. All key material must be treated as MOZILLA CONFIDENTIAL GROUP RESTRICTED data, meaning that:

4096 Ssh Keygen

  • Only individual with specific training and need-to-know should have access to key material.
  • Key material must be encrypted on transmission.
  • Key material can be stored in clear text, but only with proper access control (limited access).

This includes:

  • OpenSSH server keys (/etc/ssh/ssh_host_*key)
  • Client keys (~/.ssh/id_{rsa,dsa,ecdsa,ed25519} and ~/.ssh/identity or other client key files).

Client key size and login latency

In order to figure out the impact on performance of using larger keys - such as RSA 4096 bytes keys - on the client side, we have run a few tests:

On an idle, i7 4500 intel CPU using OpenSSH_6.7p1, OpenSSL 1.0.1l and ed25519 server keys the following command is ran 10 times:

time ssh localhost -i .ssh/id_thekey exit

Results:

Client keyMinimumMaximumAverage
RSA 4096120ms145ms127ms
RSA 2048120ms129ms127ms
ed25519117ms138ms120ms

Ssh 4096

Keep in mind that these numbers may differ on a slower machine, and that this contains the complete login sequence and therefore is subject to variations. However, it seems safe to say that the latency differences are not significant and do not impact performance sufficiently to cause any concern regardless of the type of key used.

Reference documents