Security Difficulty: Beginner

Hardening SSH access

Key-based login instead of passwords and no root login: the most important moves to harden the front door to your server.

Jul 14, 2026 · 4 min read ·Duration: approx. 20 minutes

Table of contents

After the initial setup we harden the SSH access – the most important door to your server.

What are we building?

By the end you log in with an SSH key instead of a password, and neither root login nor password login is possible from outside. This makes the automated password-guessing attacks that hit every publicly reachable server around the clock run completely into the void. Tested with OpenSSH on Debian 13.

Prerequisites

A set-up server with a sudo user
Access to the VNC console in the netcup SCP as a safety line in case you lock yourself out

Step by step

Step 1: Generate an SSH key pair

If you don’t have a key yet, generate an Ed25519 key pair on your own machine (not on the server!):

Terminal

ssh-keygen -t ed25519

You can accept the suggested storage location with Enter. Set a passphrase – it protects the key if your machine falls into the wrong hands. Two files are created: ~/.ssh/id_ed25519 (private, stays with you) and ~/.ssh/id_ed25519.pub (public, goes on the server).

Step 2: Transfer the public key

ssh-copy-id appends your public key to the ~/.ssh/authorized_keys file of your server user – with the correct file permissions:

Terminal

ssh-copy-id koch@YOUR_SERVER_IP

Test right afterwards that key login works:

Terminal

ssh koch@YOUR_SERVER_IP

You should now be logged in without a server password (only the passphrase of your key may be requested). Only once that works do we continue – in the next step we disable password login.

Open the SSH server configuration:

Terminal

sudo nano /etc/ssh/sshd_config

Set (or uncomment) these two lines:

Ausgabe

PermitRootLogin no
PasswordAuthentication no

PermitRootLogin no blocks direct root login completely, PasswordAuthentication no allows only key logins.

Warning

On cloud images there are often extra files under /etc/ssh/sshd_config.d/ that override these settings. Check with grep -r PasswordAuthentication /etc/ssh/sshd_config.d/ and adjust any matches as well.

Step 4: Test the configuration and restart SSH

Check the configuration for syntax errors before you restart – a typo would otherwise take the SSH service down:

Terminal

sudo sshd -t

No output means: all good. Then apply it:

Terminal

sudo systemctl restart ssh

Caution

Test key login now in a second terminal session before you close the first – otherwise you might lock yourself out.

To verify: sudo systemctl status ssh should show active (running), and a login attempt as root must now be rejected with Permission denied (publickey).

When things go wrong

SSH keeps asking for the password despite ssh-copy-id. Usually the file permissions on the server are wrong. SSH ignores authorized_keys if the permissions are too open: chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys. Also check that you’re connecting with the right user (koch@…, not root@…).

Permission denied (publickey) – and you can’t get in at all anymore. Password login was disabled before the key worked. No drama: open the VNC console in the netcup SCP, log in locally there, set PasswordAuthentication yes, restart SSH and start again at step 2.

After the restart the SSH service no longer starts. Syntax error in the sshd_config (that’s why you always run sshd -t before restarting). Log in via the VNC console; sudo sshd -t shows you the file and line number of the error.

The settings seem to take effect, but password login still works. A file under /etc/ssh/sshd_config.d/ (often 50-cloud-init.conf) overrides your values. sudo sshd -T | grep -i passwordauthentication shows the actually effective setting – adjust matches in the extra files and restart SSH.

Maintenance & backups

Back up the private key: Without ~/.ssh/id_ed25519 you can only reach the server via the VNC console. Back up the key encrypted (e.g. in a password manager) or add a second key from another device to authorized_keys.
Keep an eye on logins: sudo journalctl -u ssh --since today shows you all login attempts. Failed attempts on port 22 are normal and, thanks to the key requirement, harmless – you can automatically ban such bots later with Fail2ban.
Updates: OpenSSH gets its security updates through the normal apt upgrade – so keep the server up to date, ideally automated.

Last updated: Jul 19, 2026

#netcup #debian #ssh #beginner

Send feedback: feedback@serverkueche.de