OpenBSD sshd hardening with pf and sshguard
Goals:#
- a sane sshd configuration
- a sane pf configuration
- optional: add sshguard (but recommended)
setup SSH for your main user#
Your main user is not root. :)
Change to your regular user and setup SSH dirs:
su - niall
mkdir -p /home/niall/.ssh
chmod 700 /home/niall/.ssh
chown niall:niall /home/niall/.ssh
# add pubkey(s) you want to be authorised for login
nano /home/niall/.ssh/authorized_keys
<paste in>
Default /etc/ssh/sshd_config has key-based login enabled, so test it now:
ssh <youruser>@your_bsd_host
harden sshd_config#
important: did your regular user login work? important: can they correctly “doas” to switch to root user?
Okay - if tests pass, now we can tweak sshd_config.
Disable root login:
# nano /etc/ssh/sshd_config
PermitRootLogin no
And we will also disable password authentication:
# nano /etc/ssh/sshd_config
PasswordAuthentication no
Restart sshd:
rcctl restart sshd
Can you still login? If no, use console to recover, and recheck your config with steps above.
tighten pf firewall in /etc/pf.conf#
This hardens from default and adds extra protection for SSH with rate limiting.
Step 1: Basics#
Delete this line:
pass # establish keep-state
Add these lines in its place:
pass out keep state
pass in proto tcp to port 22 keep state
Step 2: pf logging#
This is optional but suggested to start with maximum visiblity and tune down from there.
NB: if you enable logging, you must also enable log rotation! else you may exhaust your disk. Do log rotation before proceeding to pf.conf itself:
# nano /etc/newsyslog.conf, and append this line:
# 5 files of 1MB each; gzip old files
/var/log/pflog 600 5 1000 * Z
#
# load new config
newsyslog
Step 3: /etc/pf.conf#
Here is a suggested starting point for pf.conf, the explanation is in inline comments.
Note: ports 80, 443, and 51820 are opened here for web and Wireguard respectively; delete these if you are not using them!
# machine should always be able to talk to itself; no pf on loopback
set skip on lo
# drop policy is drop (ignore) rather than return;
# return means TCP sends RST, and ICMP sends Unreachable; drop is stealthier
# in nmap terms, a scanner will show drop -> filtered; RST -> closed
set block-policy drop
# declare a persistent table; we will add SSH bruteforcer IPs here
# by persistent, the table always exists but a reboot will flush its contents
table <ssh_bruteforce> persist
# enable antispoofing; we should not receive packets inbound where SRC = us
# the macro "egress" maps to whichever interface(s) carries your default route
# quick keyword means if matched here, stop processing rules, just apply drop
antispoof quick for egress
# block anything inbound; later statements open up selectively
# we have "block in log"; you can remove the log keyword if you are not logging
block in log
# permit outbound traffic and keep state
pass out keep state
# Basic ICMP / PMTU sanity in and out; tune if you know what you are doing
pass inet proto icmp all keep state
pass inet6 proto icmp6 all keep state
# SSH with simple rate limiting
# "quick" means if match found, stop processing any further rules
block in quick from <ssh_bruteforce>
# permit 22/tcp and keep state; we are open to SSH connections
pass in proto tcp to port 22 flags S/SA keep state \
# but we limit connections to 10 per minute, and 20 maximum overall
# if IP exceeds 10 per minute, add to <ssh_bruteforce> table
# flush global means reset state for any existing connections for this IP
# adjust to taste according to your use case; don't lock yourself out!
#
# SRC IPs in this table persist until manual clear or system reboot
# .. don't cut yourself off ..
#
# .. if you want time-based expiry you can do, eg:
# doas pfctl -t ssh_bruteforce -T expire 3600
#
# .. if so, put in root's cron (crontab -e):
# */10 * * * * pfctl -t ssh_bruteforce -T expire 3600
#
(max-src-conn 20, max-src-conn-rate 10/60, overload <ssh_bruteforce> flush global)
# Web - permit regular http/80 https/443 traffic and keep state
pass in proto tcp to port { 80 443 } keep state
# WireGuard - permit Wireguard clients to the standard port and keep state
pass in proto udp to port 51820 keep state
# this is in the default pf.conf, but it overrides our block policy 'drop'
# this will result in TCP RST being sent out, rather than our drop policy
# so comment out or remove, or change 'return' to 'drop' to be doubly covered
#
# By default, do not permit remote connections to X11
#block return in on ! lo0 proto tcp to port 6000:6010
# this is in the default pf.conf, keep it
#
# Port build user does not need network
block return out log proto { tcp udp } user _pbuild
reload - dry run#
Check for syntax errors:
pfctl -nf /etc/pf.conf
reload for real#
pfctl -f /etc/pf.conf
pf verification commands#
rcctl ls on <-- shows services which start at boot
pfctl -s info <-- shows pf runtime state
tcpdump -i pflog0 <-- view live blocked traffic
pfctl -sr <-- verify running status, counters, limits
pfctl -ss <-- show active states
pflog0 example#
Monitor live blocked traffic:
# tcpdump -n -e -ttt -i pflog0
tcpdump: WARNING: snaplen raised from 116 to 160
tcpdump: listening on pflog0, link-type PFLOG
Mar 22 16:17:40.185911 rule 1/(match) block in on em0: 10.0.2.1.56690 > 10.0.2.15.666: S 122688001:122688001(0) win 65535 <mss 1460>
Mar 22 16:17:41.166677 rule 1/(match) block in on em0: 10.0.2.1.50270 > 10.0.2.15.666: S 120384001:120384001(0) win 65535 <mss 1460>
Mar 22 16:17:41.166958 rule 1/(match) block in on em0: 10.0.2.1.51163 > 10.0.2.15.666: S 120320001:120320001(0) win 65535 <mss 1460>
Mar 22 16:17:42.016383 rule 1/(match) block in on em0: 10.0.2.1.59740 > 10.0.2.15.666: S 120512001:120512001(0) win 65535 <mss 1460>
View log; this is not text it is binary and you must use tcpdump:
tcpdump -n -e -ttt -r /var/log/pflog
You should definitely check out:
Step 4: sshguard#
pf rate limiting -> overload -> <ssh_bruteforce> table is pretty good at network layer protection.
sshguard goes a step further as it works at the application layer, reacting to authentication failures in logs.
smarter and stealthier brute forcers avoid a rate limit ban by slowing their hits, but they fail authentication and show up in /var/log/authlog:
- non-existent user
- extant user but bad password
- extant user but oops no ssh pubkey offered
.. whereupon each infraction is scored and tallied by sshguard.
Therefore pf rate limiting and sshguard auth monitoring complement each other very well.
Great, let’s get it running:
install it#
pkg_add sshguard
configure /etc/sshguard.conf#
I’m using reasonably aggressive settings here for a low-use VPS:
#
# THRESHOLD=25 means 2~3 failed SSH attempts okay, then block; auth fail = 10 points, timeout = 2 points
# this is okay for legitimate users because all legit users will use SSH key
# so you can fail upto two logins due to key not loaded, oops, then load correct key and you will get in
# if you have ~/.ssh/config setup correctly on your client, this should never happen, so, I consider safe
#
# brute force connection attempts which timeout on auth score 2 points; 13th timeout = ban
#
# BLOCK_TIME 12h Entries age out every 12h
# However, repeat offenders get 1.5x block time for every offence
#
# DETECTION_TIME 24h Stealth brute force; 2 fails in 24h = ban
#
THRESHOLD=25
BLOCK_TIME=43200
DETECTION_TIME=86400
Note the detection window (DETECTION_TIME), the ban duration (BLOCK_TIME), and the consequence of repeat offending: ban duration x 1.5 for every infraction.
update /etc/pf.conf#
table <sshguard> persist <-- near top of file
block in quick from <sshguard> <-- near top of file
reload pf.conf#
pfctl -f /etc/pf.conf
enable and start sshguard#
doas rcctl enable sshguard
doas rcctl start sshguard
wait about 4 seconds#
internet crashes against VPS
view results in table#
pfctl -t sshguard -T show
tune /var/log/authlog#
OpenBSD default newsyslog.conf setting:
- keep 7 files; rotate every 7 days (and GZip); no size limit
- you might optionally adjust this for longer retention, more files/faster rotate, etc.
- but sshguard relies on network state not authlog itself.
/var/log/authlog root:wheel 640 7 * 168 Z
sshguard in action#
<snip>
# lots of brute-force activity from 118.193.61.170>
<snip>
Mar 24 12:45:06 <host> sshd-session[13633]: Invalid user arm from 118.193.61.170 port 35096
Mar 24 12:45:07 <host> sshd-session[13633]: Received disconnect from 118.193.61.170 port 35096:11: Bye Bye [preauth]
Mar 24 12:45:07 <host> sshd-session[13633]: Disconnected from invalid user arm 118.193.61.170 port 35096 [preauth]
<snip>
Mar 24 12:47:24 <host> sshd-session[19120]: Invalid user develop from 118.193.61.170 port 59878
Mar 24 12:47:25 <host> sshd-session[19120]: Received disconnect from 118.193.61.170 port 59878:11: Bye Bye [preauth]
Mar 24 12:47:25 <host> sshd-session[19120]: Disconnected from invalid user develop 118.193.61.170 port 59878 [preauth]
<sshguard enabled here>
Mar 24 12:49:37 <host> sshguard[<PID>]: Attack from "118.193.61.170" on service SSH with danger 10.
<snip>
Mar 24 12:49:37 <host> sshguard[<PID>]: Attack from "118.193.61.170" on service SSH with danger 10.
Mar 24 12:49:37 <host> sshguard[<PID>]: Blocking "118.193.61.170/32" for 43200 secs (2 attacks in 0 secs, after 1 abuses over 0 secs.)
Step 5: reboot persistence#
This is not generally recommended as it is not considered strictly necessary, and potentially undesirable, eg:
- if you lock yourself out, now there is a smidge more complexity
- bad IPs may not remain bad for very long
- I advise not to incorporate reboot persistence on the ssh tables themselves ..
- .. but you can do as I do and curate a bad hosts table and persistence file for really stubborn annoyances
- out of scope for this post: geofencing and blocking continents, I mean, you can~ if you wish
For your consideration then:
read#
If you wish for pf to read from a “bad hosts” persistence file on disk, you can use this syntax, eg:
table <bad_hosts> persist file "/etc/pf.badhosts"
Where /etc/pf.badhosts is like:
10.20.30.40/32
192.168.99.0/24
<...>
write#
If you wish to write to a “bad hosts” persistence file on disk, you can export state and overwrite the file, eg:
pfctl -t bad_hosts -T show > /etc/pf.badhosts
You want to overwrite not append else it will contain duplicates.
If you are doing this, you might want to automate with root’s crontab, edit via crontab -e and you can try something like this:
# every 10 mins, dump bad_hosts running table, sort it, save to .tmp file
# if that succeeds, then compare this output to our 'real' file; and
# if there is no difference, delete the .tmp
# else overwrite real file with the .tmp
*/10 * * * * pfctl -t bad_hosts -T show | sort > /etc/pf.badhosts.tmp && \
(cmp -s /etc/pf.badhosts.tmp /etc/pf.badhosts && rm /etc/pf.badhosts.tmp || mv /etc/pf.badhosts.tmp /etc/pf.badhosts)
Possibly you want to make this safer, and run from a ksh or bash script rather than what is here.