I also happily use OpenBSD on my firewall. Every year or two I'm tempted by pfSense, and each time I'm scared away by the thought of running PHP on my firewall, and other security issues as outlined so well in this thread by lima [1].
The one thing I most envy is pfSense's multi-WAN failover support. I have a USB LTE modem plugged into my firewall. OpenBSD can see the device, I can use it as my default gateway for Internet access, etc. However, I only want to use it when my primary ISP is down, for some definition of "down" (e.g., major packet loss). I want the firewall to detect this condition; switch to the LTE modem as the egress interface/default route; reload a different pf.conf, tuned for the LTE modem; restart ospfd, ospf6d, and rtadvd; and then watch the primary interface until it recovers, at which point the firewall reverts to its primary config and restarts the routing daemons again.
It appears that pfSense supports this scenario out-of-the-box, via "gateway groups" and multi-level tiers [2]. OpenBSD does not -- its multi-WAN support is intended for load-balancing, and/or assumes a carrier-grade routing environment using BGP or equivalent. People have asked for help with this scenario on the OpenBSD mailing lists, and the answer is usually, "this is what ifstated is for, write a script"; which is fine, I guess, but it would be nice if everyone who wanted to do this didn't have to write their own bespoke solution.
Speaking of which, does anybody here do this with their OpenBSD firewall and has a nice, clean, well-documented example of how it works? I once trawled through the pfSense source code to see how they do it, and their implementation was all wrapped up in a bunch of PHP. Yuck!
the bespoke part is "what is your definition of down?". is it 10% packet loss? is it physical loss of link? is it something else? is it a combination? is it 15 seconds of $X or a full minute?
My solution to this is to have my servers' console plugged into a separate switch and LTE router than their ethernet cards so not only do I have a redundant WAN, but a redundant router and switch as well.
The one thing I most envy is pfSense's multi-WAN failover support. I have a USB LTE modem plugged into my firewall. OpenBSD can see the device, I can use it as my default gateway for Internet access, etc. However, I only want to use it when my primary ISP is down, for some definition of "down" (e.g., major packet loss). I want the firewall to detect this condition; switch to the LTE modem as the egress interface/default route; reload a different pf.conf, tuned for the LTE modem; restart ospfd, ospf6d, and rtadvd; and then watch the primary interface until it recovers, at which point the firewall reverts to its primary config and restarts the routing daemons again.
It appears that pfSense supports this scenario out-of-the-box, via "gateway groups" and multi-level tiers [2]. OpenBSD does not -- its multi-WAN support is intended for load-balancing, and/or assumes a carrier-grade routing environment using BGP or equivalent. People have asked for help with this scenario on the OpenBSD mailing lists, and the answer is usually, "this is what ifstated is for, write a script"; which is fine, I guess, but it would be nice if everyone who wanted to do this didn't have to write their own bespoke solution.
Speaking of which, does anybody here do this with their OpenBSD firewall and has a nice, clean, well-documented example of how it works? I once trawled through the pfSense source code to see how they do it, and their implementation was all wrapped up in a bunch of PHP. Yuck!
[1] https://news.ycombinator.com/item?id=13615724
[2] https://doc.pfsense.org/index.php/Multi-WAN#Failover