When I have both banmotd and whitelistmotd enabled only the whitelistmotd messages ever show for all combinations of ban/unban/whitelist/not whitelisted. I'm on Paper 1.16.5 with the latest version of the plugin and ProtocolLib.

Hey, this is expected behavior. All motd extensions (whitelist, ban, restricted mode) have a priority resulting in one motd being able to overwrite another one. The priorities are the following, from lowest to highest: Spigot Banmotd, MaxBans Banmotd, BanManager Banmotd, WhitelistMotd, RestrictedMode motd (source). So WhitelistMotd always overwrites a BanMotd.

If it's not that urgent, give me some time and you will get a modified version of ServerlistMOTD with your expected behavior. I am pretty busy atm so I am not sure when I can find some time.