A whitelisted server for friends barely needs a security section. A public server is a different job, because the moment your address is open, you inherit everyone who finds it, including the people looking for something to break. The busier and more popular your server gets, the more likely it is to be attacked, and that is exactly when going offline hurts most. This post is the practical checklist for running a public SMP without handing it to the first person who wants to ruin it. It assumes you have the single server built from running a Minecraft SMP and, if you have grown past one box, the network from scaling to a network with Velocity.
Security here is a set of habits, each closing a different door. Work through them in order and the common attacks stop being your problem.
Whitelist versus open, and how to run open safely
The whitelist is the strongest control you have, because a name that is not on it never gets in. If your server is genuinely for a known group, keep the whitelist on and most of this post is optional. Going open is a real decision that trades that safety for reach, so make it deliberately. If you do open up, do it in layers rather than all at once: keep the protection and rollback tools from your first-day setup active, keep new players in a limited default permission group, and make sure spawn and key builds are claimed or protected before strangers arrive. Open does not have to mean unprotected. It means the other habits below stop being optional.
Keep online-mode true
This is the one that is not negotiable. Online-mode true means the server verifies every player against Mojang's authentication, so an account name genuinely belongs to the person using it. Turn it off, or run a cracked or offline server, and anyone can log in as any name, including yours, which hands an attacker your operator account and every permission attached to it. Do not expose an offline-mode server to the internet. If you run a proxy, this same trust lives in the proxy configuration, and the backends must sit behind it rather than being reachable directly, which is covered next.
Firewall backends behind the proxy
If you have scaled to a network, the single most important security rule is that only the proxy is exposed and every backend is firewalled to accept connections only from it. With Velocity's modern forwarding, a backend trusts player identity that is signed with the shared secret, so a backend reachable from the open internet can be connected to directly and have that trust abused. Bind backends to the internal network, block their game ports for everyone except the proxy address, and expose only the proxy to players. The forwarding and connection detail is in scaling to a network with Velocity. A single server without a proxy simply exposes its one game port and nothing else.
Anti-cheat, and when it is worth it
An anti-cheat costs tick time, so it is a trade rather than a free win, and the answer depends on your server. On a small whitelisted friends server it is usually overkill, because you already trust everyone on it. On a public server, and especially one with PvP where cheating directly ruins other players' experience, it earns the tick it costs. Add it when the playerbase is large enough or the PvP serious enough that hand-moderation cannot keep up, and skip it while a couple of trusted people can watch the whole server themselves.
DDoS protection
A denial-of-service attack does not break into your server. It drowns the connection so real players cannot reach it, and popular servers attract exactly that. The common way to stop an attack is to null-route the target, which stops the attack by taking the server offline with it, which is not much of a win for you. The approach we take instead is to filter attack traffic inside the network fabric at line rate, so legitimate players keep getting through while an attack is happening rather than everyone going dark. The full story of how that filtering works, in the switches rather than in a bottleneck box downstream, is in our network. One deliberate note from that piece: we do not publish our capacity figures, because a stated number just becomes a target to aim at, and the useful thing to know is that filtering happens in the fabric rather than by pulling your server off the internet.
Login rate-limiting
Some attacks are not floods of traffic but floods of connection attempts, bots hammering the login repeatedly to exhaust resources or brute their way past controls. Rate-limiting logins caps how many connection attempts a single source can make in a window, so one address cannot tie up the server by knocking constantly. Both a proxy and a plugin can enforce this, and on a public server it is worth having so a single misbehaving client cannot degrade the experience for everyone else. If you run a Velocity proxy, this is built in: the login-ratelimit setting in the [advanced] section of velocity.toml is the minimum time in milliseconds between connection attempts from one IP, and it defaults to 3000, so one attempt every three seconds per address. That default is sensible for most servers. Lower it towards 1000 if a lot of your players share an address behind NAT, such as a school or a household, and set it to 0 only if you genuinely need to turn it off. A single server without a proxy has no equivalent built-in control, so if you are open without one, a dedicated anti-bot plugin such as EpicGuard can add per-IP connection rate-limiting, ping-before-join checks and username filtering on top.
Keep software updated
Old versions are where public exploits live, because a patched vulnerability is also a published one, and unpatched servers are the easy targets. Keep the server software, the proxy and every plugin current, and pay particular attention to security releases for anything that faces players directly. Updating is dull and it is the highest-value security habit you have, because most real-world compromises use a known hole that a routine update would have closed weeks earlier.
Backups are part of security
The last habit is the one that assumes the others sometimes fail. A tested, off-box backup is what turns a successful grief, a bad update or a compromise from a disaster into an inconvenience, because you can roll back to a known-good world instead of rebuilding from memory. Treat backups as a security control in their own right: automatic, stored off the server, and restored at least once so you know the restore actually works. The full approach is in Minecraft server backups.
None of these habits is exotic, and that is the point. Online-mode on, backends firewalled behind the proxy, anti-cheat where PvP warrants it, DDoS filtered in the fabric, logins rate-limited, software current, and backups tested. Do them and the common attacks stop being events. If you want a network layer that filters attacks in-fabric rather than null-routing you off the internet, you can browse game hosting and build on it.