🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

netcode.io and MMOs

Started by
2 comments, last by Randy Gaul 5 years, 2 months ago

So I've never made an MMO before, but I know some of you guys have worked on them! I have been studying netcode.io and wanted to ask if anyone here had opinions or insights on why netcode would not be a very good fit for huge MMO style games.

In case anyone is not familiar, netcode was mostly designed for games with around 2-64 players or so, like FPS games or agar.io style games, and uses UDP. It has a well designed secure connection authentication system, using opaque connect tokens derived by piggy-backing off of a secure HTTPS REST call to a webserver authenticator. Once a token is obtained by the client by making a REST call, they hand an encrypted section of the token to a dedicated server and begin the connection handshake with a secret keyset obtained from the webserver over HTTPS. It's all actually quite simple, and documented in the netcode standard. To me, this is the most appealing feature of netcode and libyojimbo (libyojimbo is a C++ wrapper around netcode that also adds in HTTPS support, basic matchmaking, and reliability features).

For context, say we are talking about an MMO where anywhere from 2-3000 players can reside within an active instance of the game on a single server. This figure can be thought of as the peak capacity for any of the MMO dedicated servers. The gameplay typically looks like Oldschool Runescape - lots of click and watch the player slowly walk to the click point. Event/RPC style programming would be pretty typical, and over TCP would work just fine traditionally. In the case of netcode, the reference implementation keeps a pre-allocated array of client instances. A quick test shows all pre-allocated data comes out to just ~30MB for 3000 players, for one dedicated server instance, which is really quite small (this figure is just describing the memory cost on the protocol level, not the per-client memory cost of the actual game simulation).

netcode has some tuning parameters for connection timeouts and packet send rates, which are ten times per second. If these are too high for an MMO, this can be tuned.

Of course, netcode itself does not handle packet reliability. However, netcode can be wrapped with a reliability implementation (like libyojimbo), so let us assume the reliability adds another 30-100 MB at peak server capacity, totaling to around 100-150MB for the server instance when no packets are in flight. Of course, when packets are in flight, memory consumption goes up as packets get buffered. I'm not sure how much, but let us assume a gigabyte of RAM should be reserved for packet traffic and the server protocol instance itself, along with all clients. This could leave ~4-6 GB of space to run the instance on a device with around ~8 GB of RAM total.

Okay, that's probably enough context. With this kind of story, in your experience what are the key requirements for an MMO that netcode might miss? Are there any other unfavorable aspects of netcode that would prevent it from being a good MMO option?

Here was the only point I could think of: netcode's default implementation pulls packets off of the UDP stack only via polling. Instead, I think a single dedicated thread could be used to read off of the UDP socket, and queue packets up. The queue could be polled by the user calling into netcode. This can prevent the UDP stack buffer from filling up and causing packet loss.

Advertisement

Network communication, as in the basics of authentication and getting packets to and from machines, are not a challenge. (I mean, yes, it's a challenge to someone starting from scratch, not having done networking before, but it's not a "hard" problem in the sense that there exists no good known solutions.)

Twenty years ago, it used to be that a single server computer only had the capacity for between 20 and 200 players (20 if physically simulated, 200 if more basically collision-tested for impossible movement.) At that time, there was a significant challenge around trying to make the game support "large areas" even when individual computers couldn't do that, and a number of simulation distribution mechanisms came up. If you want to still do this today, there are some interesting challenges there, but on a modern multi-core CPU with tons of cache, you may be able to get to 5,000 players in a MMO on a single host.

Additionally, there are significant challenges around MMOs when it comes to interest management -- you don't want each player to see all 5,000 other players in the world, but instead only see the players and objects that are "interesting," which usually means something like "in the group, or targeted by the group, or targeting the group, or close to the player." Thus, calculating the interesting set of objects to send over the network connection is a challenge for MMOs, and it gets hard when you need to efficiently support the "this is now interesting" and "this is now not interesting" transitions. There's also the challenge that many players want to gather in one area -- be it an arena, a market square, a big raid, or something like that. When N players each interact with (N-1) other nearby players, you run into an N-squared problem that eventually will bring any kind of system to its knees, and you need a good solution to this (e g, load shedding, in a way that's masked to the player.)

MMOs are also challenging in that gameplay itself needs to be designed to support the specific networking of the game. If you don't do full physical simulation on the server, but instead just sweep spheres or rays against a static collision mesh to verify that movement is allowed, then you need to design gameplay that is interesting and plays well, under these limitations. If a player can't see far away (because of interest management,) then sniper rifles or spells with very long range or simple binoculars will be challenging. If it's bad to have a thousand players all in the same market square, then you need to design game systems that fulfill the need for social interaction and trading, but naturally funnel people into smaller, separate, areas, say by interest, or level, or whatever.

Finally, the main reason MMOs are challenging is the sheer amount of content that needs to be created. This has nothing to do with networking, and everything to do with budget and production schedules. If you can't create a large world that can entertain thousands of players at the same time, then you need to figure out how to shard your playerbase across the same content you CAN create, and do so in a way that the social interaction element of MMOs still draws players to the game.

So, what networking library you use -- netcode, lidgren, enet, unreal, proton, raknet, straight TCP sockets, or anything else -- really doesn't matter much at all in the grand scheme of things. Pick, or build, something that works alright, and then solve the REAL problems with MMO games.

enum Bool { True, False, FileNotFound };

Thanks @hplus0603! I suspected this may be the case, and it's great to hear your thoughts. One thing I noticed is it can be a bit difficult for me to read online because a lot of information and opinions are outdated, and said in a time where hardware and memory constraints were a lot tighter.

I think some older articles and posts mix together the MMO/game problems with the protocol layer code, and have tight coupling. But as time went on there are more open sources and more resources for learning, that are higher quality without such tight coupling.

This topic is closed to new replies.

Advertisement