🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Distrubuted actor model for a realtime game server

Started by
1 comment, last by wodinoneeye 8 years, 8 months ago

We've (work team) used the approach to successfully scale out a few high demand event driven applications, so I'm toying with the design of a game server that would use this approach.

I'm running in to some sticking points:

1. Where to put the loop...

One way would be to have an actor that publishes a tick event and then all the other entity actors do a simulation step. This seems a bit too centralized though, one very busy actor which can quickly become a bottle neck.

Another approach I've seen is that each entity actor runs it's own loop, but then you still need a synchronization mechanism which will also become a bottleneck.

2. Sharing state

State sharing obviously violates the model, nor does it work, so this creates an interesting problem.

Entities are going to need to be able to "see" the world. Passing a reference to the world is all that's normally needed, but in the case of actors, it becomes very costly to do this because of sending such a large message frequently to many entities.

Requesting a glimpse/chunk might be a workable approach, but they would then need to get this from a "World/Map/ZoneActor", making that a single very busy actor.

Would probably use either Akka.NET or Project Orleans for the model implementation.

I'm just looking for any ideas or input from people who've maybe gone down this route before.

Advertisement
First of all: How many actors? If it's 50, anything you do will work.
That being said, a thread per actor is the worst possible idea. I hope I don't even need to explain why.

Second of all: What physics engine will you be using?
Most physics engines want to do the physics thing for themselves, so your update needs to synchronize with that.
E g, you'll have to ask all actors what to do with the world state or input they have, then step physics, then tell all actors to update their post-physics reactions.

Third of all: Why do you think "reference to world" is a big thing to send? It should be a single pointer.
If you're trying to use Enterprise programming libraries and object serialization within your game engine, you are doomed to fail.
Those systems are built to serve very different use cases than game engines.

Can you use reactive programming in games? Yes! But beware libraries that try to make bindings fancy, or that prefer allocating new objects to re-configuring existing objects in place.
Can you use threading in games? Yes! But you'd want a work queue, and N worker threads (one per CPU,) and each actor/step/whatever is a work item. You'd also want cheap synchronization on this queue (a la lock-less FIFO) if you have lots of actors.

Btw, everyone "touching" the world was one of the poor design decisions that Project Darkstar (Sun's ill fated MMO engine) did.
They ended up locking way too much. And the scalability ended up being negative -- the more servers they added, the fewer entities the system could siulate overall.

A game engine architecture that scales better, at least across CPUs, is one where state is double-buffered. The "current state" (state of the last tick) is available for all actors to view; actors compute the new state into a "new state" area. So there are two sets of state; an actor is not allowed to read "new state." Once all actors have updated and you commit the step, swap the state areas, and "new state" becomes "current state."
You can also scale this across networks, where actors would send a replica of their new state to other listeners.

The other problem you'll run into is that simulation is inherently an n-squared problem, because all actors could want to congregate in a crowded area -- dogpile; auction house; indoors firefight; whatever. When all actors can affect all other actors, that's a N-squared problem that can't be reduced. Only game design can help alleviate this problem.
enum Bool { True, False, FileNotFound };

Marshelling (inter)actions (at a sufficient time granularity) and then combining and arbitrating them together is preferable to a immediate tick activation of all involved objects (where you may also have to refigurer what other object are involved EVERY TIME).

Depending on you simulation, you might have a cascade of sideeffects (which just cause more activations ad nauseum).

--------------------------------------------[size="1"]Ratings are Opinion, not Fact

This topic is closed to new replies.

Advertisement