I am just thinking of how I should be architecting a server in which it has to serve clients, and also able to receive requests coming from other servers. My network programming experience is limited, so I am having problem picturing this in my mind.

Clients connect to the server, and maintain active TCP connections. In the application code level, this is represented by an object or some handle. Other servers will talk to this server, and each will also have its own active connection objects.

Suppose there is a scenario in which another server sends a command that should be broadcasted to all (or perhaps select) clients. Does this mean that the application code will have to iterate through all the client connections, pick the relevant (or all of them) clients, and broadcast the message?

This also means that the application code can only exist in one beefy box. Clustering them into several boxes with a load balancer does not seem to be a good idea, as the clients connections will be spread out throughout the boxes. Does this sound correct? How do MMO servers handle thousands active connections?

There could still be something like an announcement though right, even within a region.

The "world state" for each user could include a queue of messages. A broadcast would then mean creating the broadcast message, and linking it into the outgoing queue of each user. (Ref counting may help save memory here.) However, if a particular player logs in even one simulation step after the world broadcast is sent, that user will not see the broadcast. Thus, it's much better to sync state (and control information) than it is to sync events, for most cases. (Events can still be useful for some things.)

What I'm trying now is having (edit: as in, I'm playing around with. You may want to check into this as well)

1. everything is represented as a "record" (buildings, characters, spells, chat messages, spell casts, etc)

2. records "attach" connected clients by going through the server's client list and filtering (example: is this client in range of me?) at creation

3. clients notify records that it should be attached (records filter this like above) when a client is created or moves a certain distance

4. records go through their attached-client list checking the last updated time stored in this list and update each time a record is changed

(TLDR: records notify the clients instead of clients polling the records for updates)

(I call my objects that encapsulate server-client stuff "server_session"s and the client side has "client_sessions"s which inherit from "session" and servers and clients themselves are "session_processor"s. Basically I try to reuse as much code as possible between the client and server. Whether or not my code compiles into a client or server is determined by a preprocessor #define :p )

Suppose there is a scenario in which another server sends a command that should be broadcasted to all (or perhaps select) clients. Does this mean that the application code will have to iterate through all the client connections, pick the relevant (or all of them) clients, and broadcast the message?

Yes.

Modern games tend to discourage large groups, but in older games you would encounter situations where thousands of players would all be in one location. Anytime these players started casting spells or anything the game world would freeze for a second or two because the server is stuck in that iteration through all clients sending copies of the same exact game world updates to all of the thousands of players.

This is why MMO's switched over to instancing and made group encounters very small. Basically every instance and zone in an MMO is a separate server executable and typically even a separate machine. Of course the downside is that newer games aren't as fun anymore. They feel more like fantasy death-matches than an MMO.

It comes down to server tiering and instancing, as others have mentioned. Each shard could consist of a handful of top-level world servers, and beneath each of them could be multiple region servers, and beneath each one of those many instance servers, etc. Thousands of players may be playing your game at once, but each server would only have a handful of players of connected at once (or not have to do much work per player). You'd also probably have separate servers for logging in, chat, auctioning, finding instances, persisting player data, etc., that serve very specific purposes. Basically, the work is spread out across many servers that are each designed to do one task really well.

Of course, the game design itself would have to be amendable to this sort of stratification. If you're trying to design a game where thousands of players can battle it out at once ala EVE, you'll need a whole separate bag of tricks!