MegaGlest Forum

Archives (read only) => Glest Advanced Engine => General discussion => Topic started by: daniel.santos on 19 December 2008, 11:46:50

Title: chop, chop, chop (network code)
Post by: daniel.santos on 19 December 2008, 11:46:50
I'm chopping up the networking code now, so I wont have an 0.2.11a.  Instead, I'm going to try to bring a number of things in line that I've wanted to do for some time and it's appearing as though its also the best way to achieve the desired results.  I will post again when I have something to test.  I'll also try to get a post out that outlines the new networking classes architecture for those technically interested.

In short, it will be hybrid client-server/peer-to-peer, support network game pause and speed change (after agreement from all players), and each network connection will have it's own thread to receive data and use IPC to notify other (interested) threads when that data is ready.  Unfortunately, I'll have to have a thread for each network connection (which really isn't all that bad).  It's very technically possible to do this with just one thread (::WaitForMultipleObjects() on windows and select() or pselect() on linux), but that would require me to implement pthread support in all of the Glest threading classes -- actually, that's only if I want t have a thread be able to wait on either I/O or an IPC eventm, so SCRATCH THAT! ... Take Two:  I would have to modify the Socket class to be able to use select() (linux) or WaitForMultipleObjects() (windows) to support that -- hah! I'm glad I wrote this post because I overlooked that (my earlier design was going to use thread conditions for inter-thread signalling).

Either way, one thing is should provide is less lag and a better game synchronization.  Also, the peer-to-peer support is optional, so if two clients can't talk to each other (i.e., firewall or routing issues), but they can both talk to the server, the server will transparently relay events as it does now.  This will also set the stage for supporting the ability for other clients to take over as server should the server drop.

I already did a pretty big re-write of the network messaging layer, and that's going largely unchanged (except to add support for exchanging status data better).

EDIT: Oh yea, and one more thing.  I think this game really needs UDP support.  So if anybody out there is in the mood to examine how the messaging layer works and is good ad UDP-based protocols, please knock yourself out.  I'm actually good at it myself, but it's a time issue.  We would also probably have to modify the NetworkMessage classes to specify an importance, so some packets would need to be delivered, need to be in order and arrive within X time, while others can be out or order and get dropped and it's no big deal (obviously, corrupt is never ok and UDP's checksum is too small to be reliable, so our own checksum would need to be tagged on as well).
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 20 December 2008, 04:07:53
Sounds good. I'm planning on doing  "Internet Protocols And Services" unit in the first semester so I could look into it when I'm finished with the GUI, if you haven't done it yourself by then.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 20 December 2008, 09:01:06
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 20 December 2008, 09:34:25
hah! I said it was a work in progress and I already changed the states.  I realized that it wont ever allow you to be in progress of changing both pause and speed states at the same time, so we don't need them separate.  So here are the new states
Code: [Select]
enum State {
STATE_UNCONNECTED, // not yet connected
STATE_LISTENING, // not yet connected, but listening for connections
STATE_CONNECTED, // established a connection and sent (or sending) handshake
STATE_INTRODUCED, // handshake completed
STATE_READY, // ready to start the game
STATE_LAUNCHING, // starting game
STATE_PLAY, // game started (normal play state)
STATE_PAUSED, // game paused
STATE_QUIT, // quit game requested/initiated
STATE_END, // game terminated

STATE_COUNT
};

enum ParamChange { // T - target frame required or not
PARAM_CHANGE_NONE, // n - no pending state changes
PARAM_CHANGE_REQUESTED, // y - state change requested
PARAM_CHANGE_REQ_ACK, // y - pause request acknowledged
PARAM_CHANGE_REQ_ACCEPTED, // y - pause request accepted
PARAM_CHANGE_REQ_DENIED, // y - pause request denied
PARAM_CHANGE_COMMITED, // n - state change committed (paused, unpaused, speed changed, etc.)

PARAM_CHANGE_COUNT
};
enum GameParam {
GAME_PARAM_NONE,
GAME_PARAM_PAUSED,
GAME_PARAM_UNPAUSED,
GAME_PARAM_SPEED,

GAME_PARAM_COUNT
};So now ParamChange is used to pause, unpause and change speed where GameParam will specify which parameter of the game is being changed (new target speed specified separately).  Also, this doesn't mean that there always has to be approval, for instance, when you start the game you may specify that anybody can pause the game without confirmation, but speed changes require confirmation.  You may also have speed changes only take effect 10 seconds after requested so there's time to warn everybody about the impending change.

If it's not too much work, I may add a mechanism to replace dropped connections with an Ai and also to allow dropped players to re-connect, although it will probably require forcing a pause (at least, it will be easiest to implement that way. :) )  I eventually want it to be something that players in a lobby (maybe spring lobby?) can browse games and there be a game they can join at any time, even though it's already started.  I'm not in a hurry there though.
Title: Re: chop, chop, chop (network code)
Post by: wciow on 20 December 2008, 10:27:50
Argh my brain just melted!  :mrgreen:

Seriously though I'm sure its gonna be awesome!

p.s I'm off to make a smiley of a brain melting.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 20 December 2008, 12:38:42
lol!! Sorry, it was really intended more for hailstone and other C++/OO experienced people who would be interested :)
Title: Re: chop, chop, chop (network code)
Post by: titi on 21 December 2008, 01:14:23
:lol:

Hi,
please explain me the following situation i didn't get how it will work:
We have two clients and one server. client are called C1 and C2 server is called S.
Everyone can reach each other ( so no routing is needed )
If C1 wants to execute a command he sends it directly to S and C2.
This command mus be executed with the next keyframe( or when? ).
But what happens if the command transmission needs too long to reach C2, but it reaches S in time?
The command will be executed on S and C1 but not on C2 in the same key frame.
How does C2 know that he has to wait for a command when he reaches the keyframe?

In the current implementation C2 knows that he has to wait for command from the server because he gets a command(probably empty one) for every keyframe. So C2 knows, he has to wait when he reaches a key frame without getting a command from the server.

How will this work in the new implementation?
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 22 December 2008, 06:26:07
I think this is how it works.

frameUpdateBegin is called to specify which frame is being referred to, messages such as commands are sent, then frameUpdateEnd is called to say that information for a frame has ended.

Is there a distinction made between a game host (the creator of the game) and a network host?
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 22 December 2008, 07:56:23
hurray for titi!  I spent 20 minutes typing a response and then I learned some things from your question.

So first off, hailstone has it pretty close.  Actually, frameUpdateBegin() and frameUpdateEnd() (for the network interface classes) are called before and after the "world" is updated.  The only reason there are two calls is to mark the beginning of the frame (tell the ClientInterface or ServerInterface what the new frame is ) and do any pre-world update calculations and then the frameUpdateEnd() to tell the network layer to send (any) data queued up while updating the world.  I want it to get sent at the end of updating the world so that it will be smaller (less packets means less headers and other overhead, plus if we're compressing the data, you get better compression ratios when compressing larger amounts of data).

But the short answer to titi's question, as it stands now is that I don't care about key frames anymore and I just send commands to everybody as soon as they are issued.  More precisely, C1 would send the new command to both S and C2, S would know not to relay the command to C2 and both would process the command as soon as they received it (no matter which frame it was).  However, I have a better idea now.

Before I go into that, the new class NetworkMessageStatus is used to transmit status changes between clients, servers and peers and is also the base class for many other messages (intro, ready, launch, command list, etc.).  It's one of those bit mangeling classes, so let me just post it's current code (which is sure to change further).  Note that the term "peer" is generic and can refer to any remote relationship (server to client, client to server or client to client)
Code: [Select]
class NetworkMessageStatus : public NetworkMessage {
    enum DataMasks {
        DATA_MASK_SOURCE            = 0x0000000fu,
        DATA_MASK_STATE             = 0x000000f0u,
        DATA_MASK_PARAM_CHANGE      = 0x00000700u,
        DATA_MASK_GAME_PARAM        = 0x00001800u,
        DATA_MASK_GAME_SPEED        = 0x0000e000u,
        DATA_MASK_HAS_FRAME         = 0x00010000u,
        DATA_MASK_HAS_TARGET_FRAME  = 0x00020000u,
        DATA_MASK_FRAME_IS_16_BITS  = 0x00040000u
    };

    uint8 connections;  /** bitmask of peers to whom a connection is established */
    uint32 data;        /** contains various data packed into 32 bits */
    uint32 frame;       /** (optional) the current frame at the time this message was generated */
    uint32 targetFrame; /** (optional) the frame that actions specified in this packet are intended for */

public:
    NetworkMessageStatus(NetworkDataBuffer &buf, NetworkMessageType type = NMT_STATUS);
    NetworkMessageStatus(const Host &host, NetworkMessageType type = NMT_STATUS,
            bool includeFrame = true, GameSpeed speed = GAME_SPEED_NORMAL, uint32 targetFrame = 0);
    virtual ~NetworkMessageStatus();

    uint8 getConnections() const        {return connections;}
    bool isConnected(size_t i) const    {assert(i < GameConstants::maxPlayers); return connections & (1 << i);}
    uint32 getData() const              {return data;}
    uint8 getSource() const             {return static_cast<uint8>       (data & DATA_MASK_SOURCE);}
    State getState() const              {return static_cast<State>      ((data & DATA_MASK_STATE) >> 4);}
    ParamChange getParamChange() const  {return static_cast<ParamChange>((data & DATA_MASK_PARAM_CHANGE) >> 8);}
    GameParam getGameParam() const      {return static_cast<GameParam>  ((data & DATA_MASK_GAME_PARAM) >> 11);}
    GameSpeed getGameSpeed() const      {return static_cast<GameSpeed>  ((data & DATA_MASK_GAME_SPEED) >> 13) ;}
    bool isResumeSaved() const          {return static_cast<bool>        (data & DATA_MASK_IS_RESUME);}
    bool hasFrame() const               {return static_cast<bool>        (data & DATA_MASK_HAS_FRAME);}
    bool hasTargetFrame() const         {return static_cast<bool>        (data & DATA_MASK_HAS_TARGET_FRAME);}
    uint32 getFrame() const             {return frame;}
    uint32 getTargetFrame() const       {return targetFrame;}

    virtual size_t getNetSize() const;
    virtual size_t getMaxNetSize() const;
    virtual void read(NetworkDataBuffer &buf);
    virtual void write(NetworkDataBuffer &buf) const;
   
protected:
    void init(const Host &host);
   
    void setConnection(size_t i, bool value) {
        assert(i < GameConstants::maxPlayers);
        uint8 mask = 1 << i;
        data = value ? data | mask : data & ~mask;
    }

    void setConnection(size_t i) {
        assert(i < GameConstants::maxPlayers);
        data = data | 1 << i;
    }

    void setConnections(bool *values) {
        connections = 0;
        for(size_t i = 0; i < GameConstants::maxPlayers; ++i) {
            if(values[i]) {
                data = data | 1 << i;
            }
        }
    }

    void setSource(uint8 value) {
        assert(value < GameConstants::maxPlayers);
        data = (data & ~DATA_MASK_SOURCE) | value;
    }

    void setState(State value) {
        assert(value < STATE_COUNT);
        data = (data & ~DATA_MASK_STATE) | (value << 4);
    }

    void setParamChange(ParamChange value) {
        assert(value < PARAM_CHANGE_COUNT);
        data = (data & ~DATA_MASK_PARAM_CHANGE) | (value << 8);
    }

    void setGameParam(GameParam value) {
        assert(value < GAME_PARAM_COUNT);
        data = (data & ~DATA_MASK_GAME_PARAM) | (value << 11);
    }

    void setGameSpeed(GameSpeed value) {
        assert(value < GAME_SPEED_COUNT);
        data = (data & ~DATA_MASK_GAME_SPEED) | (value << 13);
    }

    void setResumeSaved(bool value) {
        data = value ? data | DATA_MASK_IS_RESUME : data & ~DATA_MASK_IS_RESUME;
    }
   
    void setFrame(uint32 frame) {
        data = data | DATA_MASK_HAS_FRAME;
        this->frame = frame;
    }

    void setTargetFrame(uint32 targetFrame) {
        data = data | DATA_MASK_HAS_TARGET_FRAME;
        this->targetFrame = targetFrame;
    }
};
So this class conveys a lot of status information and packs it down into a maxiumum of 13 bytes.  Using other techniques, I can get this thing down to 5 bytes max, but this is good enough for now.  The connections byte specifies a bit mask of each peer that the originator of this message is connected to.  The data section uses 4 bits to specify the originator.  This isn't usually necessary because that information can be learned from the socket it comes in on its self, but it may be relayed from the server, so that way a client can know the update of the status of one of it's peers that it may not be able to communicate directly with.  But most importantly, there is a frame and targetFrame.

The targetFrame was originally intended to coordinate pause and speed change requests so that it could all happen at the same time on every participant in the game.  It occurred to me that a game could be kept better in sync if commands were not executed immediately on the local machine.  Instead, they could be given a target frame in the future and queued up locally and on the peers so that the command will (ideally) be executed at the same time on every machine.  By having this behavior on the server as well, it should cancel out the "home field advantage" (i.e., being the server wont have any advantage).

This does not go for auto-commands by the way, these are always executed locally and are not transmitted at all because it's presumed that (as long as the data is the same on each machine) they can each figure these out on their own.  Perhaps it would be helpful to transmit these however so the server can verify that an auto-command being executed on a client is accurate and be able to correct a client if needed.  As an example, if a client thinks his unit is close enough to see an enemy unit and attack it but on the server, they are one cell off (not an unlikely condition), the server can send correcting information to update both the unit that attempted to execute the auto command and the unit it thought it saw.  It may look funny because the unit would start to attack but then warp back though, but that hopefully shouldn't take long (maybe 400 milliseconds).

As a side note on the NetworkMessageStatus class, it can still be trimmed down a lot.  I'm not using the DATA_MASK_FRAME_IS_16_BITS field and I can also cram part of the frame bits into the unused portion of the data field and only use extra bytes for frame.  Also, I can have one message that sends the full frame as a 32 bit number but is only sent when the top 24 bits change.  The rest of the time, it can transmit only the lower 8 bits of the frame number.  Perhaps even better, I can use part of the data to specify how many bytes of the frame and target frame I'm sending and the upper portion will be recycled from the previous update.  I wont mess with that until after all of these other issues are resolved, but there's a lot that can be done in very few bits.

Also, one thing that is changing is the way clients wait for a laggy server.  At present, if a client hasn't received an update from the server for a key frame, the client stops rendering, updating sound, etc., it essentially suspends the main thread until that is received or a max wait timeout has expired.  Now, it will behave as though it's paused, so rendering continues, the mouse cursor continues to move around, sounds don't stop and you can still issue commands, etc.  You may have never experienced this before unless you play a game where the server is very slow (like when you compile a debug build and you're debugging it with millions of sanity checks running a second :) ), but it's a pain to deal with.  I have some other ideas for ways to address this that I'll worry about once this networking rework is done.
Title: Re: chop, chop, chop (network code)
Post by: titi on 22 December 2008, 21:49:00
ok, I think i got it now but I'm not really shure if it will work.
Every client plays its own local game, influenced by commands and updates it gets from outside. You are no longer trying to keep the games of the clients in sync.
I think this will cause jumping units with nearly every update. Even after an update command which was sent from the server to the clients the currently units will not be in sync, because the update commands delay( network ) is different for every client !
By this you will play a whole game out of sync with nearly every unit, only hold together by server updates.
A typical problem this will cause:
You see one of your units next to a line of trees in the west. Now you give a command to walk to the west and your plan is to have your units noth of this line of trees. But when your command is executed on the server the unit is a bit more south of the tree line, so it will walk south of the tree line. Now if the updates for this unit arrives your client, suddenly you see your unit in the south of the tree line, not in the north as you want it. this will be something very annoying for players, because their commands are not executed the way they wanted/planned it.

But I must admit that this asyncronous implementation is very very complicated to understand( and to debug!!) and I probably didn't get it right now.
I must say that I would still prefer an improved version of the original way it was done.
Title: IDEA!
Post by: titi on 26 December 2008, 10:17:40
IDEA!!!

What about this:

- Commands are bound to key frames.
- Clients don't wait for commands which doesn't reach them
- All Clients(including the server) have a rollback system, so they can rollback and execute commands in the past which reach them too late.

More excatly:
Before executing the commands in a keyframe you copy all current unit states into a list.
Now you store the commands that will be executed with this unit list.
Unit list and command list are your state for this key Frame.
After storing the state you execute the commands of this state.
This is done for every keyframe and let's say the last 10 keyframes are stored this way.

What happens if some commands reaches the client which were meant for an old keyframe?
Add the old commands to the command lists of the keyframes for which they where meant. .
Now restore the keyframe state of the oldest command and playback the whole game up to the current frame.
Now everything should be fine again.

If you are reached by a command which is meant for a keyFrame which is no longer in the rollback list, the game is really out of sync and you can stop with an error message or something else ( full update from server for example ).

By this you will get no more "waiting for server" lag and you can play with very bad conections without getting too much trouble.
BUT: This will probably very cpu/memory intensive ?

A problem that will be there are the update commands which are send by the server. These update commands are based on the servers state, which must not be correct because he doesn't has all information yet if you don't run a full server client concept. So I would prefer this client server concept because the server is always up to date.

What do you think of this?
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 29 December 2008, 02:01:03
I like the rollback idea. Would need checkpoints, like you said, so it doesn't redo the whole lot. It might have performance problems - think of how many transactions go into a database for a bank which would use such a system.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 29 December 2008, 10:13:44
titi, thanks for the input.  The idea of being able to rollback a command sounds good in some ways, but it also sounds very tedious to code and can have fairly significant CPU overhead (probably not much memory overhead though).  I say "can" have because commands aren't sent all that often, so it shouldn't be too much of a load.  None the less, it will still be a LOT of code to implement.

I think a better idea may be to use the "targetFrame" but have the lead time on commands vary according to the max lag being experienced in the game.  That way, if a server gets a command late, it can (maybe) fast-forward the command and send updates to peers, but re-adjust the min lead time for the targetFrame of newly issued commands.

As far as key frames, I'm completely eliminating them except for using them to force clients to wait for the server and visa-versa when a client gets too far behind (maybe 1 second or so).

So right now, every command issued will be given a targetFrame, even locally. So if I'm a client on frame 421 and I want to tell my guys to walk somewhere, I send the command and queue it up locally to occur on frame 431 (for instance), a 10 frame lead time (or 250 mS).  Since I'm also sending to peers, they should get it in time as well.  Then, when I reach frame 431, it comes out of the command queue.

I think that most of the units jumping around that you've seen is due to the problem with the server not sending its commands!! :O  I'm also going to change updates so that instead of units instantly appearing at their new location when an update is received, they will warp there over a period of 200-ish milliseconds so it's a little easier to see what's happening.  Alternately, I may figure out a way to cause them to move to their updated location more casually (very fast walking for instance).

Either way, I have a lot of re-assembling (of the code) to do.  Adding a networking thread will actually improve many characteristics of network play because the thread is only there to deal with net communications and I've done a LOT of multithreaded programming in my life (13-ish years?)  Also, I just added a condition/event class to make synchronization more fun in addition to a wait() function in Socket that allows you specify multiple sockets to wait on -- that way you can use non-blocking socket calls, but keep your thread asleep until there's really something to do.  This reduces CPU while allowing the most optimal response times to I/O (and only requires one extra thread).  In addition, when the network thread receives data, it notifies the main thread so no more polling is needed (GameInteface::onReceive()).

I'll post an updated class diagram when I get home.
Title: Re: chop, chop, chop (network code)
Post by: titi on 29 December 2008, 22:12:02
Is it really so hard to implement this rollback system?

To create a "checkpoint" you ?simply? have to clone the list of units(deep clone).
and add the list of commands which are sent by the server (or do you need a clone of them too? ) .
Thats all you need to have a checkpoint.

Now for a rollback:
1. remove all units from the current unit list.
2. add all units from the checkpoints list
now you should have the state of the old checkpoint.

The thing I don't really know is how to fast forward gamesplay  back to the current frame.
Isn't there something like this "fast forward" already implemented in glest?
Or is this the point where the things get complicated?
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 1 January 2009, 12:29:55
Some interesting articles on multiplayer programming:
http://www.gamasutra.com/view/feature/3 ... g_for_.php (http://www.gamasutra.com/view/feature/3230/dead_reckoning_latency_hiding_for_.php)
http://www.gamasutra.com/view/feature/3 ... r_math.php (http://www.gamasutra.com/view/feature/3221/multiplayer_math.php)

@titi:
It seems like you are talking about period saving with loading latest save when out of sync rather than a transaction style rollback system.
Title: Re: chop, chop, chop (network code)
Post by: titi on 1 January 2009, 19:10:20
I will think about the articles, probabaly there is something useful in them I will study them...

For my idea:
I'm not talking about transactions based on every command like its possible in databases for example. In my idea the checkpoints to which you can rollback are made by hand ( and there is more than one checkpoint, I talked about 10 of them! ).
If you say "begin Transaction" in a database system a "checkpoint" is made to which you can return with a rollback. In a database system it's in fact not a copy of the whole state, but it has the same effect. Most database systems log all changes made until a commit is made, but this is useless for glest, because you cannot log all changes(movements of units and so on).

But I'm definitly NOT talking about periodical savings. I talked about some kind of rollbacks to execute commands which didn't reach you in time.
This is especially interisting when you use UDP instead of TCP for communication which doesn't guarantee that messages arrive.
Title: Re: chop, chop, chop (network code)
Post by: Ayrin Greenflag on 6 January 2009, 11:42:27
don't know if it could be useful ... i remember the old diablo network game that has some kind of jump coz of lag (56k connection) you was walking and suddenly a monster appairs and hit you...or you suddenly jump back coz you was hitten...i think they used a kind of sync check between server and clients if it doesn't match server information overhead clients ones.
Pretty annoying with lag but worked pretty good in other cases.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 7 January 2009, 10:39:52
I don't think I'll be pursuing a transactional mechanism at this time, although I do intend to enhance the "fast forward" mechanism.  Currently, when a command packet arrives late or an update arrives, the unit's state can be fast forwarded, but only to the point of finishing the current cycle of the skill they are on (or that is specified in the update).  I'll be expanding this so that it can complete an iteration of a skill.  But for now, I believe that adding a command delay should be sufficient to solve most of these issues.  The initial value will be 250 milliseconds (10 world frames), but the server will adjust this as needed based upon ping times, I'm figuring making it the average ping time plus 2 world frames (50 milliseconds).  This has the added benefit of removing some of the advantage that the player on the server would otherwise have, because everybody's commands will get executed at the same time (in theory).

If a command does arrive late, I can issue the command to the unit and then tell it to fast forward execution by however late it was and that will solve most of the problems associated with late arrival.  It will not solve problems of the units being located in the wrong cell however (for instance, if they were walking and you issued a stop command that arrived too late).  None the less, when a late condition is detected, an update can also be sent from the server (or requested by a client) for the effected units.

I'm home now, so I'm back on Linux, hurray!  I'll redo the UML class diagram with the current design tomorrow or friday and post it for your viewing pleasure.  But it's coming along, but it's a lot of rework (I'm mostly finished at this point).
Title: Re: chop, chop, chop (network code)
Post by: titi on 7 January 2009, 16:00:43
Adjustable keyframe times based on server pings are a good idea!
I think this will make the game more stable and I can play with someone from america ( pings from 300ms up to 600ms ).
I would not lower the keyframes under 250 ms. I think 250 ms is the current value and it works quite OK.

I hope it will work, but I see lots of problems with your asyncronous way of playing the game. I fear it will result in an update orgy after a while.
But probably I'm wrong, I will help you where I can (tests ... :) ).
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 29 January 2009, 02:24:48
Hey everybody, sorry I haven't been on the forum much lately, I've been busy with other stuff and probably will continue to be so for another week.  None the less, I have been doing some work on the networking code and I decided it was appropriate to do a reassessment of the requirements and design.  Upon doing so, I discovered a lot of things that will further change the design of GAE, but I haven't decided how much of this should go into 0.2.12 yet so I present it here.  Hailstone, any feedback would be appreciated.  I think I have a pretty clear path forward for 0.2.12, taking the middle road.

Below are the requirements and the portion I'm thinking should go in 0.2.12.  Perhaps at some point (very soon) I should write a formal word document and keep it in source control, but this is good enough for now.  This may read as verbose to some, but this is the nature of a requirements/specification text: it must be verbose to eliminate ambiguity.  Ambiguity kills.  I've done software engineering, consulting, business, software and systems analysis for over a decade, trust me on this, I know what I'm talking about.  The version in green is only my current proposed target version.  It's once thing to discuss ideas, but when spelling out requirements prior to building an application, completeness is important.  Imagine trying to build a building with incomplete blue prints.  Anyway, here they are (I'll likely revise them, so look for a "LAST UPDATED") comment at the end of this post.
.

Definitions
First some definitions for clarity's sake:
.

General Networking and Lobby
.
 
Disconnects, Reconnects and New Game-Time Connections
The following items target 0.2.12 specifically:
The following overrides the 0.2.12 specifications above and target a release >0.2.12 (maybe even the 0.3 branch for some requirements).
.
 
Game settings, speed and pause
.

I've reached the maximum message size for this forum, so I'll resume this in the next post...

EDIT: 2009-01-28: Added weather sync requirement
EDIT: 2009-03-05: Made modifications (in blue) consistient with hailstone's feedback here (https://forum.megaglest.org/index.php?topic=4015.msg22077#msg22077).  Outstanding issues:
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 29 January 2009, 02:26:02
World updates and rendering
Some of these requirements have been mentioned in other places and some are not network-related, but important enough to mention here since they are closely tied to the way game play will function.
.
 
Network game play and World Updates
.

I still want to cover the new messaging, so I'll do that with yet another post.  I'll probably draw up a formal sequence diagram, because I think that works well.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 29 January 2009, 05:13:20
grr, I can't get either Argouml nor Umbrello to make descent sequence diagrams for me so I'll spell it out with text  >:(

So each message will have a source and destination (just like a real sequence diagram).  For brievity, I'll use S for Server, C1 for Client1 and C2 for Client2.  This diagram (psudo-diagram) illustrates a network conversation for a 3- player network game.  Note that sequence IDs with alpha suffix attempts to denote async operations.  When this sequence completes, a new game will have started.

New Network Game Sequence Diagram (network communications)
Code: [Select]
1   S             : listen
2   S <- C1       : connect
3   S -> C1       : accept
4   S -> C1       : handshake       Game::Net::NetworkMessageHandshake (formerly
                                    NetworkMessageIntro) contains version info (of server software)
                                    and the id and uid the server is assigning to the new client.
5   S <- C1       : handshake       Client responds sending it's version, and echoing the newly
                                    assigned id and uid.
6   S <- C1       : player info     Sends a NetworkMessagePlayerInfo object which encapsulates a
                                    Game::PlayerInfo object and status (NetworkPlayerStatus) info.
                                    PlayerInfo contains player name, relevant config preferences and
                                    network info (Game::Net::NetworkInfo) from the perspective of
                                    the remote client.  Thus, their percieved IP address may be a
                                    VPN address. The PlayerInfo object's networkInfo.localHostName
                                    will contain the host name as the remote host percieves it.
7   S -> C1       : game info       NetworkMessageGameInfo contains GameSettings and
                                    Game::PlayerInfo + status for all players.
8        C1       : listen          C1 is now considered to be fully connected and will now start
                                    listening for peer connections
9   S <------- C2 : connect
10  S -------> C2 : accept
11  S -------> C2 : handshake
12  S <------- C2 : handshake
13  S <------- C2 : player info
14a S -> C1       : game info
14b S -------> C2 : game info
15             C2 : listen
16       C1 <- C2 : connect         Onus is on the newly connected client to attempt to contact peers
17       C1 -> C2 : accept
18       C1 -> C2 : handshake       UID of C1 sent to C2 along with handshake
19       C1 <- C2 : handshake       UID of C2 sent to C1 as well
20a S <- C1       : status update   clients must report that they have established a connection to
20b S <------- C2 : status update   each other
21a S -> C1       : game info       from this point forward, the server will no longer relay
21b S -------> C2 : game info       commands from C1 to C2 or visa-versa
22  S             : host uses UI to change game settings
23a S -> C1       : game info
23b S -------> C2 : game info
24  S             : host chooses "launch game" from UI
25a S -> C1       : launch
25b S -------> C2 : launch
26a S             : loads           Loads all game data (map, tileset and faction tree) and builds a
26b      C1       : loads           Shared::Util::Checksums object
26c            C2 : loads
27a S             : wait until all clients are ready
28a S <- C1       : report ready    Game::Net::NetworkMessageReady sent to server with Checksums
28b S <------- C2 : report ready    object.  If it doesn't match the server, then the server pukes
                                    on them.
29a S -> C1       : begin game at x time
29b S -------> C2 : begin game at x time
30a S             : starts game
30b      C1       : starts game
30c            C2 : starts gameNote that at various points in this sequence, alternate routes may occur for items 3 and 10 if a client is banned by the server (by IP address) and also for 18 and 19 if the UID, IP address and player name provided by each client does not match the values sent by the server in the game info message. 

Saved Games
When restoring a saved network game, the saved game file is sent from the server to all clients at some point in this sequence prior to step 25.  Probably, it should not be sent with the GameInfo necessarily.  Perhaps there should be a delay of about 15 seconds between the time the server selects the saved game file and it decides to transmit it to clients.  The reason for this is that we don't want to send every saved game file as they scroll through them, perhaps trying to find the appropriate saved game to resume.  At every change, a new NetworkMessageGameInfo should be sent immediately however (or within 1000 milliseconds) so that each client has an updated view of what they are about to play.  This message (without the entire saved game) is much smaller and can be sent more frequently.  Of course, if less than 15 seconds have passed since the host/server selected the saved game to restore and they hit the launch button, it should force the saved game file to be transmitted immediately.

EDIT: Correction in sequence (there was also two step 5s)
EDIT: 2009-03-26: Updated sequence, changed numbering (fixed numbering problems), added more details in sequence, added information for how saved games should be handled.
EDIT: 2009-04-14: Moved assignment of player ID and uid from the game info message to the handshake and clarified how checksums are managed.
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 31 January 2009, 03:01:26
Looks good.

OT:
Quote
but this is the nature of a requirements/specification text
Apparently this is a specific language. I read this about the C++ Standard by James Kanze at http://groups.google.com/group/comp.lang.c++/browse_thread/thread/28b14a1308974070:
"The standard is written in a variant of English sometimes called standardese. It's designed (or at least intended) to be
absolutely precise and unambiguous, even at the cost of understandability. It doesn't always succeed with its intent,
but in all cases, precision and a lack of ambiguity have precedence over readability."

Anyway back on topic now.

Quote
[0.2.12]: Clients should be able to connect when server is in lobby but there's not an open slot set to "network" (i.e., be in limbo for a time while host changes settings)

Will the server want the first people in to automatically be assigned to a slot? and it doesn't explain what happens when there are two people in limbo and a slot is opened.

I propose that a que (with a max amount of items) be created where players are sent to when they join. The server can then add or remove the desired network or CPU players to or from a slot (the position of which is moveable by the server). Empty slots should be visual and numerical.

Perhaps before clients connect server can adjust settings then press a listen button to allow clients to enter the que (something useful for a master server so games don't appear immediately).

How will specators be dealt with?

Quote
# GsAutoRepairEnabled - rather or not auto-repair is allowed
# GsAutoReturnEnabled - rather or not units will automatically return from auto-commands like auto-attack, auto-repair, etc.

I don't know if I agree with GsAutoRepairEnabled and GsAutoReturnEnabled being forced on clients. It seems like personal preference to me.

Also here are some things that you didn't mention: syncing tech trees or downloading maps, password protected joining.
Title: Re: chop, chop, chop (network code)
Post by: Omega on 31 January 2009, 10:58:43
By syncing, do you mean the ability to play on a map or tileset that one player does not have? If so, that is a must. Not sure how that'll cope with a tileset though, seeing the size of that.

Why can't we have each player choose his or her own tileset? The as long as the walkable/unwalkable parts are the same, it shouldn't change gameplay. (most tilesets have the same items as walkable/unwalkable anyway).

Lookin' good. I never could understand network code. That's the one thing worse than AI. (AI isn't that bad actually, unless we are talking about RTS AI, which I imagine is the hardest).
Title: Re: chop, chop, chop (network code)
Post by: titi on 17 February 2009, 10:02:58
Do you have any news for us here? Is there any progress or trouble?

One little wish (if you are still on it), please keep in mind that we once want a master server to setup/find games.
We need no implementations yet, just keep it in mind when you design protocols.
Title: Re: chop, chop, chop (network code)
Post by: Omega on 17 February 2009, 12:04:52
A master server... <3  *sigh*

heaven (on earth) ;D
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 24 February 2009, 05:16:33
Quote from: titi on 17 February 2009, 10:02:58
Do you have any news for us here? Is there any progress or trouble?

One little wish (if you are still on it), please keep in mind that we once want a master server to setup/find games.
We need no implementations yet, just keep it in mind when you design protocols.

Hey buddy, sorry for the late response!!

Yea, lots of progress, but lots of work.  I've also been fiddling with a few other projects, Lincity-NG, KDevelop and gdb recently.  I might try to start working on gdb's Project Archer (http://sourceware.org/gdb/wiki/ProjectArcher) and if I like it, I might try to get a paying job working on gdb even!  I've also have had a lot of other work to take care of over the past few months, but progress indeed is being made.  No real problems.  I posted a minor enum quandary recently, but nothing I'm worried about, I'm using the copy & paste mechanism for now because it's simple.

OT: On the topic of languages and enums (but off of this topic :) ), I'm not even happy with Java 5's enums :(  But that's ok, it's better (in most ways) than how we did enums before in Java (and I still have projects that used old-school Java enums -- basically it enforces it's behavior via private & protected constructors & such).  In fact, I have a Java project that uses polymorphic Java enums, that one's a trip!  It takes a lot of rules to enforce it, but they have come out cleanly thus far and work great over network (RMI over IIOP) and other serialization situations -- compact on the wire, yet safe for ensuring the same bytecode is using it on each end.  I rarely use the default Java object serialization across the wire, because I think it's too fat (even compressed).  All of this while handing a variable inheritance tree is tricky, but it served the specific application very nicely.  Oh yea, they were also annotated for use in Java 5 Persistence (via Hibernate) and that's where they were mostly used.  But, I'm WAAAY off topic! (sorry guys! :) )

So since I'm doing the networking from the ground up, I'm just trying to make sure the quality is top notch.  I've refactored and reexamined the design a number of times now (sometimes on paper, UML or text/word doc, other times in code).  I'm getting happier with the design, I just want to make sure it's nice and clean. :)  If I find a UML editor (preferably one that at least does descent C++ reverse engineering) I'll try to spit out another class diagram and maybe convert the text I posted earlier into a real visual sequence diagram (although I think the text at least serves it's purpose).  I want to make sequence diagrams for each major networking operations during game play as well (what I covered is just in the lobby).  Most of this is already in place, it's just getting altered.

Also, I've added this Printable C++ class (an abstract interface) and an ObjectPrinter class to print Printable objects and I quite like it.  Having this in place will make debugging the networking code very nice.  Example output:
Code: [Select]
361750: sent to peer[0]: NetworkMessagePing {
  NetworkMessage {
    size = 22
    type = NMT_PING
    writing = false
    simRxTime = 292632639351
  }
  id = 213
  time = 292632534356
  timeRcvd = 0
  pong = false
}

The idea here is to cut out as many possible ways to be confounded as reasonably possible.  Silly bugs can be hard to trace down once they go across the wire, is the problem on the client or the server?  Well with all messages logged, it's easier to tell.

Quote from: hailstone on 31 January 2009, 03:01:26
Looks good.
Thanks!

Quote from: hailstone on 31 January 2009, 03:01:26
OT:
Quote
but this is the nature of a requirements/specification text
Apparently this is a specific language. I read this about the C++ Standard by James Kanze at http://groups.google.com/group/comp.lang.c++/browse_thread/thread/28b14a1308974070:
"The standard is written in a variant of English sometimes called standardese. It's designed (or at least intended) to be
absolutely precise and unambiguous, even at the cost of understandability. It doesn't always succeed with its intent,
but in all cases, precision and a lack of ambiguity have precedence over readability."
Hah! no doubt, it definitely begins to become a language of it's own.  But if I don't specify things like this, then people die.  Well.... not actually at the moment, but if I were working on code for life support systems, heavy machinery, etc., people could die from ambiguity.  Certainly projects, careers and businesses literally die, right and left as a result of these types of mistakes (it's scary!).  But I would rather be on a low-paying project with a high chance of success than a six figure project that's doomed (I've been there, it's scary as hell and not worth the pay!).

So anyway...
Quote from: hailstone on 31 January 2009, 03:01:26
Quote
[0.2.12]: Clients should be able to connect when server is in lobby but there's not an open slot set to "network" (i.e., be in limbo for a time while host changes settings)

Will the server want the first people in to automatically be assigned to a slot? and it doesn't explain what happens when there are two people in limbo and a slot is opened.

I propose that a que (with a max amount of items) be created where players are sent to when they join. The server can then add or remove the desired network or CPU players to or from a slot (the position of which is moveable by the server). Empty slots should be visual and numerical.

Perhaps before clients connect server can adjust settings then press a listen button to allow clients to enter the que (something useful for a master server so games don't appear immediately).

How will specators be dealt with?
Spectators! What a wonderful idea! (why didn't I think of that? :)  So I think that sounds like a lovely idea and very apropos.  So perhaps when a client initially connects, they automatically go into the observer pool as a substitute for "limbo".  The person on the server then has to select who they want in which slot.  Alternately, people can be automatically sucked out of the observer pool when the server sets a slot to be network controlled and kicked back to the observer pool when they close it or set it to AI, plus giving them the option to manually move them around.  I dunno how to best do that in the UI at this point, but I know how to best implement it in the supporting classes that the UI will interact with.

However, spectators will require a new enum entry for the NetworkRole enum (I've already added them :) and some other various code here and there to manage it.  Because I want to allow two players to control the same faction (optionally of course) as well as allow players to control more than one faction on their team, this presents its self at a good time.  If coded correctly, it can actually simplify things more than complicate it.  Each player basically gets a list of factions and teams they can control as well as factions and teams they can observe.  This is important for good AI when scenarios involving compound teams to best simulate the story line (i.e., having two AIs, each configured differently, both controlling the same faction can produce some interesting, if not confusing outcomes).

Quote from: hailstone on 31 January 2009, 03:01:26
Quote
# GsAutoRepairEnabled - rather or not auto-repair is allowed
# GsAutoReturnEnabled - rather or not units will automatically return from auto-commands like auto-attack, auto-repair, etc.

I don't know if I agree with GsAutoRepairEnabled and GsAutoReturnEnabled being forced on clients. It seems like personal preference to me.

Also here are some things that you didn't mention: syncing tech trees or downloading maps, password protected joining.
hmm... Some may see these as a bit of cheating because the computer is doing something for you, so I feel that it's probably important to allow the server to define rather or not they can be enabled in the game rules.  I hadn't considered that in a network game, the host allows them, but one of the players may not wish to use them, so I agree, there should be a way to disable it for the client that doesn't want it.  So perhaps it's only enabled if it's enabled in the local game settings and also the network GameSettings?  We can worry about a pretty way to present that in the UI later (as long as the actual rules make sense for now).

Thanks for your feedback hailstone!

Quote from: omega on 31 January 2009, 10:58:43
By syncing, do you mean the ability to play on a map or tileset that one player does not have? If so, that is a must. Not sure how that'll cope with a tileset though, seeing the size of that.
No, I'm referring to the synchronization of units in game.  I'm not opposed to a feature that allows you to transfer content files while in the game lobby, there is already full support for transferring files in the code (that's now network games are resumed, the server sends each client the saved game).  However, I just want a nice well-thought out paradigm.  My primary concern is that whatever functionality that allows people to share content needs to be absolutely safe from exploitation.  That's the only reason I don't quickly throw in a few lines of code to allow you to sync tile sets, maps and tech trees with others from inside the game (I want it done right basically).  My real preference is to have a system similar to Wesnoth where there are update servers and you can download mods, tilesets, etc. from there.  This gives some ability for files to be monitored and checked for exploits should any surface.  Then, if two people wanted to play but one player didn't have the files, the client could sent the file data (summary) to the other client and they could then retrieve it from the main server (from in game).  All of this is functionality I don't plan on personally pursuing very soon.

Quote from: omega on 31 January 2009, 10:58:43
Why can't we have each player choose his or her own tileset? The as long as the walkable/unwalkable parts are the same, it shouldn't change gameplay. (most tilesets have the same items as walkable/unwalkable anyway).
Currently, it can be raining on one network game and clear in another.  That's a bug (in bug database) and will be fixed soon in >=0.2.12.  Because tilesets define weather and can potentially effect visibility, I'm not sure that it would be appro0priate to allow different tile sets.  Either way, that would also require hacking up the game further and I'm not sure I'm in favor of that particular idea.
Quote from: omega on 31 January 2009, 10:58:43
Lookin' good. I never could understand network code. That's the one thing worse than AI. (AI isn't that bad actually, unless we are talking about RTS AI, which I imagine is the hardest).
Thanks!  OT: No, RTS AI isn't the hardest at all!  RPG AI (of the type we haven't seen yet) is the hardest IMHO.  I started designing a personality engine to simulate real interactions with NPCs a few years ago just to see how closely we could make a video game character behave like a real living organism.  I came up with some interesting ideas, but decided that the end result (after a lot of work) probably wouldn't benefit man kind because it could become too engrossing and video games (especially MMORPGS) are already far too immersive as it is!  I had the help of a friend who's studying psychology and already has a degree in biochemistry.  This was valuable because the first level of the design was the simulating the reptilian brain (you can't simulate a personality until you understand the layers and instincts that make up a personality, animal, human, imaginary species, or otherwise).  You have to run a simulation for each new character based upon events in their lives that form their persona, coupled with their genetics, environment and social interactions.  Once you do that, you can use the persona data to have an RPG character that interacts with you in a fashion based upon their (simulated) life experiences.  Further yet, memory continues to accumulate and the process never stops, so the next time you interact with them, they will respond according to how previous interactions went.  This was originally an extension of Everquest's crappy "faction system" when I was doing some RPG hacking long ago. -- But again, WAAAAYY off topic!

Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 6 March 2009, 01:33:00
hailstone, sorry for taking so long to address the issues you bought up.  I've updated the specification addressing your issues except for the last 3, tech tree synching, sharing maps and password-protected games.  I've gone through a number of designs and re-designs and I'm feeling pretty happy with what I have now.  I'm going to get everything working up to the exchange of player info & game info between the client and server as that will give me enough to really validate my design in practice and then I'll be ready to have you join in.  I regret that I didn't have this ready for you sooner, but such is life.  I think that I've been able to simplify things enough and separate out the various concepts like the aught to be.

The main differences is that each RemoteInterface object now performs most of the functions involved with handling the messages it receives.  It performs the processing it needs to do and then calls an onReceive() function of the GameInterface derived class (which are ClientInterface and ServerInterface), where again, the GameInterface base class handles what is generic enough and then passes control to the derived class in a sort of Chain of Responsibility pattern (but not truly so).  I'm using the NVI Idiom (Non-Virtual Interface) in the GameInterface to perform what functionality is common and then pass control to the deriving class via a private virtual function (which surprisingly enough, can be overridden by a deriving class).  This is actually a new technique that I recently learned to help clean up a variety issues that I was having otherwise.

So here's a brief text-based class diagram for summary:
Code: [Select]
                             Host
                               |
                 +--------------------------------+
                 |                                |
                 V                                V
           GameInterface                    RemoteInterface
                 |                                |
            +----+----+                +----------+-------+
            V         V                |          |       |
ClientInterface    ServerInterface     V          |       |
                            RemotePeerInterface   V       |
                                    RemoteClientInterface V
                                             RemoteServerInterface
RemoteInterface no longer uses multiple inheritance to derive from NetworkStatus (and I've renamed NetworkStatus to NetworkStatistics).  Instead, it has a "NetworkStatistics stats" object as a data member.  As before, each RemoteInterface object has a "GameInterface &owner" data member named which refers it back to whomever "owns" it (so the RemoteInterface class hierarchy is analogous to the previous ConnectionSlot class) and the ClientInterface class has a "RemoteServerInterface server" data member since it is always associated with a single server.  RemoteInterface objects are not necessarily connected by virtue of their existence any more (as the ConnectionSlot class used to function).  There is also still just one network thread (instead of one for each connection as in a much earlier design) and it calls GameInterface::execute() which contains the main network thread loop.  Since I have implemented a Socket::wait function that will wait on multiple sockets, this loop is pretty CPU-friendly now, so there's no polling anymore, it pretty much only runs when there's really something to do.  I also managed to get the DEBUG_NETWORK_DELAY functionality cleanly woven into the design (finally!  I rather agonized over this part until I found the right approach).  So now I've gotten the number of #if blocks for this reduced to a bare minimum and nicely contained in just a few places (in NetworkMessage and RemoteInterface classes only now).

So that's the basic run down on what's changed and I hope to have it ready for you to start working on some of it soon!

Title: Re: chop, chop, chop (network code)
Post by: hailstone on 6 March 2009, 09:29:10
I had a read of the NVI idiom. It looks really good.

Quote
I hope to have it ready for you to start working on some of it soon!
That's good. I have a couple of issues though. I see interface in most of the names, is this just for naming or is it an actual interface class? I assume it's not an interface class since it is being instantiated.

Quote
RemoteInterface objects are not necessarily connected by virtue of their existence any more
Does this mean that by creating a game it won't listen for connections without further action?
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 6 March 2009, 11:53:07
Quote from: hailstone on  6 March 2009, 09:29:10
I had a read of the NVI idiom. It looks really good.
Yea, it makes sense where there is stuff that should always be done, but you want to allow the core of some function to be overriden.  I'm glad I've been learning a lot of new things over the last several months. (I picked that one up from Scott Meyer's book as well)

Quote from: hailstone on  6 March 2009, 09:29:10
... I have a couple of issues though. I see interface in most of the names, is this just for naming or is it an actual interface class? I assume it's not an interface class since it is being instantiated.
Yea, I agree that it's a bit confusing and I'm certain open to alternate naming!  Truth be told, I really like Java's solution to multiplie inheritence, that there are the concept of interfaces which are separate from the concept of a class and that you can only derived from one class but that you may implement as many interfaces as you like.  I'm not using any of the nomenclature/naming-conventions that distinguishes classes from interfaces (like m$'s CMyClass and IMyInterface style), so in the networking code, "Interface" is referring to a logical "network game interface".  Also, "GameInterface" could also be named "LocalInterface".  GameInterface and it's derivitives don't actually encapsulate a socket where data is transmitted or received, only a server socket that is capible of accepting new connections.  The RemoteInterface classes do encapsulate sockets that send and receive data.  Each RemoteInterface object is dedicated to interfacing with a single remote host.

Quote from: hailstone on  6 March 2009, 09:29:10
Quote
RemoteInterface objects are not necessarily connected by virtue of their existence any more
Does this mean that by creating a game it won't listen for connections without further action?
No, it's just an internal thing because when the ClientInterface object is created it has a "RemoteServerInterface server" object that's a data member.  Thus, the RemoteServerInterface object is alive, but is not actually connected to anything.  The client will create and destroy RemotePeerInterface objects as needed, as will the ServerInterface create & destroy RemoteClientInterface objects as needed.  But the ClientInterface always has a RemoteServerInterface object that's valid, but not necessarily connected to anything.  After you look at it, if you think it's a bad design, please say so.

For instance, the ClientInterface object could change it's "server" data member to type "RemoteServerInterface *" that's NULL whenever there is no connection attempt in progress.  However, all of the RemoteInterface classes will exist through many states which aren't necessarily usable (i.e., STATE_CONNTECTING, STATE_NEGOTIATED, etc.), so care must be taken throughout that just because an object exists, that it doesn't mean that we have a valid connection to a remote host. 

Also, after examining it further, I'm wrong.  The old ConnectionSlot class did previously have instances before they were actually connected to anything.
Title: Re: chop, chop, chop (network code)
Post by: ZaggyDad on 6 March 2009, 17:52:26
You could make the limbo queue just be a slot that can contain multiple people (or slots created as the people are connecting), and when the game is started, the people in the slot/slots basically act as players with nothing to do but watch (If this is what you're planning, sorry, I only read the first page. :-\). And btw, why shouldn't the spectators be able to talk to players?
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 7 March 2009, 00:30:21
Quote
GameInterface and it's derivitives don't actually encapsulate a socket where data is transmitted or received, only a server socket that is capible of accepting new connections.  The RemoteInterface classes do encapsulate sockets that send and receive data.  Each RemoteInterface object is dedicated to interfacing with a single remote host.
This reminds me of Remote Procedure Calls a little. http://msdn.microsoft.com/en-us/library/aa373935.aspx
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 26 March 2009, 22:25:50
Sorry again for the delayed response.

Quote from: hailstone on  7 March 2009, 00:30:21
This reminds me of Remote Procedure Calls a little. http://msdn.microsoft.com/en-us/library/aa373935.aspx
OT: Yes, and RMI is the Java equivalent of m$'s RPCs.  RPC is a microsoft-specific technology, and RMI can share the criticism that it is Java-specific.  However, RMI offers a IIOP layer so that your RMI objects are accessible just like any other CORBA object (thus, the open standard).  When I've used RMI, I always use the IIOP layer, even when I'm only interfacing with other Java VMs.

But anyway, yes, it is similar, although nowhere near the level of thoroughness or completeness as either of the afore mentioned technologies.  A RemoteClientInterface talks across the wire to RemoteServerInterface on the other side and RemotePeerInterface objects talk to each other in turn.  The GameInterface derived objects (either ClientInterface of ServerInterface) glue them all together into a complete "network game interface" mechanism that is exposed to the rest of the executable.  If C++ had a way to make classes "package private" like java (i.e., private within a namespace) then all of then RemoteInterface and all of it's derived classes would be package private.  Now that's that part of the concept of the design.

Now, in an ideal world, since RemoteInterface objects take care of most of their own affairs on their own, they should generally run themsevles as well (i.e., have their own threads).  This is where performance considerations overstep prettiness and we do stuff like the WaitOnMultipleOjbects/muliti-select and expose their internal socket data to manage all of their I/O from a single thread (and to remove polling).

Quote from: ZaggyDad on  6 March 2009, 17:52:26
You could make the limbo queue just be a slot that can contain multiple people (or slots created as the people are connecting), and when the game is started, the people in the slot/slots basically act as players with nothing to do but watch (If this is what you're planning, sorry, I only read the first page. :-\). And btw, why shouldn't the spectators be able to talk to players?
Well, maybe spectators/observers should be able to chat globally only.  If we allowed them to spectate on everybody and then chat privately to individual teams and/or players, then they could act as spies, creating an unfair advantage for another player.  But if we made all of their chat global, it would eliminate that problem because the person they were spying on would see their chat as well and be able to complain and have them kicked by the game host. :)

So maybe we'll just call the "limbo queue" the Spectators and have them appear at either the above or below the list of factions.  I haven't seen hailstone's new GUIs yet, so however hailstone wants to figure that in is fine with me.  And actually, the spectators was hailstone's idea, I wasn't even thinking about it, but I totally agree with it.  Also, remember that we're planning on supporting multiple players controlling a faction in the future, so each of these faction rows in the GUI will eventually need to support more than one actual player being connected to it with a primary player (who's config rules will be applied to the faction -- specifically the auto-repair and auto-return preferences).  So, the "Spectators" row can work similarly.  But this is only proposed, I encourage you to decide for yourself how the UI should look as UI has never been my area of expertise.
Title: Re: chop, chop, chop (network code)
Post by: titi on 27 March 2009, 15:06:57
Just a note: RPC is NOT something Microsoft invented. They only implemented it much later than others and of course with incompatible extentions to assure their monopol. ( like they do with every standard they use )
I thought it was SUN Microsystems who invented it, but I was wrong, they only made a very popular implemention.
More here :
http://en.wikipedia.org/wiki/Remote_procedure_call
Title: Very OT!
Post by: daniel.santos on 27 March 2009, 16:38:02
Very OT!!!
cool, ty for that! :)  Yes, the practice m$ uses is called "Embrace, Extend, Extinguish".  I know that SOAP or other XML-based messaging protocols are the hot thing these days and I'm not up on those techs.  Something inside of me just cringes at the thought of bloated texty messages squirming around on the internet.  I know that we have jillions of times the bandwidth that we used to have, but I guess it's my own issue to deal with. (can't they at least gzip them? :) ).  But back to m$, if you want a good read on how these mechanisms work (the monopoly, propaganda tricks, etc.) check out the Halloween Documents (http://en.wikipedia.org/wiki/Halloween_Documents), particularly the first one with annotations by Eric Raymond -- I found this exceedingly enlightening (link: http://catb.org/~esr/halloween/halloween1.html (http://catb.org/~esr/halloween/halloween1.html) -- this used to be on the opensource.org web site, but I guess this is where it lives now).  Also note that you don't necessarily have to "extinguish" as in "remove" a technology to accomplish your mission.  I've heard native american activists say that you can commit genocide if you can kill a people's language, traditions and culture -- they wont know who they are any more, the threat is extinguished.  Similarly with technology, if you can cause a technology to become irrelevant, then you have accomplished your mission.

But back to the topic!
I'm checking in my code now.  the "svn diff" was 7603 lines long, so I have a lot of explaining to do (I always examine the diff to make sure I know what I'm checking in).  Hopefully, this will get done today because I'll have to leave in an hour.  The sources compile (on Linux) but I don't expect them to work (haven't actually tried to run it! :) ). They are still very messy (lots of commented out code & such).  All-in-all, I'm hoping that it's worth it in the end.  I've gotten a lot of very nice mechanisms in and I'm hoping that stuff like the ObjectPrinter and Printable class (interface) makes future development, debugging and troubleshooting much easier, especially in the case of users reporting errors that aren't up to debugging.  I like the way the core mechanisms have turned out, so it's just a matter of beating everything else into shape.
Title: Re: chop, chop, chop (network code)
Post by: titi on 30 March 2009, 10:29:05
Quote
"svn diff" was 7603 lines
I assume this will take a while to get it stable and working .....
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 4 April 2009, 20:36:08
Sorry, you're right.  It's a radical departure in the networking code.  On the bright side, I've been able to implement or design hooks into this layer to support most of the future functionality we have planned that the networking code needs to support.  So we shouldn't have to "break" the networking code much in the future.  This also affects the GameSettings and other such items and will support all of the ideas we've discussed about UI changes for starting new games, joining network games and restoring saved network games.  In addition, I've addressed in the GameSettings all of the ideas we've discussed about enhancing the AI (in a generic fashion), so this code shouldn't need to be changed a significant amount when we get to implementing those features.  I've done a lot of pen and paper design and re-design work to try to make sure I'm doing this part as well as possible.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 16 April 2009, 13:38:08
The network re-write is really starting to shape up!  The lobby is now fully functional for the most part (for new games only).  Now, whenever the server changes the game settings, they are correctly updated on the client and the "join game" screen now displays (crudely) the game information so you can see what settings you are about to play with and they are updated as the server changes them.  I don't plan on getting very fancy in 0.2.x with this, just a text display will be good enough for now.

Peer-to-peer functionality will also remain un-implemented in 0.2.x, although most of the mechanisms are in place, just disabled.  This is because I wanted the new networking code built to support it, but we also want to keep our goals reasonably attainable and that's just not needed to meet the minimal requirements for 0.2.12.

All of the functionality for multiple players per faction is also in place at the networking code level, but inaccessible through the GUI screens.  When we get to 0.3, that's where we're going to finish this.

So here is a snapshot of the "New Game" menu and the corresponding "Join Game" screen on the client.  All data is updated in real-time as the server changes it:
Server UI
(http://glest.codemonger.org/files/gae_0.2.12-wip_new_game_snapshot.png)

Client UI:
(http://glest.codemonger.org/files/gae_0.2.12-wip_join_game_snapshot.png)

As a final note on this, the status for each player will be displayed in this format:
Player Name: last_ping (average_ping) Tx/Rx bytes_sent_per_second/bytes_received_per_second

Here is what each of these fields are more precisely:

The duration of the history used for each of these items can be easily changed in the code if desired (in case you care, it's in the Game::Net::RemoteInterface constructor, you change the parameters passed to the construction of the "stats" data member which is of type Game::Net::NetworkStatistics).  But I figured these were good values to show the type of connection health info that we usually care about.  Packet loss is not shown since that cannot be determined when using TCP (it hides that from you).

So hailstone,
I'm finally totally ready for your help!! :)  I've checked in all of my code and I think it's been de-uglified enough that I won't be completely embarrassed to have somebody else look at it!  ;D  There are still some things that I didn't think out well enough in the design, but this should be good enough for now. (I still need to gen the API docs again and upload them as a reference.)

Also, if you want to check in your change for the 0.3 branch, I'll start working on a merge (if you're ready for me to do that).  The ugliest part will be the UI screens because I've changed the guts of how they work, but not much at all of the actual UI components.  I've tried to leave the UI in charge of making it's demands of the network interface objects and polling them during their update() functions, rather than have the network interface objects drive the UI components (i.e., the network interface objects don't know about the UI at all).

If you start two instances of glestadv, you can go into the "new game" screen of one and the "join game" screen of the other and try stuff out.  It's not uncommon at all for a failed assertion to happen (SIGABRT) as there's still lots of bugs.

Here are a few outstanding issues as far as the game lobby is concerned:

Most network failures no longer cause the game to crash.  Game::Net::GameInterface::execute() is the main network thread function.  When it calls the RemoteInterface::update() function (game/network/game_interface.cpp:214) that function should never throw anything.  Nothing that RemoteInterface::update() calls should throw anything except a GlestException or it's sub-classes (PosixException, ProtocolException, etc.) -- at least this is the way it's supposed to be, there's still more to do on that front.  Most of the networking code is executed there, however.  So RemoteInterface::update()'s primary duty is to retrieve new messages and call RemoteInterface::dispatch().  The dispatch() function is still rather clumsy. :(  I recall somewhere wanting to keep some type of NetworkMessage object, so it has this nasty "should I delete this message when you are done" mechanism that is ugly and should eventually get fixed (sorry).

So when a GlestException is thrown, the RemoteInterface::update() catches it and calls GameInterface::onError() passing it the exception which is stored (poorly) in a variable of GameInterface.  The UI classes should check this on each update and actually display the error message to the user or at least tell them that there was a network error.  The current behavior is that the information is written to the network log and the peer is disconnected.

Next, I've gotten rid of most of the old network messages that can be summarized as a simple change of state.  These are instead replaced with NetworkMessageStatus which contains the remote peer's state.  The networking code doesn't yet check for all of the state changes that signal something needs to be done, like the server going from STATE_INITIALIZED to STATE_LAUNCHING (the replacement for the old NetworkMessageLaunch message).  Once the server is done launching, but before game play actually starts, it will have created a Checksums object with all of the checksums for each file it loaded.  Each client will create these too and they should transmit them to the server once they have completed the launch sequence (also before starting game play).  This is sent in the NetworkMessageReady message.  Once the server receives them all and (if NetConsistencyChecks is enabled in the glestadv.ini) verifies that all files match, then the server should set a game start time, probably 5 or 10 seconds in the future.  The cool thing about this is that the NetworkStatistics class, through the data exchanged in pings, can provide the approximate time on a remote host using the ping history to account for latency.  Thus, the server can transmit transition to STATE_READY and set a start time, specifying it in the remote host's local time (so differences in the system clocks of different computers don't matter).  Then the UI should ideally show a count down timer so the player knows when the game start is coming.

So for now maybe you can just check out the code and post any questions you have and you let me know what part you want to work on first.  I'll check in frequently and until we have a commit mailer daemon, I'll post every time I check in.  How does that sound?
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 17 April 2009, 15:22:35
It's good you're making progress.

I've tried to compile r307 but getting this error (using "jam" without any args):
Code: [Select]
game/network/network_message.cpp: In member function ‘void Game::Net::NetworkWriteableXmlDoc::uncompress()’:
game/network/network_message.cpp:437: error: call of overloaded ‘toStr(uLongf&)’ is ambiguous
./shared_lib/include/util/conversion.h:125: note: candidates are: static const std::string& Shared::Util::Conversion::toStr(bool)
./shared_lib/include/util/conversion.h:129: note:                 static std::string Shared::Util::Conversion::toStr(int)
./shared_lib/include/util/conversion.h:135: note:                 static std::string Shared::Util::Conversion::toStr(unsigned int)
./shared_lib/include/util/conversion.h:141: note:                 static std::string Shared::Util::Conversion::toStr(Shared::Platform::int64)
./shared_lib/include/util/conversion.h:147: note:                 static std::string Shared::Util::Conversion::toStr(Shared::Platform::uint64)
./shared_lib/include/util/conversion.h:177: note:                 static std::string Shared::Util::Conversion::toStr(float)

    g++ -c -o ./build/i686-pc-linux-gnu/optimize/game/network/network_message.o -D_XOPEN_SOURCE=600 -DPACKAGE_NAME="gae" -DPACKAGE_TARNAME="gae" -DPACKAGE_VERSION="0.2.12-wip" -DPACKAGE_STRING="gae 0.2.12-wip" -DPACKAGE_BUGREPORT="http://bugs.codemonger.org" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DUSE_POSIX_SOCKETS= -DX11_AVAILABLE=1 -DHAVE_GLOB_H=1 -DHAVE_SYS_IOCTL_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_BYTESWAP_H=1 -DUSE_SDL= -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -I. -I/usr/include/SDL -D_GNU_SOURCE=1 -D_REENTRANT -pthread -pthread     -I./game/. -I./game/ai -I./game/facilities -I./game/game -I./game/global -I./game/graphics -I./game/gui -I./game/main -I./game/menu -I./game/network -I./game/sound -I./game/type_instances -I./game/types -I./game/world -I./shared_lib/include -I./shared_lib/include/platform -I./shared_lib/include/platform/sdl -I./shared_lib/include/util -I./shared_lib/include/graphics -I./shared_lib/include/graphics/gl -I./shared_lib/include/sound -I./shared_lib/include/sound/openal -I./shared_lib/include/xml   -Wall -W -Wno-unused -Wno-sign-compare -fno-strict-aliasing -O3 -g3 -DNDEBUG -ffast-math -ftree-vectorize -Winit-self -Wmissing-include-dirs -Wwrite-strings -Wextra -Winvalid-pch game/network/network_message.cpp

...failed C++ ./build/i686-pc-linux-gnu/optimize/game/network/network_message.o
For the 0.3 GUI code, it mostly changes menu files. The project files will need to be modified (Visual Studio instructions here (https://forum.megaglest.org/index.php?topic=4180.0)). I haven't tried CEGUI in Linux yet.

I've installed Xubuntu 8.10 in a VirtualBox VM hoping the 3D acceleration support would work for Glest. I installed 3.2.1 and got it to work partially without shadows but no models are showing - only rain, terrain and some GUI. I suspect the VirtualBox Guest Additions Graphics Driver doesn't include some or all of the required OpenGL extensions. And the VM just closed itself  ???

I should be able to work on the GUI code in the VM, at least until I get a non-virtual installation of Linux (the computer I was going to use decided to die).

I'm all for the frequent check ins.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 20 April 2009, 07:56:27
Thanks!  I looked this one up and it appears that the type uLongf (a type that zlib defines and uses) is defined in a very platform specific fashion, so I definitely need to cast this.

As for the OpenGL, I'm amazed they have a VM graphics driver that supports any of it!  Until I can build a new windows machine, I'm going to try and set up a build under wine.  For now, I'll get this fix checked in as soon as I can, which will probably be tomorrow.
Title: Re: chop, chop, chop (network code)
Post by: hailstone on 28 April 2009, 11:15:00
I'm able to compile and connect to another instance with the loopback address (anything else locks it up). Compiler still says it is ambiguous. Changing long to int works. Perhaps another overloaded method for unsigned long would be better?

The errors are much nicer.

off topic:
On the codemonger website it says, "For linux, you must download glestadv-data separately and extract." but there is no archive of this except on the sourceforge downloads page. Or there is and it's not clear which one.
Title: Re: chop, chop, chop (network code)
Post by: daniel.santos on 28 April 2009, 22:57:23
For the errors, I think we'll eventually want to have the long description go to the log file and display a more succinct message in-game, but this is good enough for now.

As for the ambiguity error, I think I see the problem! :)  I only implemented toStr for a small handful of types.  I'll add a fatter catch-all that uses a stringstream and accepts a templatized type.  This will result in more code being inlined than the way the others are done because we have to create and destroy a stringstream object, but it'll work for the odd cases.
Code: [Select]
    // ugly (i.e., fat) catch-all
    template<typename T>
    static string toStr(T v) {
        stringstream str;
        str << v;
        return str.str();
    }

Unfortunately, the box I was going to convert into a windows machine is now missing a power supply, so I'll have to wait a bit longer before I can install windows again and begin testing & debugging there.  I guess we're both getting a taste of computers deciding to die. :o 

Thanks for letting me know about the messed up glestadv-data thing, I'll check that out.  I think that it's outdated now since we're including all of those files with the compiled binaries.