chop, chop, chop (network code)

[Omega]

A master server... <3  *sigh*

heaven (on earth)

[daniel.santos]

Do you have any news for us here? Is there any progress or trouble?

One little wish (if you are still on it), please keep in mind that we once want a master server to setup/find games.
We need no implementations yet, just keep it in mind when you design protocols.

Hey buddy, sorry for the late response!!

Yea, lots of progress, but lots of work.  I've also been fiddling with a few other projects, Lincity-NG, KDevelop and gdb recently.  I might try to start working on gdb's Project Archer (http://sourceware.org/gdb/wiki/ProjectArcher) and if I like it, I might try to get a paying job working on gdb even!  I've also have had a lot of other work to take care of over the past few months, but progress indeed is being made.  No real problems.  I posted a minor enum quandary recently, but nothing I'm worried about, I'm using the copy & paste mechanism for now because it's simple.

OT: On the topic of languages and enums (but off of this topic ), I'm not even happy with Java 5's enums   But that's ok, it's better (in most ways) than how we did enums before in Java (and I still have projects that used old-school Java enums -- basically it enforces it's behavior via private & protected constructors & such).  In fact, I have a Java project that uses polymorphic Java enums, that one's a trip!  It takes a lot of rules to enforce it, but they have come out cleanly thus far and work great over network (RMI over IIOP) and other serialization situations -- compact on the wire, yet safe for ensuring the same bytecode is using it on each end.  I rarely use the default Java object serialization across the wire, because I think it's too fat (even compressed).  All of this while handing a variable inheritance tree is tricky, but it served the specific application very nicely.  Oh yea, they were also annotated for use in Java 5 Persistence (via Hibernate) and that's where they were mostly used.  But, I'm WAAAY off topic! (sorry guys! )

So since I'm doing the networking from the ground up, I'm just trying to make sure the quality is top notch.  I've refactored and reexamined the design a number of times now (sometimes on paper, UML or text/word doc, other times in code).  I'm getting happier with the design, I just want to make sure it's nice and clean.   If I find a UML editor (preferably one that at least does descent C++ reverse engineering) I'll try to spit out another class diagram and maybe convert the text I posted earlier into a real visual sequence diagram (although I think the text at least serves it's purpose).  I want to make sequence diagrams for each major networking operations during game play as well (what I covered is just in the lobby).  Most of this is already in place, it's just getting altered.

Also, I've added this Printable C++ class (an abstract interface) and an ObjectPrinter class to print Printable objects and I quite like it.  Having this in place will make debugging the networking code very nice.  Example output:

Code: [Select]

361750: sent to peer[0]: NetworkMessagePing {
  NetworkMessage {
    size = 22
    type = NMT_PING
    writing = false
    simRxTime = 292632639351
  }
  id = 213
  time = 292632534356
  timeRcvd = 0
  pong = false
}

The idea here is to cut out as many possible ways to be confounded as reasonably possible.  Silly bugs can be hard to trace down once they go across the wire, is the problem on the client or the server?  Well with all messages logged, it's easier to tell.

Looks good.

Thanks!

OT:

but this is the nature of a requirements/specification text

Apparently this is a specific language. I read this about the C++ Standard by James Kanze at http://groups.google.com/group/comp.lang.c++/browse_thread/thread/28b14a1308974070:
"The standard is written in a variant of English sometimes called standardese. It's designed (or at least intended) to be
absolutely precise and unambiguous, even at the cost of understandability. It doesn't always succeed with its intent,
but in all cases, precision and a lack of ambiguity have precedence over readability."

Hah! no doubt, it definitely begins to become a language of it's own.  But if I don't specify things like this, then people die.  Well.... not actually at the moment, but if I were working on code for life support systems, heavy machinery, etc., people could die from ambiguity.  Certainly projects, careers and businesses literally die, right and left as a result of these types of mistakes (it's scary!).  But I would rather be on a low-paying project with a high chance of success than a six figure project that's doomed (I've been there, it's scary as hell and not worth the pay!).

So anyway...

[0.2.12]: Clients should be able to connect when server is in lobby but there's not an open slot set to "network" (i.e., be in limbo for a time while host changes settings)

Will the server want the first people in to automatically be assigned to a slot? and it doesn't explain what happens when there are two people in limbo and a slot is opened.

I propose that a que (with a max amount of items) be created where players are sent to when they join. The server can then add or remove the desired network or CPU players to or from a slot (the position of which is moveable by the server). Empty slots should be visual and numerical.

Perhaps before clients connect server can adjust settings then press a listen button to allow clients to enter the que (something useful for a master server so games don't appear immediately).

How will specators be dealt with?

Spectators! What a wonderful idea! (why didn't I think of that?   So I think that sounds like a lovely idea and very apropos.  So perhaps when a client initially connects, they automatically go into the observer pool as a substitute for "limbo".  The person on the server then has to select who they want in which slot.  Alternately, people can be automatically sucked out of the observer pool when the server sets a slot to be network controlled and kicked back to the observer pool when they close it or set it to AI, plus giving them the option to manually move them around.  I dunno how to best do that in the UI at this point, but I know how to best implement it in the supporting classes that the UI will interact with.

However, spectators will require a new enum entry for the NetworkRole enum (I've already added them and some other various code here and there to manage it.  Because I want to allow two players to control the same faction (optionally of course) as well as allow players to control more than one faction on their team, this presents its self at a good time.  If coded correctly, it can actually simplify things more than complicate it.  Each player basically gets a list of factions and teams they can control as well as factions and teams they can observe.  This is important for good AI when scenarios involving compound teams to best simulate the story line (i.e., having two AIs, each configured differently, both controlling the same faction can produce some interesting, if not confusing outcomes).

# GsAutoRepairEnabled - rather or not auto-repair is allowed
# GsAutoReturnEnabled - rather or not units will automatically return from auto-commands like auto-attack, auto-repair, etc.

I don't know if I agree with GsAutoRepairEnabled and GsAutoReturnEnabled being forced on clients. It seems like personal preference to me.

Also here are some things that you didn't mention: syncing tech trees or downloading maps, password protected joining.

hmm... Some may see these as a bit of cheating because the computer is doing something for you, so I feel that it's probably important to allow the server to define rather or not they can be enabled in the game rules.  I hadn't considered that in a network game, the host allows them, but one of the players may not wish to use them, so I agree, there should be a way to disable it for the client that doesn't want it.  So perhaps it's only enabled if it's enabled in the local game settings and also the network GameSettings?  We can worry about a pretty way to present that in the UI later (as long as the actual rules make sense for now).

Thanks for your feedback hailstone!

By syncing, do you mean the ability to play on a map or tileset that one player does not have? If so, that is a must. Not sure how that'll cope with a tileset though, seeing the size of that.

No, I'm referring to the synchronization of units in game.  I'm not opposed to a feature that allows you to transfer content files while in the game lobby, there is already full support for transferring files in the code (that's now network games are resumed, the server sends each client the saved game).  However, I just want a nice well-thought out paradigm.  My primary concern is that whatever functionality that allows people to share content needs to be absolutely safe from exploitation.  That's the only reason I don't quickly throw in a few lines of code to allow you to sync tile sets, maps and tech trees with others from inside the game (I want it done right basically).  My real preference is to have a system similar to Wesnoth where there are update servers and you can download mods, tilesets, etc. from there.  This gives some ability for files to be monitored and checked for exploits should any surface.  Then, if two people wanted to play but one player didn't have the files, the client could sent the file data (summary) to the other client and they could then retrieve it from the main server (from in game).  All of this is functionality I don't plan on personally pursuing very soon.

Why can't we have each player choose his or her own tileset? The as long as the walkable/unwalkable parts are the same, it shouldn't change gameplay. (most tilesets have the same items as walkable/unwalkable anyway).

Currently, it can be raining on one network game and clear in another.  That's a bug (in bug database) and will be fixed soon in >=0.2.12.  Because tilesets define weather and can potentially effect visibility, I'm not sure that it would be appro0priate to allow different tile sets.  Either way, that would also require hacking up the game further and I'm not sure I'm in favor of that particular idea.

Lookin' good. I never could understand network code. That's the one thing worse than AI. (AI isn't that bad actually, unless we are talking about RTS AI, which I imagine is the hardest).

Thanks!  OT: No, RTS AI isn't the hardest at all!  RPG AI (of the type we haven't seen yet) is the hardest IMHO.  I started designing a personality engine to simulate real interactions with NPCs a few years ago just to see how closely we could make a video game character behave like a real living organism.  I came up with some interesting ideas, but decided that the end result (after a lot of work) probably wouldn't benefit man kind because it could become too engrossing and video games (especially MMORPGS) are already far too immersive as it is!  I had the help of a friend who's studying psychology and already has a degree in biochemistry.  This was valuable because the first level of the design was the simulating the reptilian brain (you can't simulate a personality until you understand the layers and instincts that make up a personality, animal, human, imaginary species, or otherwise).  You have to run a simulation for each new character based upon events in their lives that form their persona, coupled with their genetics, environment and social interactions.  Once you do that, you can use the persona data to have an RPG character that interacts with you in a fashion based upon their (simulated) life experiences.  Further yet, memory continues to accumulate and the process never stops, so the next time you interact with them, they will respond according to how previous interactions went.  This was originally an extension of Everquest's crappy "faction system" when I was doing some RPG hacking long ago. -- But again, WAAAAYY off topic!

[daniel.santos]

hailstone, sorry for taking so long to address the issues you bought up.  I've updated the specification addressing your issues except for the last 3, tech tree synching, sharing maps and password-protected games.  I've gone through a number of designs and re-designs and I'm feeling pretty happy with what I have now.  I'm going to get everything working up to the exchange of player info & game info between the client and server as that will give me enough to really validate my design in practice and then I'll be ready to have you join in.  I regret that I didn't have this ready for you sooner, but such is life.  I think that I've been able to simplify things enough and separate out the various concepts like the aught to be.

The main differences is that each RemoteInterface object now performs most of the functions involved with handling the messages it receives.  It performs the processing it needs to do and then calls an onReceive() function of the GameInterface derived class (which are ClientInterface and ServerInterface), where again, the GameInterface base class handles what is generic enough and then passes control to the derived class in a sort of Chain of Responsibility pattern (but not truly so).  I'm using the NVI Idiom (Non-Virtual Interface) in the GameInterface to perform what functionality is common and then pass control to the deriving class via a private virtual function (which surprisingly enough, can be overridden by a deriving class).  This is actually a new technique that I recently learned to help clean up a variety issues that I was having otherwise.

So here's a brief text-based class diagram for summary:

Code: [Select]

                             Host
                               |
                 +--------------------------------+
                 |                                |
                 V                                V
           GameInterface                    RemoteInterface
                 |                                |
            +----+----+                +----------+-------+
            V         V                |          |       |
ClientInterface    ServerInterface     V          |       |
                            RemotePeerInterface   V       |
                                    RemoteClientInterface V
                                             RemoteServerInterface
RemoteInterface no longer uses multiple inheritance to derive from NetworkStatus (and I've renamed NetworkStatus to NetworkStatistics).  Instead, it has a "NetworkStatistics stats" object as a data member.  As before, each RemoteInterface object has a "GameInterface &owner" data member named which refers it back to whomever "owns" it (so the RemoteInterface class hierarchy is analogous to the previous ConnectionSlot class) and the ClientInterface class has a "RemoteServerInterface server" data member since it is always associated with a single server.  RemoteInterface objects are not necessarily connected by virtue of their existence any more (as the ConnectionSlot class used to function).  There is also still just one network thread (instead of one for each connection as in a much earlier design) and it calls GameInterface::execute() which contains the main network thread loop.  Since I have implemented a Socket::wait function that will wait on multiple sockets, this loop is pretty CPU-friendly now, so there's no polling anymore, it pretty much only runs when there's really something to do.  I also managed to get the DEBUG_NETWORK_DELAY functionality cleanly woven into the design (finally!  I rather agonized over this part until I found the right approach).  So now I've gotten the number of #if blocks for this reduced to a bare minimum and nicely contained in just a few places (in NetworkMessage and RemoteInterface classes only now).

So that's the basic run down on what's changed and I hope to have it ready for you to start working on some of it soon!

[hailstone]

I had a read of the NVI idiom. It looks really good.

I hope to have it ready for you to start working on some of it soon!

That's good. I have a couple of issues though. I see interface in most of the names, is this just for naming or is it an actual interface class? I assume it's not an interface class since it is being instantiated.

RemoteInterface objects are not necessarily connected by virtue of their existence any more

Does this mean that by creating a game it won't listen for connections without further action?

[daniel.santos]

I had a read of the NVI idiom. It looks really good.

Yea, it makes sense where there is stuff that should always be done, but you want to allow the core of some function to be overriden.  I'm glad I've been learning a lot of new things over the last several months. (I picked that one up from Scott Meyer's book as well)

... I have a couple of issues though. I see interface in most of the names, is this just for naming or is it an actual interface class? I assume it's not an interface class since it is being instantiated.

Yea, I agree that it's a bit confusing and I'm certain open to alternate naming!  Truth be told, I really like Java's solution to multiplie inheritence, that there are the concept of interfaces which are separate from the concept of a class and that you can only derived from one class but that you may implement as many interfaces as you like.  I'm not using any of the nomenclature/naming-conventions that distinguishes classes from interfaces (like m$'s CMyClass and IMyInterface style), so in the networking code, "Interface" is referring to a logical "network game interface".  Also, "GameInterface" could also be named "LocalInterface".  GameInterface and it's derivitives don't actually encapsulate a socket where data is transmitted or received, only a server socket that is capible of accepting new connections.  The RemoteInterface classes do encapsulate sockets that send and receive data.  Each RemoteInterface object is dedicated to interfacing with a single remote host.

RemoteInterface objects are not necessarily connected by virtue of their existence any more

Does this mean that by creating a game it won't listen for connections without further action?

No, it's just an internal thing because when the ClientInterface object is created it has a "RemoteServerInterface server" object that's a data member.  Thus, the RemoteServerInterface object is alive, but is not actually connected to anything.  The client will create and destroy RemotePeerInterface objects as needed, as will the ServerInterface create & destroy RemoteClientInterface objects as needed.  But the ClientInterface always has a RemoteServerInterface object that's valid, but not necessarily connected to anything.  After you look at it, if you think it's a bad design, please say so.

For instance, the ClientInterface object could change it's "server" data member to type "RemoteServerInterface *" that's NULL whenever there is no connection attempt in progress.  However, all of the RemoteInterface classes will exist through many states which aren't necessarily usable (i.e., STATE_CONNTECTING, STATE_NEGOTIATED, etc.), so care must be taken throughout that just because an object exists, that it doesn't mean that we have a valid connection to a remote host. 

Also, after examining it further, I'm wrong.  The old ConnectionSlot class did previously have instances before they were actually connected to anything.

[ZaggyDad]

You could make the limbo queue just be a slot that can contain multiple people (or slots created as the people are connecting), and when the game is started, the people in the slot/slots basically act as players with nothing to do but watch (If this is what you're planning, sorry, I only read the first page. ). And btw, why shouldn't the spectators be able to talk to players?

[hailstone]

GameInterface and it's derivitives don't actually encapsulate a socket where data is transmitted or received, only a server socket that is capible of accepting new connections.  The RemoteInterface classes do encapsulate sockets that send and receive data.  Each RemoteInterface object is dedicated to interfacing with a single remote host.

This reminds me of Remote Procedure Calls a little. http://msdn.microsoft.com/en-us/library/aa373935.aspx

[daniel.santos]

Sorry again for the delayed response.

This reminds me of Remote Procedure Calls a little. http://msdn.microsoft.com/en-us/library/aa373935.aspx

OT: Yes, and RMI is the Java equivalent of m$'s RPCs.  RPC is a microsoft-specific technology, and RMI can share the criticism that it is Java-specific.  However, RMI offers a IIOP layer so that your RMI objects are accessible just like any other CORBA object (thus, the open standard).  When I've used RMI, I always use the IIOP layer, even when I'm only interfacing with other Java VMs.

But anyway, yes, it is similar, although nowhere near the level of thoroughness or completeness as either of the afore mentioned technologies.  A RemoteClientInterface talks across the wire to RemoteServerInterface on the other side and RemotePeerInterface objects talk to each other in turn.  The GameInterface derived objects (either ClientInterface of ServerInterface) glue them all together into a complete "network game interface" mechanism that is exposed to the rest of the executable.  If C++ had a way to make classes "package private" like java (i.e., private within a namespace) then all of then RemoteInterface and all of it's derived classes would be package private.  Now that's that part of the concept of the design.

Now, in an ideal world, since RemoteInterface objects take care of most of their own affairs on their own, they should generally run themsevles as well (i.e., have their own threads).  This is where performance considerations overstep prettiness and we do stuff like the WaitOnMultipleOjbects/muliti-select and expose their internal socket data to manage all of their I/O from a single thread (and to remove polling).

You could make the limbo queue just be a slot that can contain multiple people (or slots created as the people are connecting), and when the game is started, the people in the slot/slots basically act as players with nothing to do but watch (If this is what you're planning, sorry, I only read the first page. ). And btw, why shouldn't the spectators be able to talk to players?

Well, maybe spectators/observers should be able to chat globally only.  If we allowed them to spectate on everybody and then chat privately to individual teams and/or players, then they could act as spies, creating an unfair advantage for another player.  But if we made all of their chat global, it would eliminate that problem because the person they were spying on would see their chat as well and be able to complain and have them kicked by the game host.

So maybe we'll just call the "limbo queue" the Spectators and have them appear at either the above or below the list of factions.  I haven't seen hailstone's new GUIs yet, so however hailstone wants to figure that in is fine with me.  And actually, the spectators was hailstone's idea, I wasn't even thinking about it, but I totally agree with it.  Also, remember that we're planning on supporting multiple players controlling a faction in the future, so each of these faction rows in the GUI will eventually need to support more than one actual player being connected to it with a primary player (who's config rules will be applied to the faction -- specifically the auto-repair and auto-return preferences).  So, the "Spectators" row can work similarly.  But this is only proposed, I encourage you to decide for yourself how the UI should look as UI has never been my area of expertise.

[titi]

Just a note: RPC is NOT something Microsoft invented. They only implemented it much later than others and of course with incompatible extentions to assure their monopol. ( like they do with every standard they use )
I thought it was SUN Microsystems who invented it, but I was wrong, they only made a very popular implemention.
More here :
http://en.wikipedia.org/wiki/Remote_procedure_call

[daniel.santos]

Very OT!!!
cool, ty for that!   Yes, the practice m$ uses is called "Embrace, Extend, Extinguish".  I know that SOAP or other XML-based messaging protocols are the hot thing these days and I'm not up on those techs.  Something inside of me just cringes at the thought of bloated texty messages squirming around on the internet.  I know that we have jillions of times the bandwidth that we used to have, but I guess it's my own issue to deal with. (can't they at least gzip them? ).  But back to m$, if you want a good read on how these mechanisms work (the monopoly, propaganda tricks, etc.) check out the Halloween Documents (http://en.wikipedia.org/wiki/Halloween_Documents), particularly the first one with annotations by Eric Raymond -- I found this exceedingly enlightening (link: http://catb.org/~esr/halloween/halloween1.html -- this used to be on the opensource.org web site, but I guess this is where it lives now).  Also note that you don't necessarily have to "extinguish" as in "remove" a technology to accomplish your mission.  I've heard native american activists say that you can commit genocide if you can kill a people's language, traditions and culture -- they wont know who they are any more, the threat is extinguished.  Similarly with technology, if you can cause a technology to become irrelevant, then you have accomplished your mission.

But back to the topic!
I'm checking in my code now.  the "svn diff" was 7603 lines long, so I have a lot of explaining to do (I always examine the diff to make sure I know what I'm checking in).  Hopefully, this will get done today because I'll have to leave in an hour.  The sources compile (on Linux) but I don't expect them to work (haven't actually tried to run it! ). They are still very messy (lots of commented out code & such).  All-in-all, I'm hoping that it's worth it in the end.  I've gotten a lot of very nice mechanisms in and I'm hoping that stuff like the ObjectPrinter and Printable class (interface) makes future development, debugging and troubleshooting much easier, especially in the case of users reporting errors that aren't up to debugging.  I like the way the core mechanisms have turned out, so it's just a matter of beating everything else into shape.

[titi]

"svn diff" was 7603 lines

I assume this will take a while to get it stable and working .....

[daniel.santos]

Sorry, you're right.  It's a radical departure in the networking code.  On the bright side, I've been able to implement or design hooks into this layer to support most of the future functionality we have planned that the networking code needs to support.  So we shouldn't have to "break" the networking code much in the future.  This also affects the GameSettings and other such items and will support all of the ideas we've discussed about UI changes for starting new games, joining network games and restoring saved network games.  In addition, I've addressed in the GameSettings all of the ideas we've discussed about enhancing the AI (in a generic fashion), so this code shouldn't need to be changed a significant amount when we get to implementing those features.  I've done a lot of pen and paper design and re-design work to try to make sure I'm doing this part as well as possible.

[daniel.santos]

The network re-write is really starting to shape up!  The lobby is now fully functional for the most part (for new games only).  Now, whenever the server changes the game settings, they are correctly updated on the client and the "join game" screen now displays (crudely) the game information so you can see what settings you are about to play with and they are updated as the server changes them.  I don't plan on getting very fancy in 0.2.x with this, just a text display will be good enough for now.

Peer-to-peer functionality will also remain un-implemented in 0.2.x, although most of the mechanisms are in place, just disabled.  This is because I wanted the new networking code built to support it, but we also want to keep our goals reasonably attainable and that's just not needed to meet the minimal requirements for 0.2.12.

All of the functionality for multiple players per faction is also in place at the networking code level, but inaccessible through the GUI screens.  When we get to 0.3, that's where we're going to finish this.

So here is a snapshot of the "New Game" menu and the corresponding "Join Game" screen on the client.  All data is updated in real-time as the server changes it:
Server UI


Client UI:


As a final note on this, the status for each player will be displayed in this format:
Player Name: last_ping (average_ping) Tx/Rx bytes_sent_per_second/bytes_received_per_second

Here is what each of these fields are more precisely:

• last_ping - the most recent ping time in milliseconds (pings occur every 1 second)
• average_ping - the average ping time over the last 10 seconds
• bytes_sent_per_second - average data transmitted (in bytes) over the last 5 seconds
• bytes_received_per_second - average data received (in bytes) over the last 5 seconds

The duration of the history used for each of these items can be easily changed in the code if desired (in case you care, it's in the Game::Net::RemoteInterface constructor, you change the parameters passed to the construction of the "stats" data member which is of type Game::Net::NetworkStatistics).  But I figured these were good values to show the type of connection health info that we usually care about.  Packet loss is not shown since that cannot be determined when using TCP (it hides that from you).

So hailstone,
I'm finally totally ready for your help!!   I've checked in all of my code and I think it's been de-uglified enough that I won't be completely embarrassed to have somebody else look at it!    There are still some things that I didn't think out well enough in the design, but this should be good enough for now. (I still need to gen the API docs again and upload them as a reference.)

Also, if you want to check in your change for the 0.3 branch, I'll start working on a merge (if you're ready for me to do that).  The ugliest part will be the UI screens because I've changed the guts of how they work, but not much at all of the actual UI components.  I've tried to leave the UI in charge of making it's demands of the network interface objects and polling them during their update() functions, rather than have the network interface objects drive the UI components (i.e., the network interface objects don't know about the UI at all).

If you start two instances of glestadv, you can go into the "new game" screen of one and the "join game" screen of the other and try stuff out.  It's not uncommon at all for a failed assertion to happen (SIGABRT) as there's still lots of bugs.

Here are a few outstanding issues as far as the game lobby is concerned:

Most network failures no longer cause the game to crash.  Game::Net::GameInterface::execute() is the main network thread function.  When it calls the RemoteInterface::update() function (game/network/game_interface.cpp:214) that function should never throw anything.  Nothing that RemoteInterface::update() calls should throw anything except a GlestException or it's sub-classes (PosixException, ProtocolException, etc.) -- at least this is the way it's supposed to be, there's still more to do on that front.  Most of the networking code is executed there, however.  So RemoteInterface::update()'s primary duty is to retrieve new messages and call RemoteInterface::dispatch().  The dispatch() function is still rather clumsy.   I recall somewhere wanting to keep some type of NetworkMessage object, so it has this nasty "should I delete this message when you are done" mechanism that is ugly and should eventually get fixed (sorry).

So when a GlestException is thrown, the RemoteInterface::update() catches it and calls GameInterface::onError() passing it the exception which is stored (poorly) in a variable of GameInterface.  The UI classes should check this on each update and actually display the error message to the user or at least tell them that there was a network error.  The current behavior is that the information is written to the network log and the peer is disconnected.

Next, I've gotten rid of most of the old network messages that can be summarized as a simple change of state.  These are instead replaced with NetworkMessageStatus which contains the remote peer's state.  The networking code doesn't yet check for all of the state changes that signal something needs to be done, like the server going from STATE_INITIALIZED to STATE_LAUNCHING (the replacement for the old NetworkMessageLaunch message).  Once the server is done launching, but before game play actually starts, it will have created a Checksums object with all of the checksums for each file it loaded.  Each client will create these too and they should transmit them to the server once they have completed the launch sequence (also before starting game play).  This is sent in the NetworkMessageReady message.  Once the server receives them all and (if NetConsistencyChecks is enabled in the glestadv.ini) verifies that all files match, then the server should set a game start time, probably 5 or 10 seconds in the future.  The cool thing about this is that the NetworkStatistics class, through the data exchanged in pings, can provide the approximate time on a remote host using the ping history to account for latency.  Thus, the server can transmit transition to STATE_READY and set a start time, specifying it in the remote host's local time (so differences in the system clocks of different computers don't matter).  Then the UI should ideally show a count down timer so the player knows when the game start is coming.

So for now maybe you can just check out the code and post any questions you have and you let me know what part you want to work on first.  I'll check in frequently and until we have a commit mailer daemon, I'll post every time I check in.  How does that sound?

[hailstone]

It's good you're making progress.

I've tried to compile r307 but getting this error (using "jam" without any args):

Code: [Select]

game/network/network_message.cpp: In member function ‘void Game::Net::NetworkWriteableXmlDoc::uncompress()’:
game/network/network_message.cpp:437: error: call of overloaded ‘toStr(uLongf&)’ is ambiguous
./shared_lib/include/util/conversion.h:125: note: candidates are: static const std::string& Shared::Util::Conversion::toStr(bool)
./shared_lib/include/util/conversion.h:129: note:                 static std::string Shared::Util::Conversion::toStr(int)
./shared_lib/include/util/conversion.h:135: note:                 static std::string Shared::Util::Conversion::toStr(unsigned int)
./shared_lib/include/util/conversion.h:141: note:                 static std::string Shared::Util::Conversion::toStr(Shared::Platform::int64)
./shared_lib/include/util/conversion.h:147: note:                 static std::string Shared::Util::Conversion::toStr(Shared::Platform::uint64)
./shared_lib/include/util/conversion.h:177: note:                 static std::string Shared::Util::Conversion::toStr(float)

    g++ -c -o ./build/i686-pc-linux-gnu/optimize/game/network/network_message.o -D_XOPEN_SOURCE=600 -DPACKAGE_NAME="gae" -DPACKAGE_TARNAME="gae" -DPACKAGE_VERSION="0.2.12-wip" -DPACKAGE_STRING="gae 0.2.12-wip" -DPACKAGE_BUGREPORT="http://bugs.codemonger.org" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DUSE_POSIX_SOCKETS= -DX11_AVAILABLE=1 -DHAVE_GLOB_H=1 -DHAVE_SYS_IOCTL_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_BYTESWAP_H=1 -DUSE_SDL= -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -I. -I/usr/include/SDL -D_GNU_SOURCE=1 -D_REENTRANT -pthread -pthread     -I./game/. -I./game/ai -I./game/facilities -I./game/game -I./game/global -I./game/graphics -I./game/gui -I./game/main -I./game/menu -I./game/network -I./game/sound -I./game/type_instances -I./game/types -I./game/world -I./shared_lib/include -I./shared_lib/include/platform -I./shared_lib/include/platform/sdl -I./shared_lib/include/util -I./shared_lib/include/graphics -I./shared_lib/include/graphics/gl -I./shared_lib/include/sound -I./shared_lib/include/sound/openal -I./shared_lib/include/xml   -Wall -W -Wno-unused -Wno-sign-compare -fno-strict-aliasing -O3 -g3 -DNDEBUG -ffast-math -ftree-vectorize -Winit-self -Wmissing-include-dirs -Wwrite-strings -Wextra -Winvalid-pch game/network/network_message.cpp

...failed C++ ./build/i686-pc-linux-gnu/optimize/game/network/network_message.o
For the 0.3 GUI code, it mostly changes menu files. The project files will need to be modified (Visual Studio instructions here). I haven't tried CEGUI in Linux yet.

I've installed Xubuntu 8.10 in a VirtualBox VM hoping the 3D acceleration support would work for Glest. I installed 3.2.1 and got it to work partially without shadows but no models are showing - only rain, terrain and some GUI. I suspect the VirtualBox Guest Additions Graphics Driver doesn't include some or all of the required OpenGL extensions. And the VM just closed itself  ???

I should be able to work on the GUI code in the VM, at least until I get a non-virtual installation of Linux (the computer I was going to use decided to die).

I'm all for the frequent check ins.

[daniel.santos]

Thanks!  I looked this one up and it appears that the type uLongf (a type that zlib defines and uses) is defined in a very platform specific fashion, so I definitely need to cast this.

As for the OpenGL, I'm amazed they have a VM graphics driver that supports any of it!  Until I can build a new windows machine, I'm going to try and set up a build under wine.  For now, I'll get this fix checked in as soon as I can, which will probably be tomorrow.

[hailstone]

I'm able to compile and connect to another instance with the loopback address (anything else locks it up). Compiler still says it is ambiguous. Changing long to int works. Perhaps another overloaded method for unsigned long would be better?

The errors are much nicer.

off topic:
On the codemonger website it says, "For linux, you must download glestadv-data separately and extract." but there is no archive of this except on the sourceforge downloads page. Or there is and it's not clear which one.

[daniel.santos]

For the errors, I think we'll eventually want to have the long description go to the log file and display a more succinct message in-game, but this is good enough for now.

As for the ambiguity error, I think I see the problem!   I only implemented toStr for a small handful of types.  I'll add a fatter catch-all that uses a stringstream and accepts a templatized type.  This will result in more code being inlined than the way the others are done because we have to create and destroy a stringstream object, but it'll work for the odd cases.

Code: [Select]

    // ugly (i.e., fat) catch-all
    template<typename T>
    static string toStr(T v) {
        stringstream str;
        str << v;
        return str.str();
    }

Unfortunately, the box I was going to convert into a windows machine is now missing a power supply, so I'll have to wait a bit longer before I can install windows again and begin testing & debugging there.  I guess we're both getting a taste of computers deciding to die.  

Thanks for letting me know about the messed up glestadv-data thing, I'll check that out.  I think that it's outdated now since we're including all of those files with the compiled binaries.