Author Topic: GAE 0.2.9 for testing  (Read 2510 times)

daniel.santos

  • Guest
GAE 0.2.9 for testing
« on: 12 November 2008, 23:23:55 »
Hello all.  I would like to do some multiplayer testing with GAE 0.2.9, maybe I can run into somebody tomorrow in IRC to help me, or if somebody else doesn't mind trying it out and posting your results, I would greatly appreciate it.  I posted the info here: http://http://glest.org/glest_board/viewtopic.php?f=15&t=3229&p=19489#p19489.  I'm not going to be too surprised if this release is a little rough around the edges, I was mostly focused on fixing the multiplayer problems.  Thanks.

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: GAE 0.2.9 for testing
« Reply #1 on: 13 November 2008, 19:34:15 »
We tested it....
And now for the bugs:

not multiplayer relevant:
1. More red arrow problems:

a)
- select a unit
- give unit a walk command with right mouse key
- give unit attack command with buttons on right ( attack one of your houses )

=> red arrow will be shown on old walk commands position....

b)
- select a unit
- give unit a walk command using right mouse button
- now select this unit and another withg an rectangle

=> when doing so (selecting) the red arrow sometimes flashes up(very short!) in a completely wrong position.
 
c)
- select a unit
- give unit walk command with right mouse button
- while arrow is shown quickly press with left mouse button to another position

=> the red arrow is moved to the left mouse position

-------------------------------------

And now for Multiplayer:
1.
- connecting to a computer who doesn't run glest
=> crash ( last message was "Error connecting socket (Error: Connection refused)"

2.
corrupt multiplayer in general:
What happens:
Game starts and everything loks good, but after a while it begins ....
On the server evrything seems to run fine, but the client gets completly out of sync ( without crashing! )
- Units jump around ... ( it begins with small little "fast moves" and gets more and more bad )
Then it gets worse:
- units execute commands given on the client only on the server. Than, suddenly the client rolls back the whole thing and is up to date with the server( this one unit! )

its completely unplayable then and we stopped testing....


----------------
should I add all this to the bugzilla?
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

daniel.santos

  • Guest
Re: GAE 0.2.9 for testing
« Reply #2 on: 13 November 2008, 20:09:21 »
Thank you titi, this is very helpful.  Yea, probably would be better to make bugs for each of them.  At least the game didn't crash though :)
I have a few ideas that might solve it, or at least part of it

daniel.santos

  • Guest
Re: GAE 0.2.9 for testing
« Reply #3 on: 14 November 2008, 00:36:58 »
I still haven't figured out why units keep warping around, it's rather frustrating.  I'm going to work on it more tomorrow, as I just can't find a logical reason for it yet, but I can see it happening very clearly, even when I force server updates every 1000 milliseconds (every second).

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: GAE 0.2.9 for testing
« Reply #4 on: 14 November 2008, 00:42:32 »
I'm back from icehockey training now and I added all the bugs to the bugzilla now.
Probably its a good idea to log all the messages ( on client and server )
These messages should be numbered to make a debug possible.
By this we have a server and a client logfile and I can help you find bugs.(its probably slow but helps)
 I'm only a (professional)java software developer (6 years java) and my C-times are long ago(7 more years but never c++!), but I know a lot of things about software development in general and typical errors.
So probably I can help you find the bugs if you give me these logs. I'm not able to learn c++ currently because I doesn't have enough time for this :( .
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

daniel.santos

  • Guest
Re: GAE 0.2.9 for testing
« Reply #5 on: 14 November 2008, 11:20:37 »
Thanks titi.  Yea, there's a crude level of debug logging if you turn DebugMode on in the .ini, the server spits out a glestadv-server.log and the client a glestadv-client.log, but like I said, it's pretty crude.  In experimenting more, I found some ways to smooth out animations on the client, so I'm going to shuffle some of the code after I figure this part out.  But now, I'm going to add in some pre-processor enabled logging and display what each update is changing so I can isolate what's going on better.

On a side note, you may or may not be aware that the original Glest game can get out of sync as well, but you usually wont notice unless you are running two games side by side.  Since you can't save your game or pause in multiplayer, it's a bit harder to demonstrate, but I've won on two different computers before (a long time ago).  Part of the complications in GAE is because the server makes sure that the client is playing the same game that's on the server by sending unit data to the client.

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: GAE 0.2.9 for testing
« Reply #6 on: 14 November 2008, 13:31:58 »
I will try the debug switch today.......

I hope I understand whats going on in MP:

original glest:
Every user command is send to the server. In the next command cycle the command is sent to all clients and everyone executes the command.
Next commands are only send by the server if all clients already send their commands(even empty ones) for the next command cycle to the server.
A problem is that the result of the command  differs on the computers, especially when using different platforms/compilerversions.
This gets the game out of sync, because differnet things will happen on the clients.

GAE:
Tries to correct these problems, by constantly sending updates of unit states from the server to the clients. This should correct the different calculations which should not be so big when things start getting out of syn.
( How is this done when a unit is moving for example? The server has to estimate where the unit will be when the update-command is executed on a client? Or does the server execute the commands one cycle before the clients?  )


Some more ideas( I don't know if it is good or implemented ):
Die command:
The server is the only one who can send a valid "die command" if a unit dies ( on the server ). So a unit cannot die on the client when it is not dead on the server. If this happens ( client has dead unit server not ) the game is out of sync and a server update should happen ( probably client can send a message to the server to request an update for this unit if this happens.
Same could be done with a birth command.
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: GAE 0.2.9 for testing
« Reply #7 on: 14 November 2008, 17:21:13 »
Now I tested a bit (with log)

I use the same binary for master and server (different computers):
after some update commands I suddenly get this debug line on the client:
45420: update farm (29)   pos 234, 227 => 234, 227   hp 2956 => 3000
and later the things get much more worse:
295140: update worker (102)   pos 236, 30 => 239, 32   hp 600 => 543
296140: update worker (102)   pos 239, 32 => 239, 32   hp 492 => 430
296720: update daemon (15)   pos 226, 30 => 224, 41   hp 2 => 0
this should not happen( or am I wrong here? )
I played about 100 network games(full length!!) up to now with original glest3.1.2 and we doesn't had any more crash, (no out of sync)!
We always use these 3 computers next to another and we can't see any differences on server and client when we play.
But I think this update says that there is a difference now....
---
this greps for all update commands and you quickly see how the trouble starts:
grep ": update" glestadv-client.log
----
When is an update command sent?
----
I'm in the chat now.....
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
GAE 0.2.10
« Reply #8 on: 14 November 2008, 19:59:18 »
GAE 0.2.10:

We played about 20 minutes.
The units start to jump around but it was better than in 0.2.9 .
It was nearly playable.
But than a big fight starts and suddenly the client crashes:

Code: [Select]
Crash
Version: Advanced Engine v0.2.10
Time: Fri Nov 14 20:17:25 2008
Description: SIGSEGV: address not mapped to object
Address: (nil)
Backtrace:
./glestadv [0x815d98b]
[0xffffe440]
./glestadv [0x813fe3c]
./glestadv [0x8140c50]
./glestadv [0x814189d]
./glestadv [0x806b451]
./glestadv [0x8095a50]
./glestadv [0x8099aaf]
./glestadv [0x8099c41]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xd2) [0xb766bea2]
./glestadv(__gxx_personality_v0+0xe9) [0x804e6b1]

=======================

I lost my log files because I had to restart glest to get my X/Desktop working again :(
The server doesn't crash in this moment, just the client! We don't know why the server crashes3 minutes later, my son was playing around in the menu structure when it happens. I don't think it has something to do with it.
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: GAE 0.2.9 for testing
« Reply #9 on: 15 November 2008, 17:58:47 »
0.2.10: ( newly downloaded and compiled today )

We tested some more multiplayer:

problems we saw:
-  Tn GAE the whole environment is different in client and server. There are different trees which are randomly used this random must be somthing different on server and client which results in different playgrounds in server and client. I think this causes the units to walk different ways in client and server.

- about 1 minute before the end we saw a thunderbird which was attacked. My son(server) gives him the command to go away, but it was only executed on the server! On my client, the thunderbird stays where it was and fights all the enemies. He was deadly attacked svereal times, but he doesn't died. ( I know die command from server ...)  But why does he not execute the command from the server?

- It ends with a crash which on the client which says unknown command or something like this. I couldn't read it,because it was only flashed up for a second and then the game crashes completely. There was nothing in the crashlog after this!
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

daniel.santos

  • Guest
Re: GAE 0.2.9 for testing
« Reply #10 on: 15 November 2008, 21:09:58 »
Thanks again titi.  Also remember that if the client crashes, you can hit Z on the server and save them game and then pick up where you left off.... although, I can't remember if it will still work or not if you do it after the other player leaves (or crashes), I think it should.  When you go into the load screen and select a saved game that was a network game, you should get a similar interface as when starting a new network game where the clients can connect and you can't start the game until everybody is re-connected.  Part of the original code when a client crashed auto saved, but it also forced termination of the game (which I removed).  I need to go back and have it still auto-save at the moment the client disconnects and make sure the auto-save isn't overwritten if a second client disconnects (I'll have to figure that out).

I think I'm going to put the random number generator back the way Martiño had it before and maybe just add a pre-processor-enabled option to use true randomness (with my current random seeding).  I'm also looking for other ways to smooth out the client's experience (less jumping around).  But debugging on my laptop is really slow.  I've been hacking up the msvc++ build (I created a debug+opt profile) to try to get more performance out of it in debug mode, but it's tedious.  I had to completely disable pre-compiled headers to have so many different files compiled differently and compiles are a lot slower without them.  Basically, I'm turning on optimizations for .cpp files that have a lot of processing in them (rendering and path finding mostly) and leaving the rest unoptimized.  I can still debug optimized code, just not very effectively (I never learned x86 assembly, so I only have a very basic understanding of it and you have to really look at the disassembly & registers with optimized code sometimes).

Anyway, thank you again for your help and please tell your son(s) I said thanks! :)
- for the thunderbird, that one gets me.  I understand why it did what it did *after* the thunderbird was stuck there, but I don't understand why it got stuck there after he told it to go away.  I do know that it should have fixed its self within 30 seconds due to the MinFullUpdateInterval and if it crashed after that, then ... hmm, maybe there was some type of inconsistency in the network data, I did hack that code recently.  One thing I like about using UDP is that if a packet is bad, you can just drop it and move on to the next one.  But with TCP, it's just a stream of data, so an error in one place can throw the whole thing off.

I might have a good semi-optimized debug windows build, so I'm going to go for a 2 player game and see what I can come up with.  Also, if you feel like playing with it again, maybe you can compile it with something like
CXXFLAGS="-O3 -g3" jam
And don't strip it, it might create better backtraces, but I'm not 100% certain of it (and it shouldn't be much slower).

daniel.santos

  • Guest
Re: GAE 0.2.9 for testing
« Reply #11 on: 17 November 2008, 16:02:02 »
titi, I'm hoping to have something to test today or tomorrow.  I finally got it to crash!  I did also fix another minor bug, but I doubt that was one that you've crashed on.  I've got the crashed game in the debugger now, I just have to dig through it and try to figure out where the corruption came from.  Either way, I got the crash a few minutes after a large battle, although it seemed to handle the battle its self just fine.

There was also a few other oddities, like units on the client sometimes not facing the right direction when in combat.  Also I coded in some progress advancement so that when the client gets an update from the server, it uses half of the ping time, and advances that unit's state to try to make up for the lag time -- you probably wont notice the effect of that on the lan, but this will keep a moving unit from backtracking very slightly when it's updated from the server, making them appear jumpy