v3.6.0.2 frozen server

[titi]

  :'(
It happened in the middle of a multiplayer game....
suddenly the server(me) froze and everyone got diconnected. the server itself never came back to life again ..

Platform was linux 32 bit ubuntu 10.04
I used the official binary from 3.6.0.2 linux installer.

here is the stacktrace:
http://pastebin.com/2Xa0cdcP

update: a very very strange stacktrace, he is frozen in a debug output to a logfile?

[will]

[small]My guess is that it was waiting for the timeout on the socket and then it got EINTR from however you got the stacktrace so you see it on the line where its going to print that it did timeout?[/small]

[titi]

I got the stacktrace from the running process with this trick:

Code: [Select]

gdb -q -n -ex bt -batch glest.bin `pidof glest.bin` | tee backtrace-running.log
So the process itself was always frozen with black screen, just playing music all the time.

[titi]

Damn frozen server again....
Stacktrace does not look the same this time :-/

http://pastebin.com/1mRmCWeq

What I saw this time was a player(client) who had a timeout message. But looking at the ingame network state he still was connected!
His slot got slowly counted up ( the number in braces ) and then suddenly everything was frozen.
I got the stacktrace with gdb from the running(frozen) process.

[tomreyn]

My game also froze yesterday while I was hosting a running game. This was triggered by loss of network connectivity on my end. So it was perfectly normal that the game failed, however, it should have cancelled the running game displaying an error message, not freeze. Since I was running the game full screen I needed to SSH in from another computer to kill -9 (signal 1 was not sufficient) the megaglest process. I also used the gdb command to get a stacktrace on the frozen process. I did this three times and the output was always the same:

Code: [Select]

tomreyn@atibox:~/megaglest-3.6.0.2$ sudo gdb -q -n -ex bt -batch megaglest `pidof megaglest`
[Thread debugging using libthread_db enabled]
[New Thread 0x7fd7798cb700 (LWP 2644)]
[New Thread 0x7fd77b0ce700 (LWP 2643)]
[New Thread 0x7fd77b8cf700 (LWP 2642)]
[New Thread 0x7fd77c8d1700 (LWP 2641)]
[New Thread 0x7fd77a0cc700 (LWP 2640)]
[New Thread 0x7fd77deea700 (LWP 2639)]
[New Thread 0x7fd77a8cd700 (LWP 1580)]
[New Thread 0x7fd77effc700 (LWP 1493)]
[New Thread 0x7fd77f7fd700 (LWP 1492)]
[New Thread 0x7fd783fff700 (LWP 1491)]
[New Thread 0x7fd788d66700 (LWP 1489)]
[New Thread 0x7fd78cd66700 (LWP 1486)]
[New Thread 0x7fd78d567700 (LWP 1485)]
[New Thread 0x7fd789567700 (LWP 1482)]
__lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
136    ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: File or directory not found.
    in ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
#1  0x00007fd79636a200 in _L_lock_928 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007fd79636a099 in __pthread_mutex_lock (mutex=0x7fd774022a60) at pthread_mutex_lock.c:82
#3  0x00007fd7965d083e in SDL_mutexP () from /usr/lib/libSDL-1.2.so.0
#4  0x00000000009bd79c in Shared::Platform::Mutex::p (this=0x7fd7740224e0) at /home/softcoder/Code/megaglest/trunk/source/shared_lib/sources/platform/sdl/thread.cpp:176
#5  0x00000000009d8814 in Lock (this=<optimized out>) at /home/softcoder/Code/megaglest/trunk/source/shared_lib/include/platform/sdl/thread.h:138
#6  MutexSafeWrapper (ownerId=<optimized out>, mutex=<optimized out>, this=<optimized out>) at /home/softcoder/Code/megaglest/trunk/source/shared_lib/include/platform/sdl/thread.h:111
#7  Shared::Platform::Socket::disconnectSocket (this=0x7fd7740174a0) at /home/softcoder/Code/megaglest/trunk/source/shared_lib/sources/platform/posix/socket.cpp:960
#8  0x00000000009de621 in Shared::Platform::Socket::~Socket (this=0x7fd7740174a0, __in_chrg=<optimized out>) at /home/softcoder/Code/megaglest/trunk/source/shared_lib/sources/platform/posix/socket.cpp:929
0x0000000000870ba8 in Glest::Game::ConnectionSlot::deleteSocket (this=0x7fd7851438a0) at /home/softcoder/Code/megaglest/trunk/source/glest_game/network/connection_slot.cpp:1188
#10 0x00000000008716a9 in Glest::Game::ConnectionSlot::close (this=0x7fd7851438a0) at /home/softcoder/Code/megaglest/trunk/source/glest_game/network/connection_slot.cpp:1054
#11 0x000000000083aa51 in Glest::Game::ServerInterface::clientLagCheck (this=<optimized out>, connectionSlot=0x7fd7851438a0, skipNetworkBroadCast=<optimized out>) at /home/softcoder/Code/megaglest/trunk/source/glest_game/network/server_interface.cpp:674
#12 0x000000000083b69f in Glest::Game::ServerInterface::checkForLaggingClients (this=<optimized out>, mapSlotSignalledList=<optimized out>, eventList=<optimized out>, socketTriggeredList=<optimized out>, errorMsgList=...) at /home/softcoder/Code/megaglest/trunk/source/glest_game/network/server_interface.cpp:961
#13 0x00000000008426ae in Glest::Game::ServerInterface::update (this=0x7fd784ed71c0) at /home/softcoder/Code/megaglest/trunk/source/glest_game/network/server_interface.cpp:1132
#14 0x00000000006f0fe6 in Glest::Game::Program::loopWorker (this=0x1ffcf50) at /home/softcoder/Code/megaglest/trunk/source/glest_game/main/program.cpp:397
#15 0x00000000006e7ba3 in Glest::Game::glestMain (argc=<optimized out>, argv=<optimized out>) at /home/softcoder/Code/megaglest/trunk/source/glest_game/main/main.cpp:3606
#16 0x00000000006ea1d3 in Glest::Game::glestMainWrapper (argc=1, argv=0x7fff90cd5848) at /home/softcoder/Code/megaglest/trunk/source/glest_game/main/main.cpp:3762
#17 0x00007fd7937ef30d in __libc_start_main (main=0x6ea2d0 <main(int, char**)>, argc=1, ubp_av=0x7fff90cd5848, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff90cd5838) at libc-start.c:226
#18 0x00000000005bf869 in _start () at ../sysdeps/x86_64/elf/start.S:113