Author Topic: headless network server  (Read 1408 times)

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,239
    • View Profile
headless network server
« on: 25 September 2011, 05:45:57 »
The current version of svn has an experimental mode that allows mg to run as a pseudo headless server. Here is the FAQ regarding what we have so far:

1. To start a headless server (an MG server that will use minimal resources and is designed to host games) run:

Code: [Select]
./megaglest --headless-server-mode
2. The headless server currently starts a network lobby on the usual defined port.
3. The first client that connects to this game is the admin and is able to change settings in the lobby
4. The admin can change settings and start the game when ready (click play now)
5. If the admin selects data (tech, tileset or map) that does not exist on the headless server, the data is not changed and the admin's selecting is reverted to the servers data value.
6. The headless server at this point does NOT do any rendering, and neither does it take any slots so it can host up to 8 network players!
7. Once the headless server detects no more connected network players it auto quits the game and goes back to a new network game lobby to host a new game.

*NOTE: Currently this mode only handles hosting 1 game at a time. This will be changed later once we have worked out the bugs.

*Update: Current svn version does not load models or sounds now resulting is much less memory usage. I did a quick test and saw the headless server taking about 60 MB 20MB for an 8 player game during game play (compared to my client which was using about 450MB). With rev 2569 I also try to completely avoid calling opengl which has further cut down on the memory footprint.
« Last Edit: 27 September 2011, 05:34:20 by softcoder »

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: headless network server
« Reply #1 on: 25 September 2011, 06:05:04 »
And because most people do likely not know, a headless server is a server without a keyboard, monitor, or mouse; the standard input devices.
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: headless network server
« Reply #2 on: 27 September 2011, 18:24:54 »
Basically this allows you to run MegaGlest on your VPS or dedicated server, which is soooooo awesome I can hardly put it into words. :-)
This will solve the problem of lack of servers we've had until now, and make the game much more attractive to new and long term players alike.
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·

m0ellemeister

  • Guest
Re: headless network server
« Reply #3 on: 27 September 2011, 20:39:00 »
What are the system requirements for headless mode? How much Ram is needed e.g. for a Match with two CPU players and two human players?

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,239
    • View Profile
Re: headless network server
« Reply #4 on: 28 September 2011, 07:24:35 »
When launching an 8 player game i see it taking 20MB at the start. The max RAM likely depends on how the game is played, but i also just cleaned up a large amount of memory leaks which may help further stabilize the game and allow it to last much longer.

m0ellemeister

  • Guest
Re: headless network server
« Reply #5 on: 28 September 2011, 09:19:17 »
Sounds usable. Maybe i'll try it on my server some time.

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: headless network server
« Reply #6 on: 28 September 2011, 11:45:47 »
What are the system requirements for headless mode? How much Ram is needed e.g. for a Match with two CPU players and two human players?

Here's what I  just observed on my desktop PC (8 GB RAM, Intel Q9400 quad core CPU at 2.66GHz/core, non-recent dedicated ATI GPU RV560 with R300 open source driver on Ubuntu 11.04-AMD64), running one instance of r2583 as a headless server (map 'Conflict', tileset 'Karningul') and another as a network client connecting to it, taking one of the four network slots:

With no clients connected, the server instance uses 11284 kilobytes RSS, which has not changed during its 5 minutes of uptime.
Once I connect using --connecthost=127.0.0.1, this increases to 11304 kilobytes, and remains there for minutes while I'm changing and reverting game settings from the client. Changing the map and reverting to 'Conflict' does impact the servers' RSS allocation, it increases to 11792 kilobytes (possibly due to larger map sizes), then decreases to 11732 kilobytes (and remains there) after I switch back to the 'Conflict' map. After some minutes without a change, allocation increases to 11816 kilobytes. I then setup a manual TCP connection to the server (nc -vv 127.0.0.1 61357), but the memory allocation on the server remains unchanged.
I add a CPU Ultra bot on the other side of the map, and memory allocation increases to 11820 kilobytes.

After a few more minutes without a change (both to the settings and to RSS), I start the game. Memory allocation increases to 15532 kilobytes by the start of the game, then rises to 15796, increases by small chunks of kilobytes for some minutes. The enemy is preparing its first attack, and memory allocation starts to increase by around 1 megakilobyte during two minutes. Then the actual attack starts and the trumpet sounds. Memory allocation increases steadily in steps of ~1 MB per 5 seconds, to 31104 RSS after the attack is over (all attackers got killed). It stays there for a while, never decreases, but rises again after about three minutes, but just slightly to values of 31696 kilobytes.

I start to play, too (I'm Egypt). I build a desert camp and make another workerrepair my sphinx, allocation increases to 33824 kilobytes. The enemy is attacking again and memory allocation increases again to 36228 before the trumpet sounds, then to 50540 kilobytes while they remove all my buildings and workers. The "do you want to quit" prompt displays and the map is uncovered, and memory allocation increases to 52936 kilobytes. While I do nothing (but the CPU still works) menory allocation increases steadily in chunks of about 300 kilobytes per 5 seconds. After 40 minutes of runtime, the server process is at 60664 kilobytes RSS allocation. Five minutes later, it is at 66208 kilobytes RSS, with a total of 88 units in the game, 8 of which are visible (visible object count is 259, visible quad 7067.0). Another five minutes later, the CPU has build more units, it now has 115 (12 of which are visible to me) and memory allocation is at 76504 kilobytes. I confirm the prompt to quit the game and the stats screen display. Memory allocation decreases to 60792 RSS and remains there. This doesn't change for some minutes while the stats screen displays. I click on the 'Exit' button below the game stats, which does not impact on the servers' memory allocation. But I think I've already been disconnected by this point.

I quit my client and connect to the server again (same method as before). By the time I reconnect, the servers' memory allocation decreases to 60472 kilobytes, then 60148 kilobytes a minute later. I am player 3, all others are 'network' slots. I add an Ultra CPU for player 1, and memory allocation decreases to 58780 kilobytes and remains there. I change the tileset to 'Karningul' again, all other settings except my faction - I am now magic - are as they used to be last game.

I start the game and memory allocation increases to 58780 kilobytes RSS. It continually grows again then, about 300 kilobytes / 5 seconds while I'm doing nothing but the CPU is. I start playing by the time RSS reaches 61392 kilobytes. The first attack is incoming, it's a tough one because I wasn't prepared. I use a daemon but all my other units prevail while the attacking Egyptians are slain. Memory is now at 76232 kilobytes RSS. With 50 units in the game, the memory allocation (RSS) reaches 80 MB.

The next attack is coming, we're now at 90 MB and 72 units. After a long fight, 59 units remain in game and RSS is at 100,296 kilobytes RSS. I send two battle mages to the river crossing and run into a couple enemy units. Moving the units didnt impact memory allocation, but meeting the enemy did, it rises to 104,516 kilobytes. The enemy is now  attacking my main building and I'm loosing. Memory allocation is rising to 121396 kilobytes with roughly 1 MB/5 second increase during fighting.

With 100 units in game, 10 of which are visible, 132 visible objects and 144892 kilobytes RSS, I quit the game. This makes RSS drop to 114520 after 5 seconds, to 91512 after another 5 seconds and to 75624 after another five where it finally stabilizes.

I type 'test' into the server console, which increases RSS to 75636 kilobytes, but doing this again doesn't change memory allocation. So I type 'quit' and the process is gone.

My hope is that this post can help a little in both getting an idea of how much memory allocation to expect for a dedicated (X-less) game server, as well as for getting an idea where to look for more memory allocation issues...

Summing up, for a headless server I'd expect that you can fly safe if you expect a memory allocation of up to 200 MB RAM (this should give enough room for multiple players) for the first game played. It still seems advisable to restart the game server process between games using the --headless-server-mode=exit option.

I forgot to look into CPU resource consumption, but when I ran the server on my single core VPS on this same host system yesterday, 'top' on the VPS showed that 8% and 20% CPU resources were being used by the game server. This is not really much, so I expect the CPU resource drain to be mostly neglegible. However, all of these stats are based on games with just one human and one bot, so you may want to test this with 8 megas or multiple network players instead.
« Last Edit: 28 September 2011, 19:00:35 by tomreyn »
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·

uuu

  • Guest
Re: headless network server
« Reply #7 on: 9 October 2011, 15:52:39 »
...
I forgot to look into CPU resource consumption, but when I ran the server on my single core VPS on this same host system yesterday, 'top' on the VPS showed that 8% and 20% CPU resources were being used by the game server. This is not really much, so I expect the CPU resource drain to be mostly neglegible. However, all of these stats are based on games with just one human and one bot, so you may want to test this with 8 megas or multiple network players instead.
CPU usage actually matters a lot:
- pathfinder makes it huge (+stuck units)
- cpu mega uses a lot of cpu (or is it just # of units, doesn't really matter)
- some (all 2x2?) units makes pathfinder perform even worse

CPU matters, because if I understand it correctly, processing is done on the server (not like some games, where headless server is just for quasi VPN - i.e. interconnecting them). That means it's not suitable for not-so-fast VPSs (or dedicated), definitely not for resource-limited machines (e.g. embedded - routers etc.).

Anyways it's good news   :)

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: headless network server
« Reply #8 on: 10 October 2011, 11:46:58 »
CPU usage actually matters a lot:
- pathfinder makes it huge (+stuck units)
- cpu mega uses a lot of cpu (or is it just # of units, doesn't really matter)
- some (all 2x2?) units makes pathfinder perform even worse

Things in this area have changed a bit since v3.5.2.4. Which revision or version are you referring to?

CPU matters, because if I understand it correctly, processing is done on the server (not like some games, where headless server is just for quasi VPN - i.e. interconnecting them). That means it's not suitable for not-so-fast VPSs (or dedicated), definitely not for resource-limited machines (e.g. embedded - routers etc.).

I expect it to run fine on any dedicated server and on many VPS. A multi core configuration is definitely a good idea. Like most other game servers, it will not run on most embedded systems both due to both resource contraints (hardware limits) and different architectures (unless someone makes the effort to port it). Most embedded systems nowadays are based on some ARM architecture, and as far as I know there has been no efforts to port MegaGlest to this group of architectures, yet.
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·