Author Topic: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)  (Read 1300 times)

nig

  • Technician
  • ****
  • Posts: 110
    • View Profile
tonight I tried to download a tileset via ftp in the lobby of a game Jammyjamjamman was hosting.
After a few % progress I was disconnected from Jammy without an error-message.

I joined again Jammys game (was accepting players). This time I tried to download prax-Faction from him. Again after a download-progress from about 5% I was disconnected from his hosted lobby.
I connected again without restart MG an tried to cancel the offered download via FTP from Jammy. wasnt possible: again and again the download offer appeared.

How to reproduce:
Host a game and set some mods, which are not included by default installation.
join with a Win 64 client, who is missing those mods.
accept download via ftp from game-hoster.
working?

maybe Jammys wire was unstable last night again...

greets
nig

jammyjamjamman

  • Administrator
  • Horseman
  • ********
  • Posts: 210
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #1 on: 18 July 2013, 00:39:44 »
My upload speed was particularly slow on this day (150kbits/sec), so I think it was my ultra slow upload speed that was the main cause of this problem.
"All right, I've been thinking. When life gives you lemons? Don't make lemonade. Make life take the lemons back! Get mad! 'I don't want your damn lemons! What am I supposed to do with these?'" ~Cave Johnson, Portal 2

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #2 on: 18 July 2013, 08:18:51 »
tonight I tried to download a tileset via ftp in the lobby of a game Jammyjamjamman was hosting.
After a few % progress I was disconnected from Jammy without an error-message.

I joined again Jammys game (was accepting players). This time I tried to download prax-Faction from him. Again after a download-progress from about 5% I was disconnected from his hosted lobby.

Getting disconnected during a download is an effect which can be observed when the upload bandwidth of the server is remarkably lower than the download bandwidth of all simultaneously receiving clients, especially (but not exclusively) when coupled with an unreliable link between (some of) these endpoints. What I assume happens there is that the hosting party's line is so busy with the upload that it fails to properly keep the gameserver connection(s) alive, so the client(s) gets disconnected and, as a result, the mod transfer stops, too. This effect can be much worse when you have multiple clients downloading at the same time.

I guess the best solution would be to have the game server bandwidth limit download speed for each direct file transfer to:
(lines' total available egress bandwidth) / (amount of downloading clients) - 10 kbps

The lines' total available egress bandwidth should be configurable on the options screen on the network tab, stored in the INI.

Sadly bandwidth limiting can be difficult to implement on an application layer. However, I think the clients use libcurl for the file transfers, and there's a chance that curl supports this feature already (CURLOPT_MAX_RECV_SPEED_LARGE and CURLOPT_BUFFERSIZE seem to be the relevant curl options - there's a chance these already work as environment parameters on the clients).

Even when this is not implemented, there are additional counter measures. One of the "under the hood" changes in 3.8.1 beta1 is that clients will try to receive file downloads from the location stored on the game mods server instead of receiving it from the game server. This only works if the file the client is missing is available there (checksum match). Since Prax has not been released, yet, and Atze has not yet requested for his mod to be added to the mod menu, there can be no such download on the mod servers, yet. But it does work fine for other mods which are already listed on the mod menu.

Another option is to have clients download one by one (obviously this requires human intervention and even then it would not have worked in the very case discussed here since a single download exceeded the hosting parties' upload bandwidth - which is not an uncommon scenario since most home users have asymmetric bandwidth setups where upload < download bandwidth).

The final option, which, until bandwidth throttling is implemented / known to work, is to disable file uploads on hosts with limited upload bandwidth. This is something Jammy could setup for his host. This obviously means he won't be able to share his files with others just by hosting a game of MegaGlest (but would have to resort to manual upload, and the client to manual download), but then it's just a workaround.

I connected again without restart MG an tried to cancel the offered download via FTP from Jammy. wasnt possible: again and again the download offer appeared.

This one, for sure, is not a bug but works as designed. In the past a download prompt would only display once. If the client decided not to download it, and the host did not switch to a different mod and started the game, then the client was still lacking the mod and the game would start up but the client lacking the file would get disconnected right after the game started. Not a great situation.

So this was changed so that as long as the host doesn't configure a different mod, the download prompts do show up on the client repetitively. The client may cancel the download prompt, but as long as the host does not change to a different mod (knowing that the client doesn't have it, which is displayed i his/her console), the client will receive another download prompt a few seconds later. This way it's now much more unlikely that clients will get disconnected as soon as the game starts because they did not download a mod.
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·

jammyjamjamman

  • Administrator
  • Horseman
  • ********
  • Posts: 210
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #3 on: 18 July 2013, 20:54:14 »
The curse of adsl  :P
"All right, I've been thinking. When life gives you lemons? Don't make lemonade. Make life take the lemons back! Get mad! 'I don't want your damn lemons! What am I supposed to do with these?'" ~Cave Johnson, Portal 2

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #4 on: 18 July 2013, 23:04:04 »
Over here (and as far as I know, pretty much elsewhere, too), they apply the same restrictions to cable (even though it's not strictly technically necessary there).
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·

jammyjamjamman

  • Administrator
  • Horseman
  • ********
  • Posts: 210
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #5 on: 31 July 2013, 13:37:47 »
I've noticed this forced disconnection problem is definitely much worse than it was. Because of this bug I now can't download atze's prax mod from his own server and I was definitely able to do this in MG 3.7.1 and in other later revs (I'm not sure which rev this problem first occurred, I've only noticed it happening in beta3.8.0).
« Last Edit: 31 July 2013, 18:52:29 by jammyjamjamman »
"All right, I've been thinking. When life gives you lemons? Don't make lemonade. Make life take the lemons back! Get mad! 'I don't want your damn lemons! What am I supposed to do with these?'" ~Cave Johnson, Portal 2

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,239
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #6 on: 1 August 2013, 19:38:30 »
When you download the techtree does the host have the techtree 7z for better speed? Is your end downloading the full mod as a 7z or each file individually?

jammyjamjamman

  • Administrator
  • Horseman
  • ********
  • Posts: 210
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #7 on: 2 August 2013, 17:11:45 »
I remember atze posting a .7z archive of his faction onto a filesharing website for me to download when downloading from his server failed, so I assume it was a .7z archive, but I need atze to verify this.
"All right, I've been thinking. When life gives you lemons? Don't make lemonade. Make life take the lemons back! Get mad! 'I don't want your damn lemons! What am I supposed to do with these?'" ~Cave Johnson, Portal 2

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,239
    • View Profile
Re: forced disconnection during ftp-download ( 3.8.0. beta1 Win64)
« Reply #8 on: 2 August 2013, 22:14:48 »
please enable network logging and send the logs after reproducing this