[lacks info] Slow Loading

[-Archmage-]

Running on a high speed Intel SSD, when loading my JungleHD tileset for testing it can take around 30+ seconds to load. It sticks at loading the tileset itself for 5 seconds, then unloading map for 5-10, then the screen flickers and it sticks on initializing renderer for 15-20 seconds.

But I can start a game of Black ops 2 and the match loads in under 5 seconds.... DX11, max settings.. all that.

I used the task manager to watch CPU usage and Disk usage. CPU never went above ~25%, and Disk Usage blipped at 5%..... Obviously it's not using my resources properly, there is tons more juice

Ideas for speeding up loading times:
-MultiThreading+Max Use of HDD/SSD (until one bottlenecks)
-Prefetch stuff while I'm on the loading screen.

My SSD is easily capable of loading the entire MG directory into RAM in about 1 second. The CPU has 6 cores and isn't being used much. What's happening?

[titi]

I think we see this slow loading in Windows only! Not sure why, maybe it has something to do with antivir programs ? Or it is the windows filesystem doing strange things here, especially when loading things for the first time ...
But its indeed a VERY big difference if you regard linux and windows loading times :-/ .

[-Archmage-]

Titi wait to test my tileset, I'll release it tonight.

[tomreyn]

AV could indeed explain it. If a mod/tileset consists of many files and all of them are checked by the on-access scanner, this could explain it. On the other hand, with an SSD this should only slow things down if the AV scanner is badly programmed (which may well be the case, many of those engines date back to pre-threaded times).

My SSD is easily capable of loading the entire MG directory into RAM in about 1 second.

How did you test this? How much unallocated RAM (minus SSD backed swap) did you have at this point and how much do you have when you run MegaGlest and start a game?

[-Archmage-]

Dude I have 32GB of low latency ram, and usage was 3.4 and it went up .3 or something. Definitely not an issue there.

I run no anti-virus.

Why is it barely touching my hardware though?

[tomreyn]

I was asking this question in an attempt to find out. Apparently AV is not the major bottleneck, so it has to be something else. You could try to debug it (ollydbg or similar) to find out what it's doing there and why it's taking so much longer than on Linux.

Edit: Actually Windows Performance Toolkit (WPT) may be a better utility, this seems to be the Windows equivalent to perf.

[-Archmage-]

Wait till I unleash the next release of my tileset.

[Omega]

Just want to point out that if this is an engine issue, it's very specific, as I'm running Windows 7 as well and did not notice this issue. Running with four different techs on Arch's Jungle HD tileset, there was a couple of seconds used to load the factions (about 4) followed by about 10 seconds "unloading map" with less than a second of other stuff after that. This was with an ADATA SSD and a 2700K. Doesn't seem to be a Windows bug, at any rate. Not sure what you're see, Arch.

[-Archmage-]

I'm on Windows 8, not 7. Gettings hundreds of fps in-game, so my hardware should be fine...
I had a flickering issue in photoshop so I think I'm gonna wipe and reinstall my graphics drivers. If that doesn't do it then I'm gonna go to default graphics settings on MG and try. I have everything cranked up as high as possible. 1024 shadows and so on.

[-Archmage-]

Yup, I switched the settings down to low and the loading was VERY fast except for the unloading map. What is unloading map anyway? Could it be calculated in the background of the other loading or optimized somehow?

I shall tweak my settings back up 1 by 1 and report if I find a major culprit!

Edit: I put everything back up to max, then I remember that I deleted glestuser.ini as well as changing all the setting in glest.ini. I still have the glestuser.ini file if anyone wants to explore this strange issue...

Now what about the unloading map???

[titi]

 Please put the ini file on pastebin.

About unloading map: This cleans up the memory. But i have no clue why it takes so long for you. For me its usually so fast you don't even see the message. For sure I never saw something that too more then one second.

[-Archmage-]

Well as you can see, unloading map takes 10+ for both Omega and I, and both of us are running SSDs with pretty damn fast processors.

http://pastebin.com/1Mx00Ks9

[tomreyn]

I just tried a game on a 128x128 map with 6 players using a total of 4 different factions on Windows 7 running in VirtualBox, on my atibox, which is much more low-spec than your systems (specifications below). Clearly this is a different setup (the video drivers are actually running on Linux) than a bare-bone Windows system which likely means the following results are not comparable to yours (so I'm just posting them to indicate that it can be fast enough on a virtualized Windows):

Load time per faction: ~ 5-10 seconds.
Unloading map: <3 seconds.

Video settings are about as low as possible.

[softcoder]

I see nothing obvious in the ini file that would cause problems, does you current working settings have texture compression turned on? If you toggle it and restart the game does it affect load times?

Thanks

[-Archmage-]

I see nothing obvious in the ini file that would cause problems, does you current working settings have texture compression turned on? If you toggle it and restart the game does it affect load times?

Thanks

With texture compression on, it took around twice as long to load. The screen glitched once and the mouse cursor switched to the loading circle, and if I clicked the screen would brighten up and crash. But I didn't touch it and it did load. I think I'll be keeping it off..

With texture compression on, I loaded the factions with the original Jungle tileset, unloading map took about 2 seconds, and the initializing renderer took about 6.. So on average a 5 second better loading time probably.

With texture compression OFF, I loaded the factions with the original Jungle tileset, unloading map took about 1 second, and the rest flew by in another second.

[softcoder]

Is this still a problem? If so please spell out the exact steps and result so that i can try to reproduce the issue and see if i can fix it.