Author Topic: Optimizing g3d files  (Read 6051 times)

ProfEclipse

  • Summoner
  • **
  • Posts: 49
    • View Profile
Optimizing g3d files
« on: 14 September 2012, 21:33:45 »
Some of size of models are too high. I know that size of per model is affected by :
- Number of meshes.
- Number submeshes of per mesh.
- Number of points, triangles of per submesh.
- Number of frames of per submesh.
(Ex : Prince's models (Elves) are greater than 7Mb ! I detected that frame amounts is too high (Over 30, 40,...))

Seem some of meshes are not animated but have two or more frames, I guess.
Ex : Tech's Castle has two meshes, A big one (the static building) is not animated, and a small one (the flag) has 10 frames.
Does the big one have 10 frames ?

Then, What tool can optimize g3d files ?
« Last Edit: 26 September 2012, 06:51:28 by ProfEclipse »

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Optimizing g3d files
« Reply #1 on: 14 September 2012, 23:34:33 »
Well, there's quite a bit to store in a G3D file. The locations of every vertex, every UV texture coordinate, line, and normal on each frame. G3D models are also stored uncompressed. Compression (such as LZMA) can reduce the filesize by as much as 90% (although it would have to be uncompressed when loaded into memory, and since GAE already supports compressing entire mods, there's little to gain from compressing the models). However, animation between frames is heavily interpolated. A two frame animation won't appear like a flip book, but is smoothed out as movement of each vertex. The biggest current optimization that a modder can do is reduce the number of frames. Most models won't need more than eight frames, tops, and simpler models can get by with just three or four frames. The polycount is also exponential, as even if a vertex stays in the same location for all ten frames, the model still stores that exact location for each frame.

So for your castle example, the flag may be the only part animated, but all the vertexes in the castle are also stored for each of the ten frames. If any optimization were to occur, that would be the first place to change. However, if you ask me, the whole G3D format needs to go. The IQM format, currently under consideration for use in MegaGlest, is looking like the best way to go. In addition to being a more advanced and capable model format, it would shift the need to develop importers and exporters (not to mention potential G3D optimization) away from us. It also would allow several features (with coding on our end) that G3D currently cannot, such as moveable cannons, ragdolls, and interchangeable armatures.

As for what can optimize G3D models, the G3DHack tool is the only one currently in existence. It's still officially an alpha, isn't truly cross-platform, and appears to be abandoned, but it's fully featured and contains a few optimizations, such as stripping unnecessary frames.
« Last Edit: 18 June 2016, 18:59:38 by filux »
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

Yggdrasil

  • Local Moderator
  • Ornithopter
  • ********
  • Posts: 408
    • View Profile
Re: Optimizing g3d files
« Reply #2 on: 15 September 2012, 11:41:34 »
So for your castle example, the flag may be the only part animated, but all the vertexes in the castle are also stored for each of the ten frames.
That's not true. The model of the castle is split into two meshes: The animated flag and the static building. The building has only one frame.

G3dHack can help you with splitting animated parts of a mesh from static parts. Maybe check out will's python script too.
https://forum.megaglest.org/index.php?topic=8560.0

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Optimizing g3d files
« Reply #3 on: 15 September 2012, 19:35:57 »
So for your castle example, the flag may be the only part animated, but all the vertexes in the castle are also stored for each of the ten frames.
That's not true. The model of the castle is split into two meshes: The animated flag and the static building. The building has only one frame.
Really? Because testing even the most recent version of the script, exporting a static cube with 250 frames was over 100 times larger (in filesize) than exporting the same model with a single frame. I was under the presumption that all frames are exported, regardless of whether or not they're animated.
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: Optimizing g3d files
« Reply #4 on: 17 September 2012, 05:34:19 »
I use the G3D format in my own hobby projects, including webGL.

In webGL scenarios, I've found merging meshes with the same number of frames is a big win; its about reducing the draw calls, rather than the vertices.  In browsers e.g. javascript and NaCL it can add a few FPS!  I speculate that it can help even mainstream OpenGL, and I it will never hurt.  It also gives G3dHack a second chance to optimise the model into fewer vertices.

There's the whole general area of simplifying meshes that is not really being explored, too.

I made a simple tool.  I found using it as a preprocessor with G3dHack gave the best returns.

https://forum.megaglest.org/index.php?topic=8560.0

Regards model formats, I think MD5 would be the biggest win.

I've thought about how to super-compress models before - http://williamedwardscoder.tumblr.com/post/17369540725/compressing-3d-game-models - but my current preference is just go for a text format and let a text compressor compress them!  MD5.  Bones not frames.

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Optimizing g3d files
« Reply #5 on: 17 September 2012, 07:38:07 »
Regards model formats, I think MD5 would be the biggest win.

I've thought about how to super-compress models before - http://williamedwardscoder.tumblr.com/post/17369540725/compressing-3d-game-models - but my current preference is just go for a text format and let a text compressor compress them!  MD5.  Bones not frames.
What about the IQM format, which MegaGlest is considering? While their webpage makes my eyes hurt, it seems to be similar to the MD5 format, and there seems to be something about the MD5 format that makes it impractical for MegaGlest (not sure what, Softcoder didn't elaborate). I'm curious as to the pros and cons of the two formats. Besides the fact that it's public domain, neither Softcoder's post nor the IQM website really explain the advantage of the format, and the IQM website doesn't have any documentation I can understand, just a format description in code that's beyond me. The only line in the "feature list" that I could even understand was "allows combined or separate mesh and animation files"; more or less on par with MD5.
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: Optimizing g3d files
« Reply #6 on: 17 September 2012, 09:44:08 »
Yeah as softcoder said, IQM is like MD5 but with better exporters from blender and unquestionable license.

So IQM all the way, I say!

The key thing is a move from keyframe to bone animation.

It is however preferable that the loader differentiates between single frame still meshes and multi-frame ones, and bakes-in the actual vertex positions.  Bones are not free runtime-wise in GPUs.

wciow

  • Behemoth
  • *******
  • Posts: 968
    • View Profile
Re: Optimizing g3d files
« Reply #7 on: 17 September 2012, 10:38:24 »
So IQM all the way, I say!

Its not nice to ditch the g3d format but IQM is superior.

Bones are not free runtime-wise in GPUs.

The gpu cost is easily handled on modern systems. I have a 3 year old mid-range pc, it can handle much more than Glest is capable of. Playing MoW I can have dozens of ragdolls and physics objects exploding over the screen without noticeable slowdown.
Check out my new Goblin faction - https://forum.megaglest.org/index.php?topic=9658.0

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Optimizing g3d files
« Reply #8 on: 17 September 2012, 20:40:15 »
So IQM all the way, I say!

Its not nice to ditch the g3d format but IQM is superior.
I don't think we would ditch G3D any time soon (after all, since G3D doesn't have bones, that means it's impossible to convert existing G3Ds to IQM unless you still have the blend file), but rather we'd support two different formats. And when that happens, provided that both engines can support these formats, there's really not much of a use for G3D.

As for performance, I'm under the impression that most modern games use bone-based animation systems. I really have to oppose the concept of "scaling down" Glest because of a few people who refuse to upgrade their five year old PCs. We don't expect people to be able to play current generation games like Battlefield 3, Civilization V, Mass Effect 3, Skyrim, or Portal 2 on a cheap netbook, why should we expect them to be able to play Glest? Just because it's free doesn't mean it has to be "behind the pack". Seeing that you can build a reasonable desktop capable of playing games around medium settings for as low as $400 (excluding peripherals), there's not much of an excuse to be complaining about how Annex is slow on your Netburst Celeron laptop. Off topic, but how come Glest has so little graphical settings in terms of performance? Most modern PC games have settings for texture size, particle detail, etc, allowing high end computers to set their graphics to "high" and low end computers to still be able to play the game (at the expense of aesthetics) by setting their graphics to "low". Glest, however, doesn't really offer much beyond setting the resolution of the game and the option to completely remove particles.
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

wciow

  • Behemoth
  • *******
  • Posts: 968
    • View Profile
Re: Optimizing g3d files
« Reply #9 on: 18 September 2012, 16:49:22 »
We will need to keep support for old mods with g3d format, but once iqm is implemented making new models in g3d will be pointless.

Also does MG support shaders? my copy has a folder of shader files in core/data/shaders.
Check out my new Goblin faction - https://forum.megaglest.org/index.php?topic=9658.0

tomreyn

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 2,764
    • View Profile
    • MegaGlest - the free and open source cross platform 3D real-time strategy game
Re: Optimizing g3d files
« Reply #10 on: 18 September 2012, 19:01:05 »
Please find the MegaGlest forums here: https://forum.megaglest.org/index.php?board=20.0
atibox: Ryzen 1800X (8 cores @3.6GHz), 32 GB RAM, MSI Radeon RX 580 Gaming X 8G, PCI subsystem ID [1462:3417], (Radeon RX 580 chipset, POLARIS10) @3440x1440; latest stable Ubuntu release, (open source) radeon (amdgpu) / mesa video driver
atibox (old): Core2Quad Q9400 (4 cores @2.66GHz), 8 GB RAM, XFX HD-467X-DDF2, PCI subsystem ID [1682:2931], (Radeon HD 4670, RV730 XT) @1680x1050; latest stable Ubuntu release, (open source) radeon / mesa video driver
notebook: HP envy13d020ng
internet access: VDSL2+

· · · How YOU can contribute to MG · Latest development snapshot · How to build yourself · Megapack techtree · Currently hosted MG games · · ·

ProfEclipse

  • Summoner
  • **
  • Posts: 49
    • View Profile
Re: Optimizing g3d files
« Reply #11 on: 27 September 2012, 09:10:41 »
Very thanks, guys ! All models have been optimized. So my system now runs faster & safer and does not usually hang up when I running game.

 

anything