Rendering G3D using OpenGL

[will]

If I interpret the code correctly, G3D models are rendered in the following way:

Each model is number of meshes.

Each mesh has a number of frames.

Each frame describes the coordinates of each vertex.  Each frame must contain the same number of vertices - if a wagon has a wheel that turns, then the wheel might have 10 frames, and each describes the wheel at a different point of rotation.  The position of the vertices for any given rotation are computed by the model making tools and not by glest.

When loaded in the glest engine, it is stored in a vertex array.

Each screen update, the appropriate frame is selected for each mesh, and the appropriate vertex array is drawn.

My immediate thought is this:

In modern OpenGL (which I am unfamiliar with; I did OpenGL in the old days) the vertex buffer would be a VBO and you'd have only one VBO for each mesh, and each frame of animation would be a transform described in a 'uniform'.

In classic OpenGL, as each frame of animation is immutable you'd place it in a Display List.  You would have only one Display List for the mesh, and describe each frame of animation as a matrix or, better, a series of transforms (rotate/translate/scale) to make the matrix.

In both the VBO/uniform and Display List/matrix approach, you can have tweening of frames very easily.

Now the classic Display List / Model Matrix transform is 'deprecated', but my understanding is it still goes like a rocket.

Am I retreading old ground that others have thought about and discarded?  Or does it make sense to move from vertex buffers to Display Lists, and to compute matrix transforms for the 'unrolled' frames in the G3D models?

(Background: http://www.gamedev.net/community/forums/topic.asp?topic_id=558817 Display Lists are 10x faster than the vertex arrays glest seems to currently be using...?  My expectation is that Display Lists are likely a bigger gain for typical users than those numbers given in that gamedev thread suggest.)

[silnarm]

If I interpret the code correctly, G3D models are rendered in the following way:

Each model is number of meshes.

Each mesh has a number of frames.

correct,

Each frame describes the coordinates of each vertex.  Each frame must contain the same number of vertices - if a wagon has a wheel that turns, then the wheel might have 10 frames, and each describes the wheel at a different point of rotation.

correct,

The position of the vertices for any given rotation are computed by the model making tools and not by glest.
When loaded in the glest engine, it is stored in a vertex array.

incorrect,

Each screen update, the appropriate frame is selected for each mesh, and the appropriate vertex array is drawn.

Each frame, the units 'progress' (a float) is used to select two frames, based on the anim-speed of the skill, and the vertex array sent to the video card is interpolated from these two frames (see InterpolationData), depending on the movement involved, very few frames can be used, and smoothly animated in Glest (by linear interpoletion, circular movements require more frames to look correct).

My immediate thought is this:

On account of the above, VBOs for units is not going to work, all those frames of vertex data is far too much to be putting in video memory.

VBOs could well be worthwhile for for tileset objects, which aren't animated, and account for a fair proportion of the typical rendering time. Terrain and water rendering desperately needs to be converted to using VBOs and using GLSL for the actual rendering, this will be happening in the near future. This has been semi-discussed recently on the dev mailing list, sign up!

[will]

On account of the above, VBOs for units is not going to work, all those frames of vertex data is far too much to be putting in video memory.

VBOs could well be worthwhile for for tileset objects, which aren't animated, and account for a fair proportion of the typical rendering time. Terrain and water rendering desperately needs to be converted to using VBOs and and using GLSL for the actual rendering, this will be happening in the near future. This has has been semi-discussed recently on the dev mailing list, sign up!

This interpolation approach seems very CPU and RAM intensive.

With display lists for the meshes and matrices for the frames, you'd use much less of both surely?

[Yggdrasil]

It's not possible to express a character animation with matrix transformations, e.g. walking. Too many parts of the mesh move differently.

Display lists have the main disadvantage that they are static const so to say. After generating you have no way to manipulate them. VBOs can be static too and should be as performant as display lists. I'm currently trying to use static VBOs for static objects (only one frame). VBO can also behave like vertex arrays (data is send each rendering). One can also update only parts of the buffer, no need to hold all frames in video memory. Even "direct" mapping is possible (loading from disk directly into VRAM, no copy in RAM). So, i think VBO is a better and more flexible approach, and as we already use vertex arrays it should be quite easy to add them.

[will]

Yes, I'm beginning to grasp the subtle tradeoffs between VBO, vertex arrays and display lists now.

I wrote a quick renderer in python so I could play about a bit:

Code: [Select]

[img]http://img20.imageshack.us/img20/5610/glest1.jpg[/img]
My initial thoughts were about level-of-detail - calculating simplified meshes to be used at further zooms - but I think to that you *can* often turn the current animation interpolation into single mesh lists with matrices.

I will play with both.

[will]

It's not possible to express a character animation with matrix transformations, e.g. walking. Too many parts of the mesh move differently.

That's quite true.  But there are many models with the opportunity to dramatically reduce the amount of data by sharing at the mesh level.  I've been writing some code to analyse them here:  https://github.com/williame/GlestTools

For example, take the nice catapult (which is perhaps more geometric-looking than normal):

I've differed each mesh each frame and in the table below all the ys are frames that are exact duplicates

Mesh4 8 228 432 y y y y y y y IMMUTABLE
Mesh4 8 393 1200 x x x x x x x mutable
Mesh4 8 9 24 y y y y y y y IMMUTABLE
Mesh4 8 86 264 y y y y y x y mutable
Mesh4 8 29 108 y x x x y x x mutable
Mesh4 8 67 216 y y y y y y y IMMUTABLE
Mesh4 8 116 168 x x x x x x x mutable
Mesh4 8 116 168 x x x x x x x mutable

The code to find duplication can only get cleverer.  It could be extended to detect scaling, and it could re-subdivide models to get optimal packing, and also detect mirroring and, importantly, hidden triangles.  All these changes save disk space, but importantly save RAM and can give the GPU less to render.  Big performance wins I think are worth chasing.

We'd need to add a version 5 of the G3D format to support any of this, of course.

My thought is its an optimisation step available in my glest_mod_pack.py tool.

[Yggdrasil]

I've differed each mesh each frame and in the table below all the ys are frames that are exact duplicates

Mesh4 8 228 432 y y y y y y y IMMUTABLE
Mesh4 8 393 1200 x x x x x x x mutable
Mesh4 8 9 24 y y y y y y y IMMUTABLE
Mesh4 8 86 264 y y y y y x y mutable
Mesh4 8 29 108 y x x x y x x mutable
Mesh4 8 67 216 y y y y y y y IMMUTABLE
Mesh4 8 116 168 x x x x x x x mutable
Mesh4 8 116 168 x x x x x x x mutable

Interesting. Searching for simple transformations is probably a bit too complex, especially later in the file format. Immutable meshes could be interesting.

We'd need to add a version 5 of the G3D format to support any of this, of course.

That's the main obstacle here. There are also other possible optimizations like reorganizing the vertex data to use triangle strips which, as far as i know, is the fastest draw primitive. The half-edge data structure is a good way to get this done. I'm not sure if this helps much as our models are quite low poly.

[John.d.h]

If we had a way of interpreting rotation as such, we could save a whole lot of frames there while improving animations, as wheel-like animations currently involve a lot of shrinking and growing, and thus they take quite a few frames to look decent.  Just throwing that out there.

[will]

Encouraging indeed

[John.d.h]

We'd need to add a version 5 of the G3D format to support any of this, of course.

That's the main obstacle here. There are also other possible optimizations like reorganizing the vertex data to use triangle strips which, as far as i know, is the fastest draw primitive. The half-edge data structure is a good way to get this done. I'm not sure if this helps much as our models are quite low poly.

If that gets done, then we could potentially get a big boost when it comes to tilesets.  As I recall, most of the models from the default tilesets are g3d version 2, and trees eat a lot of performance.

[will]

I had really understated the amount of trivial-to-find duplication of whole-meshes - a fix to my diff-code and quite a lot more pop up.  I seem to find all the wheels and things, which means that nice smooth rotations would be possible.

The new version of the tool renders the changing meshes in red (changing) and green (unchanging) parts, and shows that it seems fruitful to divide those meshes that do change into large unchanging parts and changing parts.

My thoughts for a new format would be:

• The model has a duration value.
• Each model has a vertex and normals array.  This is per-model, not per-frame.
• That each sub-model (using this term as its not a mesh) has an arbitrary number of frames, each frame has it's own time within the total model's duration.  Therefore, each sub-model does in a model does not need the same number of frames.
• Each frame in a sub-model is made up from an arbitrary number of operations.
• Each operations is a matrix and offsets and lengths into the model's vertex and normals array and the op-code to use - triangles, triangle strip, quads etc.

Existing G3D models would be converted to the new format by glest_mod_pack.py whilst packing the mod.  Changing the export tools for future models might be nice, but there seems plenty of scope for retro-repacking the old models.

And the code that draws this uses VBOs etc hopefully from the beginning.

[will]

Here is another view of the catapult attacking something:

Code: [Select]

[img]http://img151.imageshack.us/img151/3180/glest1.png[/img]
All the normally-rendered meshes are meshes that haven't changed that frame (other than rotation/translation).  They are animation frames we can represent by matrices (smooth spinning here we come!).  This will save buckets of RAM (hopefully).

The red meshes are meshes which, in that frame, change compared to the previous frame.  The green parts of the mesh are parts that, compared to the previous frame, don't change (other than rotation/translation).  Although this shot doesn't show much green, my idea is to split that out into a separate mesh.  Saving a bit more RAM hopefully.

Here's a norse crossbow attacking something, and it shows the large unchanging parts of the mesh that could be split into separate meshes:

Code: [Select]

[img]http://img219.imageshack.us/img219/6836/glest4.png[/img]
And some of the red looks like float precision problems in my comparison.  I'll have to tweak that.

Here is a humanoid:

Code: [Select]

[img]http://img30.imageshack.us/img30/797/glest2.png[/img]
There is a bit more green.  But the thing that strikes me is the symmetry; I will add a check to see if parts of a mesh can be represented by reusing parts of the mesh - or another in the model - with a matrix.  If things are as symmetrical as they seem, this will save buckets more RAM.

Finally, lets peep under that skirt:

Code: [Select]

[img]http://img593.imageshack.us/img593/6013/glest3.png[/img]
Our reaper has legs!  Culling always hidden surfaces (on the assumption that things cannot be viewed from beneath; is there a way to infer this, or must the modder tell the tool?) will save quite a bit more RAM, and hopefully be much faster too.

[Omega]

Ideally, the modeller wouldn't make hidden parts of the model, such as legs in the first place (unless they would be used in a different animation of the model).

[silnarm]

This is indeed quite fascinating, but I'd have to agree with Yggdrasil, it's a bit overcomplicated to be splitting off parts of the mesh and transforming them separately.

Could you tweak this tool too show only the completely immutable parts (across all frames) ? Splitting that component out into a separate mesh if justified (big enough) would certainly be worthwhile, as it could be put in a (static) VBO.

The biggest problem with the current system is not RAM use or the speed of interpolation, modern machines have lots of system RAM, modern CPUs are fast, modern GPUs are fast, the system bus is the killer for us, because the vertex arrays all have to be sent from (in OpenGL speak) the client to the server. Moving static data into VBOs is likely to see the best improvements atm.

[will]

On account of the above, VBOs for units is not going to work, all those frames of vertex data is far too much to be putting in video memory.

That is my major motivation for looking for duplication to de-dupe.

And of course I'm using integrated graphics, so filling up the RAM I free with computed level-of-detail alternatives is good stretch goal...

So here are some stats from my script the whole of magitech:

=== Input ===
520 models
3314004 vertices in (75.9 MB)
1045767 indices in (4.0 MB)
1818991 vertices out (41.6 MB)
1045767 indices out (4.0 MB)
134520 matrices out (8.2 MB)

There are plenty of tricks its missing, and potentially lots of bugs and false-positives that it would have to avoid deduping.  But I think the overall number is indicative unless it also starts to merge models.

So it can go from 80MB down to 50MB.  However, on reflection, 80MB isn't so scary for geometry?  What's keeping us out of VBO territory today other than writing the code?

[Omega]

I may not be totally understanding this, but if it increases performance without any cons, and you were capable of programming to figure this out (you seem to be a good programmer based on what I've seen), why don't you download the SVN and impliment this yourself? Then the GAE team could take a look and possibly merge it into Glest? (and on that said topic, perhaps the changing texture coords that you pointed out in a different topic).

[John.d.h]

I may not be totally understanding this, but if it increases performance without any cons, and you were capable of programming to figure this out (you seem to be a good programmer based on what I've seen), why don't you download the SVN and impliment this yourself? Then the GAE team could take a look and possibly merge it into Glest? (and on that said topic, perhaps the changing texture coords that you pointed out in a different topic).

+1

GAE is open source and Will obviously knows what he's doing in this regard, unless it's just a language barrier (Python vs. C++).

[silnarm]

So it can go from 80MB down to 50MB.  However, on reflection, 80MB isn't so scary for geometry? 

Not so scary, but most model's in mods are probably bigger, and any VRAM saved on vertex data could be used for textures, so it would certainly be worthwhile.

What's keeping us out of VBO territory today other than writing the code?

Mostly its because someone would have to write the code, also the morph animation as done now means animated meshes simply wouldn't benefit, you'd have to update the entire VBO every time you wanted to render a unit of that type, so it would effectively be the same as using vertex arrays.

Yggdrasil is looking into using VBOs for static (single frame) meshes, if you can reduce the load for animated meshes, please do!

[Yggdrasil]

So it can go from 80MB down to 50MB.  However, on reflection, 80MB isn't so scary for geometry? 

Not so scary, but most model's in mods are probably bigger, and any VRAM saved on vertex data could be used for textures, so it would certainly be worthwhile.

We could also use compressed textures on the graphics card to lower VRAM use. I think megaglest started using this recently. It's quite standard for commercial games. 0ad for example uses .dds files for textures which are stored compressed on the graphics card, unlike png which is transfered uncompressed. Be aware that this is basically a dump of a DirectX structure. It's also possible to let the graphics driver compress textures on-the-fly, obviously adds overhead but should be simpler. Btw, the compression is lossy.

Yggdrasil is looking into using VBOs for static (single frame) meshes, if you can reduce the load for animated meshes, please do!

I haven't done much coding. The only quick test i made resulted in a lower framerate. I obviously did something wrong. So if someone else wants to work on this and has some time at hand, feel free to work on it. I don't have much time currently.

[will]

With much prompting by silnarm, I got pure GPU-side animation going.

g3d_stats.py at

Code: [Select]

[url=https://github.com/williame/GlestToolsy]https://github.com/williame/GlestToolsy[/url] includes a renderer, and this renderer stores all frames and the indices and normals and such in VBOs.

When drawing a model, it sends the one of the frames and the normals through as usual - glVertexPointer and glNormalPointer - but then sends the other frame through as glColorPointer (and you might find space for the normals in glSecondaryColorPointer or something).

The vertex shader now does the interpolation e.g. gl_Position = gl_ModelViewProjectionMatrix * mix(gl_Vertex,gl_Color,lerp);

Shockingly easy, but shocking too how I struggled to solve it. I hope this gives inspiration.

Now I'm off on for a holiday break, and whilst I don't doubt I'll sneak a peak at the forums, me coding this in GAE will have to wait until the new year.  Unless anyone else wants to implement this in the meantime *wink* *wink*

[wciow]

Thanks Will 

Since Hailstone started introducing shaders I've been getting reaquainted with baking normal maps in Blender. A tool that can display the normal maps for G3D models without having to fire up the whole engine will certainly be useful 

[Omega]

Awesome Will! Good job. I have high hopes and anticipations for your work after the holidays.

For normal maps, I had in mind a separate texture that would be read and intrepid for the normal map. Can the engine do it? Well, if other games with far more detailed graphics can, so can we! Of course, it sounds challenging, and graphics coding is nothing I have experience with...

[Yggdrasil]

Nice work will! I'll dive into the python code later. Hopefully my rudimentary python skills are enough...

[will]

For normal maps, I had in mind a separate texture that would be read and intrepid for the normal map. Can the engine do it?

<my comment corrected after looking at the code>

The engine seems to support it right now - svn blame suggests its silnarm's work?

For version-3 models, if a texture exists that has _normal appended to its filename then that is used as a texture map.  In version-4 models, the name of the normals map is in the mod.

I have not found a model that does use it, though.

But if you're making such normal maps - I prefer the term bump maps - then you can presumably use gae_g3dviewer.

[will]

An anecdote about performance on my Intel integrated GPU on my laptop

I am drawing a 10K polygon model that fills the screen.

I could find no measurable difference between calling glVertex/glNormal/glTexCoord for every triangle, or using glVertexArray.  They were both as slow as each other, and was around 23 fps.

Using VBOs however, and it shoots to 60 fps - my screen refresh rate, and I have vysnc glSwapBuffers() so that's expected.

That its such a big deal on integrated graphics I had not appreciated.  I hope this is an illustrative datapoint for both MG and GAE planning.