Rendering G3D using OpenGL

[hailstone]

For the left my guess is that since it's rendered to 2D it was done as a post process effect in an image manipulation program.

Actually, it appears that way in the game as well, and the game doesn't use any pre-rendered sprites.

Woops. I thought the left one was from an earlier version of Warcraft that is 2d.

Does it seem plausible to use Blender particle data in some future model format in GAE?  I think that would be far simpler and more intuitive for everyone involved.

I think it might be possible to add named nodes/vertices that could be referred to in xml - like an effect point. Then you move those as a part of the animation. Although like Omega said it's more likely we will adopt a skeleton format but I don't plan to do it in the near future so I'm not sure what's going to happen.

[will]

G3D meshes also have names - this facility could be used to tie particle effects to the loci of a model perhaps.

Think how nice this would be for jazzing up the various destruction animations when they are destroyed by a fire arrow etc

G3D is MD2/3 equiv and has many advantages in an RTS environment - I'll gather my thoughts and post to the turrets thread why I think we ought to do both

[will]

On the subject of powers-of-2 textures.

The experts are strongly in favour of limiting textures to powers of 2: http://gamedev.stackexchange.com/questions/7927/why-would-you-use-textures-that-are-not-a-power-of-2

Therefore, I suggest the engines and the export scripts police this to avoid any non-power-of-2 textures getting distributed and causing us problems and/or stopping potential texture compression and mipmapping code in the future.

[silnarm]

Couldn't agree more, powers of two are always better than non powers of two.

However, the export script can't do anything about it, texture co ordinates range from 0.0 to 1.0 remember ;-)

So, the real question is, what should the engine do about it ???

Options:

• Just write a warning to the error log, and use the non-power of two texture.
• Write a warning to the error log, then load and pad the texture, adjusting the tex co-ords as needed.
• Refuse to load the texture at all, write to log and use the default texture, thus effectively forcing the modder to fix the problem.

Personally I'm in favour of the latter, but I imagine modder's may disagree on this ??

[John.d.h]

Personally I'm in favour of the latter, but I imagine modder's may disagree on this ??

That's cool with me.

[hailstone]

However, the export script can't do anything about it, texture co ordinates range from 0.0 to 1.0 remember ;-)

It can. The trick is not to use the texture coordinates http://www.blender.org/documentation/246PythonDoc/Image.Image-class.html

It could go something like this:

Code: [Select]

if material.getTextures()[0]:
    image = material.getTextures()[0].tex.getImage()
if image:
    width, height = image.getSize()
    if !isPowerOfTwo(width, height):
        print "Texture isn't power of two, can't export."
        exit()

It should also be checked in the engine in case someone uses an export script without the check. Unless we change the model version. If we were to do that we should also export tangents, normal map filename and specular map filename.

[silnarm]

However, the export script can't do anything about it, texture co ordinates range from 0.0 to 1.0 remember ;-)

It can. The trick is not to use the texture coordinates http://www.blender.org/documentation/246PythonDoc/Image.Image-class.html

Cool, this would be worthwhile then...

It should also be checked in the engine in case someone uses an export script without the check. Unless we change the model version. If we were to do that we should also export tangents, normal map filename and specular map filename.

And also because the texture can be changed at any time after the export!

As for a new model format, I think just using Collada or MD5 would be the way to go, but I would sincerely like to see a G3D exporter that actually supports G3D v4, and exports normal and spec maps (G3D v4 was of course mostly made with Duel-Field in mind, and supports five texture slots, I recently changed the meaning of the last two though, from reflection and colour-mask to custom1 and custom2, as I don't think we need to support reflection or colour-mask maps in an RTS, and I needed custom textures for a custom shader for Constellus).

Tangents don't need to be exported from Blender, the engine code calculates them perfectly (I did some research, Martiño's code even averages them at shared verts, it's as good as we can get I believe). I have your bump mapping shader working correctly, though the 'base' lighting is still dodgy, but that can be copied (mostly) verbatim from the diffuse only shader.

This is all in a local branch atm, I have some issues to sort out because of the VBO code, and I wasn't planning to push the branch, but just merge back when everything is done.  But if you want to take a look / have a tinker yourself, let me know, I'll push the branch.

[Omega]

I'm with the hard way. Reading that afformented article, and thinking that mipmaps are probably very important, we should just take the strict approach to ensure that modders always have power of 2 textures. Face it, a warning won't make anyone but the best of the modders even pause. However, on the other hand, what about legacy mods that already have textures that aren't powers of 2 and have been out of creation for a long time? We risk not being able to play them, so this option should be toggleable in the INI.

[wciow]

Just my 2p on all of the above:

I see no reason to bother with non power of two textures. Keep things the way they are.

I see no reason not to implement MD5 alongside G3D for those who want to use it. Its just giving modders more choice and may come in handy for those limited situations where a bone structure is preferable to vertex frames.

It would be nice to allow the G3D exporter to export bump/specular maps for models, but I cant really see any advantage over Blenders native export of bump maps (tangent space maps in Blender) and simply adding "normal" to the filename.

[will]

I see no reason to bother with non power of two textures. Keep things the way they are.

Right now, non-power-of-2 textures are supported - and they shouldn't be, is the general consensus.

It would be nice to allow the G3D exporter to export bump/specular maps for models, but I cant really see any advantage over Blenders native export of bump maps (tangent space maps in Blender) and simply adding "normal" to the filename.

The filename trick only works for G3D v3 if I read the code correctly.

[silnarm]

It would be nice to allow the G3D exporter to export bump/specular maps for models, but I cant really see any advantage over Blenders native export of bump maps (tangent space maps in Blender) and simply adding "normal" to the filename.

The filename trick only works for G3D v3 if I read the code correctly.

Originally, yes, and in the master branch maybe still (?). I have a function in my render branch, Mesh::loadAdditionalTextures(), which is called after normal texture loading in both load() and loadV3(), and loads in extra textures with _normal, _specular, _custom1 & _custom2 added to the diffuse tex name.

A workable solution, but I'd prefer a new exporter myself... of course I don't model, so I guess I shouldn't care so much

I should probably push that branch, but it's a bit messy still, so I'll wait a little longer...

[will]

Any status or stats on the migration to VBOs and shaders?

I'm beginning to feel more confident in my understanding of the new OpenGL pipeline, and there's a fundamental lesson I've learnt - both empirically when trying to render very complicated spherical meshes and from this excellent presentation by Nvidia (which is the same of all GPUs in general, including my Intel integrated one): that you want to avoid drawing the map tiles in the manner we are now.

At the moment, its:

Code: [Select]

for each tile:
   if tile not visible:
       continue
   if texture != prev_texture:
       glBindTexture(..,texture)
   glBegin(GL_TRIANGLE_STRIP)
   ... draw quad
   glEnd()
   prev_texture = texture

Whereas what you want to do is to divide the map up into blocks of some size - say 20 by 20 or more.  Combine - at load-time - all the tile textures into a single texture.  In each of those blocks of tiles, prepare all the vertices in a VBO.  Now your map drawing would be:

Code: [Select]

glBindTexture(...,all_tiles_texture)
for block in map_blocks:
    if block not visible:
        continue
    .... draw vbo

You might divide it up so you have a set of blocks for each tile or batch of tile textures, or you might do as I found fastest for my globes by having all the vertices in a single VBO and using range-elements drawing.  But the fundemental thing is to move from drawing each tile individually to drawing big blocks of map in a single call.  The GPU is very good at skipping those fringes around the meshes that fall outside the screen area.  You can go further by putting these blocks of tile vertices into a single VBO and using glMultiDrawElements to draw all those blocks.

Doubtless you're all way ahead of me on this line of thinking, and I'm just stating the obvious

[ChupaReaper]

As a modder and a huge fan of the md5 format, I vote md5, just to have my say, I've tried using COLLADA with 3ds max, it's a bit of a pain, md5 is clean and open, also it's nice to pop in there with a text editor to change texture names last minute when needed! At the minute I can't run my mod with two factions again due to having too many g3ds (by removing some models it works again), I can strip away more animation frames but eventually there'll be no animation left, would md5 fix this with skeletal aniamtion?

[will]

A bone-based MD5 is slower to render than a keyframe-based format like MD2 or G3D.

Slowness now might be based on render perf, pathfinding perf or network lag.

If its on rendering, there are likely things to be improved.

Or we can just all leave Glest as it is now, and wait for faster cores and GPU

[silnarm]

Any status or stats on the migration to VBOs and shaders?

...

You might divide it up so you have a set of blocks for each tile or batch of tile textures, or you might do as I found fastest for my globes by having all the vertices in a single VBO and using range-elements drawing.  But the fundemental thing is to move from drawing each tile individually to drawing big blocks of map in a single call.  The GPU is very good at skipping those fringes around the meshes that fall outside the screen area.  You can go further by putting these blocks of tile vertices into a single VBO and using glMultiDrawElements to draw all those blocks.

Doubtless you're all way ahead of me on this line of thinking, and I'm just stating the obvious

TerrainRenderer2 (-test tr2) puts all of the vertex attribs for the entire map in a single VBO, the indices are collected in a vector and sent through as a vertex array.

from terrain_renderer.cpp [TerrainRenderer2::render()]

Code: [Select]

// build index array
m_indexArray.clear();
int tileCount = 0;
SceneCuller::iterator it = culler.tile_begin();
for ( ; it != culler.tile_end(); ++it) {
Vec2i pos = *it;
if (!mapBounds.isInside(pos)) {
continue;
}
int ndx = 4 * pos.y * (m_size.w - 1) + 4 * pos.x;
m_indexArray.push_back(ndx + 0);
m_indexArray.push_back(ndx + 1);
m_indexArray.push_back(ndx + 2);
m_indexArray.push_back(ndx + 3);
++tileCount;
}

then after all the gl state/array/texture setup, its one call,

Code: [Select]

// zap
glDrawElements(GL_QUADS, tileCount * 4, GL_UNSIGNED_INT, &m_indexArray[0]);

In order to do it this way, all the textures are splatted as normal, and then copied onto one master texture (SurfaceAtlas2). So it chokes on big maps (master tex would need to be too big, gl implementation dependant), so big maps are going to need to be split into multiple 'blocks'.

It also currently wastes lots of memory... the 'old' splatted textures aren't even deleted atm

This will be cleaned up sometime soon, for a possible 'tr3' doing the splat in a shader is the 'obvious' way forward - so we would then only need the base textures and save a bunch of texture memory, but at the cost of more vertex attribute memory & much more fragment processing...

[will]

That is an interesting strategy

How does that cope with different heights for tiles and cliffs and such?

Putting all the tile images into a single texture and then putting all the vertices with their texture coords into a VBO and drawing that ought to be amazingly fast too, and use less texture memory overall.

[will]

Hopefully useful prototyping: https://forum.megaglest.org/index.php?topic=7364.0