Author Topic: [TESTERS WANTED] Speeding up rendering  (Read 6747 times)

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #25 on: 11 July 2011, 10:44:19 »
Titi, is the slowdown also precent with my original patch?  What is your hw?

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: [TESTERS WANTED] Speeding up rendering
« Reply #26 on: 11 July 2011, 11:19:10 »
The slowdown is completly gone with latest svn. Now I have about 30-90 fps more!
The hardware I used to test is a Geforce 9600GT. I think the problem was that teh whole terrain was rendered before.

Talking about terrain rendering in General:
I am not shure the surface atlas is the ultimate way to go! Here is an alternative way which I already discussed with softcoder ( and I hope he understood what I said with my bad english ;D ) Maybe we can mix some of the ideas to get a good result.

first idea:
We use shaders to render the surface
Each surface type has its own layer and an alpha map which is used for blending using shaders later. (So blending too is no longer made manually by the CPU!)

There is a way to use one alpha map for three layers by using RGB alphas where the red/green or blue stands for one layer. By this we can render 3 of the layers at once in only one render path . So we will need 3 render passes to render all layers.

Problem:
If we would do so we would always render the whole surface! This can be solved by working tile based, but with much bigger tiles than before ( I suggest 16x16 of the former cells ).  This means we will have unique alpha maps for each of these squares, but we use the same layers for all of them.

second idea:
If we work "big tile based" like suggested to solve the problem mentioned above we should have one big texture for each layer. So one big texture which is used for 16x16 cells instead of only 1 small texture for 1 cell before. The typical size is 1024x1024 for these 16x16 cells ( (16*64) * (16*64)  because we typicaly use 64*64 bits to texture one cell ) . This big texture for the one layer is used for ALL 16x16 cell blocks!

How to get this big layer texture?
We calculate it by putting the small texture pieces we have now on the new texture, respecting their given probabilities for usage from the tileset xml.

This will also allow us to use bigger texture tiles up to 1024x1024!
For example imagine we use 128x128 textures:  We can use them for 2x2 tiles now. This means we only need 8x8 of them to fill the 16x16 cell texture.
By these bigger texture pieces we are able to show more variety! This is not meant to raise the resolution of the single cell texturing, which is described in the next step:

If we want to use a more detailed texturing ( like for example 128x128 pixels for each cell ) we can introduce a new setting in the xmls( for each layer tyoe ). This setting can be called something like  texture factor. typically/default is 1 which means 64x64 bit is used to texture one cell.
2 means 128x128 is used
3 means 256x256 is used and so on .... ( or something similar like the power of 2  saying 6,7,8 ....)
example: if we use factor 2 (128x128) we will have a 2048x2048 texture for the 16x16 cell block.

comments?

« Last Edit: 11 July 2011, 12:09:44 by titi »
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #27 on: 11 July 2011, 11:55:45 »
The slowdown is completly gone with latest svn. Now I have about 30-90 fps more!
The hardware I used to test is a Geforce 9600GT. I think the problem was that teh whole terrain was rendered before.

i think that was the purpose, but as will said it depended on a uhm feature, wich i dont remeber the name off.

but YAY!, More FPS!
WiP Game developer.
I do danish translations.
"i break stuff"

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #28 on: 11 July 2011, 21:28:26 »
First, with the latest SVN, rendering on my integrated Intel has dropped like a stone down to 10fps (20 before, and 24 with my original patch).

Its easy to mix up how to describe multi-layered maps with how to draw them.  You can have both.  Also, you don't need shaders to do it.  That's entirely optional, and until you are into geoclipmapping you don't need them.

In my prototype I talked about 'layers' being per texture but in fact they should be per combination of textures, if you are stacking them.

The texture atlas is a way of reducing the number of GPU 'batches' which is a major thing.  Its a very fundamental optimisation.  Its what you do if you have one layer per cell or 10.

So back to my original prototype.  I created a grid of w*h built from scanlines not unlike an image.

If you instead create it in tiles - say 16x16 chunks - you can decide which chunks are in any way visible and only draw them.

Furthermore, if you arrange these in a hierarchy and you put all the vertices and indices into the same arrays, so each tile is a continuous subsection in the array, you can draw whole parts of the hierarchy in a single call.  This hierarchy is called a quadtree.

I would recommend that the quadtree is per texture, and that use a texture atlas to reduce the number of batches.  The hierarchy tracks the start and stop offsets of each part in each texture's layer.

This ought to draw exceedingly quickly - pretty damn close to optimally - on all hardware and with as many layers as you chose to have.

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #29 on: 15 July 2011, 00:12:39 »
Drum roll please...

Here is a patch that renders the landscape using a quadtree and visibility culling.  It ought only render what's basically visible.

What is basically visible is still a hundred or more textures, instead of 1 or 2 texture atlases.  So this really multiplies the state-changes.

Testing should be as simple as checking out svn revision 2475 and then saving these 4 files from github:

https://raw.github.com/williame/megaglest/rendering_maps_2/source/glest_game/graphics/renderer.h
https://raw.github.com/williame/megaglest/rendering_maps_2/source/glest_game/graphics/renderer.cpp
https://raw.github.com/williame/megaglest/rendering_maps_2/source/glest_game/graphics/map_renderer.h
https://raw.github.com/williame/megaglest/rendering_maps_2/source/glest_game/graphics/map_renderer.cpp

And then compiling it.

Is this faster than the current SVN?  Is this faster than the current MG release?

How is the speed on small maps vs large maps?  On maps with a lot of vegetation and maps that are spartan?

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #30 on: 15 July 2011, 11:48:41 »
EDIT:Will, i think Windows and Visual Studio need's to manually ADD the map.rendere.h files in the Solution, im not sure the .bat file will automatically pick that up, as i figure it runs from a SLN file.

Allright, i have downgraded to 2475. (or upgraded w/e), and im going to take a few games with this revison then update it with your patch.

CPU:E6600 2,4Ghz
Memory 2GB ddr2 800Mhz Ram.
GPU:8800GTX 768Mb Vram
OS:Win7
OpenGL:Version 3.3
OpenGL renderer:GeForce 8800GTX/PCI/SSE2

 First number is wills patch second number is revison 2475.

Map:Conflict
No Fog of war:RenderFPS 210, SVN:renderFPS 240-300
With Fog of war:RenderFPS 230-240, SVN:renderFPS 350-370

Map:Land of Plenty2
No Fog of war:RenderFPS 40-45, SVN:renderFPS 160
With Fog of war:renderFPS 50-60, SVN:renderFPS 240

« Last Edit: 15 July 2011, 16:36:22 by Coldfusionstorm »
WiP Game developer.
I do danish translations.
"i break stuff"

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: [TESTERS WANTED] Speeding up rendering
« Reply #31 on: 15 July 2011, 15:20:13 »
My results: Setup was always only me playing egypt with tileset desert I clicked away the "you won" message and then I waited a while until the fps were stable:

My graphics card:
OpenGL renderer string: GeForce 9600 GT/PCI/SSE2/3DNOW!
OpenGL version string: 3.2.0 NVIDIA 195.36.24

First number is 3.5.2, second current svn and third with wills patch

Map conflict (64x64):
with fog of war: 190 fps / 235 fps / 185 fps
without fog of war: 170 fps / 206 fps / 167fps

Map Land Of Plenty( 256x256):
with fog of war: 157 fps / 185 fps / 25 fps
without fog of war:  107 fps/ 131 fps / 24 fps

The most obvious thing is that wills version renders too much on big maps ( I think that is the reason )...

Another thing I noticed is maybe a BUG:
On all 3 versions I sometimes had the state that the game starts and no objects were rendered.
If this happens once you touch the mouse and everything is ok BUT:
fps are much lower then! for example:
svn: on map Land of plenty: only 125 fps instead of 185 fps for all other starts!
Or version 3.5.2 on map Land of plenty:
149 instead of 157.
or wills version on map Conflict:
171 instead of 185 fps.

Is it maybe a missing  initialisation of what is visible and what not?
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #32 on: 16 July 2011, 12:10:43 »
Thx, brilliant feedback!

I am beginning to have a feel for what will be fastest, and why.

Was my code noticeably faster when you looked at the top-left of the map?

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #33 on: 16 July 2011, 15:38:28 »
I did not notice to be honest.
WiP Game developer.
I do danish translations.
"i break stuff"

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: [TESTERS WANTED] Speeding up rendering
« Reply #34 on: 16 July 2011, 17:44:59 »
no, no difference for me too, nearly same results on top left.
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #35 on: 17 July 2011, 21:38:27 »
I'm afraid I have a new version of my patch; I'd anticipate the numbers to be within spitting distance of the SVN values, and hopefully better than 3.5.2

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #36 on: 19 July 2011, 10:44:07 »
Well, where is the patch?
WiP Game developer.
I do danish translations.
"i break stuff"

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #37 on: 19 July 2011, 13:17:20 »
 Same links as last time

However, i think softcoder's approach is fastest...

We still badly want interleaved vertex arrays and texture atlas of course.

This might be an interesting datapoint for 0ad.

Philip

  • Guest
Re: [TESTERS WANTED] Speeding up rendering
« Reply #38 on: 19 July 2011, 15:53:32 »
Data is always interesting :). 0 A.D. already interleaves vertex attributes (everything uses a VBO with a 32-byte-aligned struct per vertex, plus a separate VBO for indexes). I expect it would be tricky to use texture atlases for terrain, because currently we rely on texture wrapping (and adjacent tiles share their vertex data so we can't just wrap the generated UVs) - I'd be interested in ways to address that. (Texture arrays would probably be ideal but aren't widely supported by half of users :()

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #39 on: 19 July 2011, 16:11:29 »
Key lesson - pre-making vbos of tiles in blocks was slower to render than creating  vbo with the exact visible tiles every time the view changes, both high-end and low-end cards.

Also, glDrawRageElements was dog slow on nvidia - speculation?

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,238
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #40 on: 20 July 2011, 05:20:38 »
I have read in a number of links around the web that in general glDrawRageElements performs worse (usually) than just an array since many drivers are more likely better able to cache array as opposed to range elements for some reason. I think our current exercise has proven this somewhat, but always more room for performance improvements.

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #41 on: 21 July 2011, 00:01:05 »
[TESTERS WANTED AGAIN]

My quadtree with glDrawArrays was nearly as fast as SVN, but not quite.

However, when looking carefully at the area of the screen being updated, I was a bit alarmed - I think the current visible area computing code is being a bit too conservative and drawing massive amounts of invisible objects.

So I quickly ported the kind of visible area code I had in mind for glestng; here's a before and after:



My patch is on github as usual.

My code is very much a prototype.  It has three missing bits:

  • doesn't special-case if the camera is looking at the horizon - the plane-line intersection code can detect this by looking if d is vanishingly close to 0 or if (n/d) is outside 0..1.
  • the quad iterator wants to the quad to be map-orientated so you'd have to swap around the coordinates as you pass through each quadrant
  • the code doesn't round up the right way - you ought to include partially visible rows too

Neither of these three omissions invalidate the approach, I think.  But its a late coding session and I need to get some sleep.

I hope this is helpful.

Testing: (remember that spinning in free camera angles is known to be broken, as is missing tiles on edge of screen; we are interested in indicative performance and not exact correctness)

The patch is based on svn 2487.  It is these two files:

https://raw.github.com/williame/megaglest/8f68c9829dabfaf057fb338d2dd44201172a0da7/source/glest_game/game/game.cpp
https://raw.github.com/williame/megaglest/8f68c9829dabfaf057fb338d2dd44201172a0da7/source/glest_game/graphics/renderer.cpp
« Last Edit: 21 July 2011, 00:22:39 by will »

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #42 on: 21 July 2011, 10:28:49 »
Congratulations Will, This patch is offically FASTER as it is now.

Pictures are taken at PEAK FPS. And under normal start game will's patch is defently faster. my fps was closer to 260-280 when the game was starting. whereas the SVN revison was closer to 230-250

Start game (the first 30 seconds of the game)

Wills Patch
http://imageshack.us/photo/my-images/146/megaglestwill2011072110.jpg/

SVN
http://imageshack.us/photo/my-images/5/megaglest20110721104450.jpg/

CURRENT Stable 140-170 FPS.

« Last Edit: 21 July 2011, 14:11:53 by Coldfusionstorm »
WiP Game developer.
I do danish translations.
"i break stuff"

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #43 on: 21 July 2011, 13:48:09 »
Very good result.

How does it compare to the official stable version?

Of course, pushing people with integrated Intel things into the 15fps+ range is a bigger deal than you folks with the monster discrete cards :)

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #44 on: 21 July 2011, 14:03:43 »
Will, gimme a second and il check that out too
WiP Game developer.
I do danish translations.
"i break stuff"

Coldfusionstorm

  • Golem
  • ******
  • Posts: 868
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #45 on: 21 July 2011, 14:12:17 »
Allright, Data post updated.
WiP Game developer.
I do danish translations.
"i break stuff"

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #46 on: 23 July 2011, 08:25:55 »
I have not got the code in front of me but my understanding is this:

We draw all visible tiles, each with the fog-of-war texture blended over them.

This seems to be the most massive inefficiency.

It would be interesting to know how much faster it is to not draw those tiles that are behind black fog-of-war texture, and to not bother blending the fog-of-war texture when it is fully transparent!

I trust that the objects are also culled with the same fog-of-war visibility logic already?

softcoder

  • MegaGlest Team
  • Battle Machine
  • ********
  • Posts: 2,238
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #47 on: 23 July 2011, 23:59:06 »
Yes only visible tiles are rendered (And same applies to fog of war i believe). Now regarding the last patch i played with it to see if I could get it working when moving the camera around but struggled as in certain angles the entire view goes black (so the current windows alha1 is the original visible quad code but with tile render improvements). I would REALLY like to get the visible quad thing accurate as i think once we nail it down it will give the extra render performance boost we need! I understand now the difference between the current visible quad logic and how true frustrum culling works (the 6 planes) so perhaps we will want to move to a proper frustrum culling model which would give much more accurate control over rendering items in the real view.

will

  • Golem
  • ******
  • Posts: 783
    • View Profile
Re: [TESTERS WANTED] Speeding up rendering
« Reply #48 on: 24 July 2011, 07:50:42 »
I use full frustum culling in glestng and it becomes hard work pretty quickly.

The unprojecting the four corners of the visible quad is better approach and fits much better with the existing 2D codebase.

Its important to remember that the glest codebase wants the visible quad to be in a strange order: top-left, bottom-left, top-right, bottom-right.  I find that unnatural and its tripped me up a couple of times.

So it should just be a matter of flipping it so that, in map coordinates until the order is true; I don't have code in front of me, but I imagine something as simple as this might work:

Code: [Select]
if(TL.x > TR.x || BL.x > BR.x) {
   swap(TL,TR);
   swap(BL,BR);
}
if(TL.y > BL.y || TR.y > BR.y) {
  swap(TL,BL);
  swap(TR,BR);
}

Regards the FOW, my point was that I distinctly recall Philip saying on IRC or somewhere that the important thing is to ensure you aren't blending fully-transparent textures... and we are doing exactly that for all fully-visible areas of the screen, which is often most of them.

I think having two sets of VBOs - one with FOW, one without.

Mr War

  • Guest
Re: [TESTERS WANTED] Speeding up rendering
« Reply #49 on: 25 July 2011, 18:05:57 »
I just reinstalled 3.5.2 to test this. Is there a minimal update download for 3.5.2? Don't worry if there isn't, just that testing requires saving all my WIP tile sets, factions etc, and then uninstalling etc.

Vaio laptop running Windows 7. It's an Intel Core i3, M380 @ 2.53GHz with 4GB RAM and 64bit op system.  ATI Mobility Radeon HD5470, 2231 MB total graphics mem (512MB dedicated)

both tests are at resolution 1600x900, it\s just the font that's bigger in the alpha

3.5.2


3.5.3 alpha
« Last Edit: 25 July 2011, 20:25:26 by Mr War »