Poll

Should glest support dds?

Yes
3 (60%)
No, because...
2 (40%)

Total Members Voted: 2

Voting closed: 11 December 2008, 15:45:22

Author Topic: Better pixmap support  (Read 9383 times)

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: Better pixmap support
« Reply #25 on: 20 January 2009, 11:22:16 »
If we are looking for a new format it should really work in linux too!
So I would prefer more common formats as PNG and/or compressed tga.

I really don't want proprietary stuff like DDS or this BLP!
And I want no directx things in linux!

wikipedia about BLP for example:
Quote
The file starts with a proprietary header, followed by the texture data. The texture data are typically stored in DXT1, DXT3, uncompressed or possibly DXT5. It is important to note that the size of the image is a power of 2.
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Better pixmap support
« Reply #26 on: 20 January 2009, 11:39:58 »
I might be able to add a library that supports lots of formats, including png, dds, gif; and also animations. Discuss at https://forum.megaglest.org/index.php?topic=4089.0
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

erandur

  • Guest
Re: Better pixmap support
« Reply #27 on: 21 January 2009, 20:41:07 »
Before doing too much work, know that it doesn't really matter. Let's say, the .dds textures, sure they are great, and seem to be small. But they only remain compressed in DirectX, because it is DirectX's format! If you try to run it with OpenGL, the dds texture will have to be recalculated anyway.

And most importantly, although I'm not sure of it for all formats, but jpeg for example becomes bmp size in the visual memory! Should do some research on this, but it's likely that it's going to be the same with other file formats, they won't remain compressed in the visual memory. That's what makes dds so special, if it remains compressed with OpenGL that is. ;)

modman

  • Guest
Re: Better pixmap support
« Reply #28 on: 23 January 2009, 22:50:04 »
Thank you for your insight!  JPEGs also have other disadvantages, such as looking bad unless you set quality to higher, which means compression is low anyways.  I like PNGs best; I wish they were allowed for attached pictures on the forum!

If we use a new file format, who volunteers to convert all the textures in Glest?  Not me.

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Better pixmap support
« Reply #29 on: 25 January 2009, 01:38:59 »
PNG? Is this a joke? Not only is PNG less compatible, but it is much larger. THe only advantage it has over JPG is the ability to carry alpha (pure alpha only). A JPG at 100 quality is still much smaller than the same PNG at either 32 or 24 bit. 8 Bit is useless since it has only 256 colors.

Animated GIF is still at the top of my wanted list. I'm rooting for you Hailstone!
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

modman

  • Guest
Re: Better pixmap support
« Reply #30 on: 29 January 2009, 03:45:44 »

Yea right.  ::) :P ::)

They have good compression, is lossless, and can be animated using an MNG file extension.  Read More Here.

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Better pixmap support
« Reply #31 on: 30 January 2009, 02:08:08 »
I agree with modman but DevIL can support lots of formats anyway.
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

modman

  • Guest
Re: Better pixmap support
« Reply #32 on: 30 January 2009, 04:00:52 »
I'm a bit confused; does this mean any type of file extension can be used for the images in the game?

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Better pixmap support
« Reply #33 on: 30 January 2009, 06:52:44 »
Any type that is supported by DevIL or other library, unless people are strongly opposed to that.
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: Better pixmap support
« Reply #34 on: 30 January 2009, 14:00:12 »
What about the alpha channel? Will this call problems when you use a file format which cannot handle transpareny..=
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Better pixmap support
« Reply #35 on: 31 January 2009, 02:03:22 »
Are you referring to the alpha for team colour or using alpha for a non-alpha supporting format, which I don't think is possible?
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Better pixmap support
« Reply #36 on: 31 January 2009, 09:16:12 »
I've looked at devIL a bit (known it since it was OpenIL) and it is the best known and the best well, everything for image support. It supports DDS, JPG, PNG, GIF, etc; I'm not sure if it allows animated GIF, but it might. And modman, take a picture, save it as 100 quality JPG, then take a PNG, save it as 32 bit, now compare the sizes. The highest qualities of each format look the same, but are drastically different in size. Unless you plan on magnifying an image too, a lower quality (like 90) can net you an even smaller size. There's a reason all cameras use JPG, and the majority of pictures on the web are JPG as well.

Yes, JPG is lossless, meaning saving it as 80% 3 times is going to affect the quality, but once again, 100% quality comes to the rescue.

Of course, if we impliment devIL, we can still use any of these formats, which is great! And transparency can still be done by any image that can hold transparency I guess (if programmed) including TGA, DDS, GIF, PNG, etc;

Am I strongly for devIL hailstone, and I hope you impliment it!  ;)
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,239
    • View Profile
    • http://www.titusgames.de
Re: Better pixmap support
« Reply #37 on: 11 February 2009, 09:09:50 »
I think we should move this topic to GAE !
Should I do it ?
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Better pixmap support
« Reply #38 on: 11 February 2009, 10:56:16 »
This topic is related to https://forum.megaglest.org/index.php?topic=4089.0

I think yes because it is a feature request.
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

ZaggyDad

  • Guest
Re: Better pixmap support
« Reply #39 on: 14 February 2009, 18:21:18 »
Ok, I've had some spare time to take a closer look at file formats sizes. Of all the popular formats, a sample picture gave me these outcomes:

  • JPG (100 quality) - 125 kb
  • JPG (80 quality) - 27.2 kb
  • PNG (32 bit) - 371 kb
  • BMP (24 bit) - 1.37 mb
  • DDS (DXT3 format) - 468 kb
  • TGA (uncompressed) - 1.37 mb
  • TIF - 552 kb
  • GIF (8 bit) - 178 kb

Of these, JPG has the smallest size and best quality, but cannot hold alpha. PNG can hold as many colours as JPG (as long as either 24 or 32 bit), but only the 8 and 32 can hold alpha and of that it is only pure alpha. BMP is useless, being the same size as a TGA, but with no alpha. DDS is far smaller than a TGA, can hold all the data a TGA can, including partial transparency, but it may be confusing, having a large number of versions, and being based for DirectX models. TIF is basically useless, being no better than JPG, and having bad compression. Finally, GIF can hold animation, but that will increase the size, and GIFs are limited to 256 colors. Dithering levels can be set to choose how the image looks with those limited colors though.

So the results are that JPG can get the most for file size when no alpha is needed, DDS can reduce the file size by a third when alpha is needed, GIF can have a very small file size as well, and can hold animation, but is only capable of 256 colors and 1 alpha.

UNTESTED:
  • MNG - It's a animation format, but I can't get my exporter to work, and it isn't very common, so...
  • RAW - Too big, totally useless, very incompatible, and few exporters
  • Software specific formats, such as XSF (the GIMP), PDN (Paint.NET), and PSD (Photoshop). Naturally, they aren't very compatible

I still stand by my decision that animated GIFs should be implimented. DDS is optional, and since my programming skills are very basic, I don't know how difficult that would be (would need to check DirectX SDK). However, DDS would reduce filesize nicely. While we're at it, might as well improve the G3D format, but that's a topic for GAE.

No! We can't use gifs, They're awful! I say what we do is use pngs and mngs (who cares if you have to do them in linux?), and keep the old formats, of course, and make it so that either alpha as alpha, alpha as team color, or alpha as alpha with a separate file for team color mapping. And pngs CAN do partial transparency. Sheesh. And so can mngs. Mngs are basically animated pngs. And, if you don't like that you have to use linux to use mngs, you should just start working on a fix for the windows plugin for Gimp. Here, I might as well just clarify the features I know about on the formats I know of:

  • Png: Partial transparency, good lossless compression
  • Mng: Partial transparency, animation, good lossless compression, basically animated png
  • Gif: Full transparency, animation, horrible quality
  • Tga: Partial transparency, no idea what the compression is like
  • Bmp: No transparency, no compression, basically raw data
  • DDS: I know nothing except what you tell me

And Modman, we don't have to get rid of the normal formats, for goodness' sake. We can keep them, and maybe someday have the mods converted, but for now, only the new ones would use the new formats.
« Last Edit: 14 February 2009, 18:25:19 by ZaggyDad »

daniel.santos

  • Guest
Re: Better pixmap support
« Reply #40 on: 15 February 2009, 09:41:43 »
A very lively discussion indeed, sorry I didn't join it earlier.  Here is my opinion: I support DevIL (as discussed in the other thread) but I'm not of any strong opinion there because I'm not well versed in the various imaging libraries and their pluses and minuses.  Note that SDL does have an imaging interface (in Gentoo, it's a separate SDL-image package), but personally, I've seen some ugly bugs in it (mostly some thread safety issues) -- I guess that's not a big deal if you're only loading from one thread.

However, I don't really care what you use.  If you use an image format that doesn't support alpha layers, it's possible to use a second image to define the alpha layer.  If you use an image format that doesn't support animation, you use a series of those image files for the animation, that's what hailstone was saying in this post:
I know of games using sprite sheets for animated images but there might be performance issues using it on a model and it would increase the size and complexity.
Memory footprint would increase, but complexity wouldn't increase much (well, beyond supporting any animated textures at all).  Because if we do support animated textures at all, we would have to store each frame, uncompressed in memory anyway.  So this becomes a matter of how we load each of those frames.

No, sprites are soooo outdated. I think when playing glest with its 36 units - and them all animated by sprites - my cpu will get melted...  <img src="{SMILIES_PATH}/icon_rolleyes.gif" alt=":roll:" title="Rolling Eyes" />
Try not to think in the box.  The config will have an option as to rather or not to enable animated textures.  Don't assume automatically that because we use the word "sprite" that it will melt your CPU or look like commodore 64 sprites (or Atari 2600 sprites for that matter :) ).  Actually, the textures will reside in the memory of the graphics card (as long as it has sufficient room) and swapping textures from frame to frame is an almost ZERO impact!!  If you lack sufficient graphics memory,then yes, there is an overhead because it means more throughput is required between main memory and the graphics card, so being able to disable it would indeed help on slower machines (also, see my comment on LOD support below).  Also, remember that the bottleneck of a particular performance problem may not always be where you expect it; it can be CPU, GPU, I/O throughput, low memory & the swapping that accompanies, etc.

Before doing too much work, know that it doesn't really matter. Let's say, the .dds textures, sure they are great, and seem to be small. But they only remain compressed in DirectX, because it is DirectX's format! If you try to run it with OpenGL, the dds texture will have to be recalculated anyway.

And most importantly, although I'm not sure of it for all formats, but jpeg for example becomes bmp size in the visual memory! Should do some research on this, but it's likely that it's going to be the same with other file formats, they won't remain compressed in the visual memory. That's what makes dds so special, if it remains compressed with OpenGL that is. ;)
Why would dds remain compressed in OpenGL, much less DirectX?  Even on the video card, you don't want to store textures compressed!  That means you'll have to decompress them for EVERY time the texture is used, for EVERY frame.  If you have a particular texture being used on 40 models or polygons and you're rendering 80 frames persecond (or trying to), you end up having to decompress it 1600 times per second! No, you don't store these things compressed in memory (either main memory or video memory).  Maybe my common sense doesn't serve me well (as I don't know DirectX) and maybe DirectX can store these compressed in main memory, but the idea seems dubious to me.

So back to separating out animation frames or the alpha or team alpha into separate image files, all of this will have to be defined in an XML somewhere and we'll have to come to agreement on a sensible spec for all of this, but that shouldn't be too hard.  Also note that with animated gifs or mngs, we would have to specify the animation speed anyway (I guess the default would be to use the animation speed specified in the image?).  Finally, we may even want to get fancy and support morphing between images in an animation sequence to keep input files smaller and the net result prettier (much like we do with g3d models).

Also, the g3d format doesn't support animated skins, so that's another issue, we would have to bump the format or (perhaps an uglier solution) override it with info passed via the XML tag for the model.  For that matter, we would have to do the same for using a separate image file for the alpha and team color layers.

As far as DDS, BLP and other proprietary formats, as long as we can read them from linux & mac w/o licensing issues, I don't care.  If not, the answer is absolutely no.

I have no problem with animated gifs, color limitations and all. I also have no problem with jpegs, although I despise super-lossy compression for textures used in a 3d game.  As somebody mentioned before, we're going to decode them into a bitmap in memory anyway.  I just *ask* that if you use jpegs, please keep quality very high (i.e., compression low)!

Back to memory consumption issues, I'm not 100% sure of exactly how this works with the OpenGL portion of the image.  Ideally, the textures are fed to the OpenGL library when the game starts up, but they do stay in main memory (uncompressed).  I'm not certain, but they may be getting fed to the OpenGL interface at every frame.  But if they are fed when the game starts, and your graphics card has sufficient memory, then even though the OpenGL library retains a pointer to where the uncompressed image data is in main memory, it may not attempt to access it for the remainder of game play.  If the data is not accessed for a while, the kernel (of whatever OS you are running) can decide to write the memory pages that hold them to swap, leaving them as "cached swap pages" and then if it runs low on memory, drop them entirely to use those memory pages for something else.  So even though all of these images remain in memory uncompressed, under ideal circumstances, they may not end up taking up actual memory (i.e., residing physically in memory).

Finally, please keep in mind a very important planned features of 0.3: LOD support (https://bugs.codemonger.org/show_bug.cgi?id=20).  When this feature is in place, each image will be stored multiple times in memory, once at full size, one at half, one quarter, one eight, etc.  When you view a model or polygon that has a texture mapped to it, the size of the texture will depend upon the distance -- this keeps the amount of GPU used for texture mapping down to a minimum.  The same will be true for models; each model will be stored multiple times in memory scaled down to lower polygon counts.  Ideally, this will be transparent to the player because by the time their camera is far enough away from a model, the change in quality will be unnoticeable (in theory! :)