MegaGlest Forum

Archives (read only) => Glest Advanced Engine => Feature requests => Topic started by: ChupaReaper on 26 February 2011, 20:10:32

Title: MD5 Model Format (or variant)
Post by: ChupaReaper on 26 February 2011, 20:10:32
I'm sure a new model format has already been suggested or maybe there's even a ticket but here I'm recommending the MD5 format. It's probably the most compatible format that supports skeltons and separate animation files (md5mesh for meshes, uv and skeleton with weights and md5anim for skeletal animation that can be used by any md5mesh with a matching skeleton). This is the format I use to convert my 3ds max models with skeletal animation across into blender where I export into the g3d format.
Here's more info on it: http://www.modwiki.net/wiki/MD5_(file_format) (http://www.modwiki.net/wiki/MD5_(file_format))
I also noticed that GlestNG is looking into it so if that engine gets big, it means more compatibility.
I'm not sure about the licensing for it, I don't think there is any as it's more to do with the models and tools than the file format which is plain text it seems. It's a much lighter format and with md5anim multiple models can share the same animations saving a lot of file space and development time for modders.

With skeletal animation we can set bones that are used as a reference so when firing projectile attacks rather then setting the offset in an xml, you can refer to a bone as an offset instead. Also moving turrets and maybe merging animations could be used (one md5anim could have a walk cycle that could then have say the upper body bones of the model taken over by something like an attacking animation so a unit can move and attack at the same time without having to make a separate animation for it).

For now though simply replacing a g3d model with an md5mesh and md5anim anims would do, future features could be implemented after. To use this in the xml, the md5mesh should be defined at the top which would then make GAE know that this unit uses md5 instead of g3d, then md5anims should be referenced to for each skill in place of where separate g3d models were.

Anyway there's a nice huge wall of text, using md5 instead of g3d means I don't have worry about any g3d limits as I have too many g3ds for two of my factions to run in the same game at the minute and reducing their aniamtion frames any more will ruins them.
Title: Re: MD5 Model Format (or variant)
Post by: will on 26 February 2011, 20:16:13
(Side question as you are a user of md5)

Have you ever seen games that specify a bones-per-vertex limit, and if so what is it?  Do exporters enforce it?
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 26 February 2011, 20:25:53
(Side question as you are a user of md5)

Have you ever seen games that specify a bones-per-vertex limit, and if so what is it?  Do exporters enforce it?
Well for Doom 3, it is 4 bones per vertex, I only really stick with one or two in rare cases, I've never tried exporting a model that breaks this limit, I don't think md5 exporters will enforce any rule as the format is used universally by multiple engines and some engines use different limits, I'm sure exporters could be told to show warnings or adapt to limitations, but they haven't been updated for a while now I think.

I sometimes use two bones for elbows and other joints and I know facial expressions would use all 4 bones per vertex but that's more for cinematics and fps that Glest doesn't really need, unless real-time cinematics are added, even then the limit of 4 seems to be fine.
Title: Re: MD5 Model Format (or variant)
Post by: Omega on 27 February 2011, 02:57:17
Support. Advantages: More universal format (people can use whatever modeling program they wish), simpler, more powerful format, and splits the models and animations.

However, I do not think we should specify the model first then the skills would only specify the animation. I, instead, think we should specify the model and animation each time for skills, as it can be the same model, but, the advantage is that you can use an entirely different model, as is sometimes necessary.
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 27 February 2011, 12:18:55
Support. Advantages: More universal format (people can use whatever modeling program they wish), simpler, more powerful format, and splits the models and animations.

However, I do not think we should specify the model first then the skills would only specify the animation. I, instead, think we should specify the model and animation each time for skills, as it can be the same model, but, the advantage is that you can use an entirely different model, as is sometimes necessary.

I never thought of that  ::), yea this would be better, much more freedom with models. Maybe multiple meshes and aniamtions could eventually be specified per skill also?
Title: Re: MD5 Model Format (or variant)
Post by: Conzar on 9 March 2011, 13:18:14
Are there any loaders available?  I know I have loaded MD5 into jmonkeyenigne which was pretty easy since someone already had written a loader for it.
Title: Re: MD5 Model Format (or variant)
Post by: will on 9 March 2011, 13:37:36
Load into what, exactly?

People modding glest with 3DSMax typically use MD5 to interchange with Blender, and then G3D export from Blender (as G3D export from 3DSMax has been broken with newer 3DSMax releases)

(trivia: the original Glest models were made with 3DSMax)
Title: Re: MD5 Model Format (or variant)
Post by: Conzar on 9 March 2011, 14:10:52
Load into what, exactly?
Into OpenGL (thats what Glest/GAE/MG use for rendering 3d graphics right)?  jmonkeyengine is a 3d engine for java (which can use opengl through other java libs) and it has the concept of loaders.  A loader is a library that can load a specific 3d format like MD5,MD3,obj,wrl,etc.
Title: Re: MD5 Model Format (or variant)
Post by: will on 9 March 2011, 14:49:42
Aha right.  Loading MD5 is not hard, its a well understood, straightforward and documented format.

Rendering bone animations is none-trivial and there is no current code in Glest; it would have to be added.

Glest itself is nice and modular and adding additional model formats ought not break things.
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 10 March 2011, 15:25:51
I'd love to see this format added, it would make modelling much easier, plus some physics could be added much later on for insane stuff like rag dolls! Imagine a massive unit firing a large projectile that sends loads of small units flying to their death (or maybe not to their death, units could have mass defined in their xml and projectiles could have force enabled, but that's another feature all together, it would make good use out of rag dolls though).

Also with md5, different skills that use the same mesh but different animations could be interpolated too, so rather than having a unit standing still that snaps into a walking animation, have a unit standing still that blends into a walking animation.
Title: Re: MD5 Model Format (or variant)
Post by: will on 10 March 2011, 16:03:11
 
Also with md5, different skills that use the same mesh but different animations could be interpolated too, so rather than having a unit standing still that snaps into a walking animation, have a unit standing still that blends into a walking animation.
:thumbup: the most immediate gain

Everyone comfortable with maxbones starting at 2?
Title: Re: MD5 Model Format (or variant)
Post by: Fluffy203 on 10 March 2011, 16:13:36
Depends i use 3dsmax 8 and just export with it , so as a 3dsmax user i don't use md5
Title: Re: MD5 Model Format (or variant)
Post by: wciow on 10 March 2011, 22:02:05
Everyone comfortable with maxbones starting at 2?

I assume "maxbones" means the maximum amount of bones that can be influencing a vertex, or to put it into Blender terms the maximum amount of vertex groups to which the vertex is assigned. 2 Seems fine since I rarely use more than two bones on a vertex anyway.

I'm excited by the prospect of being able to assign particles, lights and objects to a specific bone (maybe even some ragdoll down the line).

An example would be a dawn of war style matched animations where units can be picked up by other units (attached to a bone) and thrown around  :)
Title: Re: MD5 Model Format (or variant)
Post by: will on 10 March 2011, 22:05:29
... and turrets.  There was a thread about that too.
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 10 March 2011, 23:14:55
Some of my models use up to 4 bones per vertex (good for stuff like the sides of horses legs at the top else it looks all twisted when they walk) and some of my models are Doom 3 ones which uses up to 4.
Title: Re: MD5 Model Format (or variant)
Post by: will on 10 March 2011, 23:19:21
yes but at the scale we want to show models in an RTS (we see screenshots that are zoomed in so you can see how low-poly models really are, but would you play it at that kind of zoom?) could you get away with 2?  Or 3?

(I challenge to bring the number down to the minimum acceptable; if that's 4, then that's fine.  But we don't need to animate speaking faces here!)
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 10 March 2011, 23:53:03
True 4 isn't necesary and I'll see if I can adjust the models that have exceeded 2, but for large flying units, their animations are quite clear also cinematics in scenarios with camera angels, etc are quite possible. 4 is good mainly for compatibility with other models, if a model that exceed 4 was loaded, would it crash or could the lower weights just be ignored, if that was the case then I wouldn't really care.
Title: Re: MD5 Model Format (or variant)
Post by: will on 10 March 2011, 23:55:11
easiest to ignore the lower weights.
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 11 March 2011, 00:06:31
easiest to ignore the lower weights.
That'll work fine for me then, means I wouldn't have to go through every model and check their weights lol, could always have it increased in the future maybe but 2 should be fine for most cases.
Title: Re: MD5 Model Format (or variant)
Post by: will on 11 March 2011, 00:08:55
well we have a solid proposal then.

Lets hope Glest goes adding MD5 after the merge!
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 11 March 2011, 00:13:23
Yes, it will be excellent if it is, the memory constraints of g3d that I'm starting to suffer will be long gone as will massive file sizes.
Title: Re: MD5 Model Format (or variant)
Post by: will on 11 March 2011, 00:20:01
well MD5 takes less space, but if we want to really reduce it we can go to a triangle strip format (Stripe (http://www.cs.sunysb.edu/~stripe/) gains typically 50% and it renders quicker!) although this likely means another custom format unfortunately.

and we can dramatically reduce texture memory using DDS: https://forum.megaglest.org/index.php?topic=6615.msg68729#msg68729
Title: Re: MD5 Model Format (or variant)
Post by: -Archmage- on 11 March 2011, 03:06:59
Please have the "maxbones" at LEAST at 4. What's the deal? Filesize? Oh please, Glest is a tiny game.
I'm an animator, I'm the Lead Animator on the Constellus Team, so animation matters a lot to me. I will need a large enough "maxbones" for some animation tricks and stuff.
Title: Re: MD5 Model Format (or variant)
Post by: John.d.h on 11 March 2011, 04:21:34
Sometimes you need a lot of bones:
(http://opengameart.org/sites/default/files/previews-3d/spider-with-bones.png)
Title: Re: MD5 Model Format (or variant)
Post by: will on 11 March 2011, 07:30:56
Quote
"maxbones" means the maximum amount of bones that can be influencing a vertex

So its not how many bones in the model in total.

Doom and many games support upto 4, but they have models that are drawn much bigger and nearer the camera.

The number of bones that can influence a vertex will directly relate to rendering performance in a linear way; if the engine supports 10 bone influences per vertex, the vertex shader will render twice as slow as if it only supports 5 bone influences per vertex, even if your model only uses 2 bones per vertex.

(Vertex shaders are only a small part of the cost of drawing a model, but its not to be sneezed at.)

The nice thing is that you could make this configurable as a graphics setting, although it means patching the shader source-code and re-uploading to the GPU, it all ought to be configurable.

Still, people should have an expectation about what the likely max bone influence per vertex is, and keep that in mind when making their models.
Title: Re: MD5 Model Format (or variant)
Post by: -Archmage- on 11 March 2011, 07:33:50
Oh well I have a Radeon HD 5870, so I'm not too worried with over 2 terraFLOPs. ;D

Anyway I think default should be 4.
Title: Re: MD5 Model Format (or variant)
Post by: John.d.h on 11 March 2011, 08:23:13
Truth be told, you could make the most sophisticated animation on a super high-poly model with a high-resolution texture, and it wouldn't look appreciably better in Glest than a much simpler unit.  Keep in mind that everything is from a bird's eye view here.

If this is only a limit on the number of bones that can directly influence a vertex, then we should still be able to go crazy with target bones and other such things like on that spider, right?
Title: Re: MD5 Model Format (or variant)
Post by: will on 11 March 2011, 08:28:03
If this is only a limit on the number of bones that can directly influence a vertex, then we should still be able to go crazy with target bones and other such things like on that spider, right?

I have to think about that.   I think the shader will have to have a configured maximum number of bones in all too, but you can always have more bones by having submodels.  Anyway, this maximum number of bones per mesh is going to be quite high and nothing to worry about with spiders and things.
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 11 March 2011, 08:50:45
File size will be drastically reduced regardless:

Werewolf.md5mesh 469KB         
Werewolf_Idle.md5anim 28KB      vs   Werewolf_Idle.g3d 1,218KB
Werewolf_Move.md5anim 41KB   vs   Werewolf_Move.g3d 1,610KB
Werewolf_Attack.md5anim 24KB   vs   Werewolf_Attack.g3d 826KB
Werewolf_Death.md5anim 27KB   vs   Werewolf_Death.g3d 1,218KB
Total: 589KB   vs   4,872KB

As you can see, there is a huge file size difference and also similar meshes can share md5anims too so if you keep your skeletons under a nice format you may not have to animate as much and can save a few KBs, not that is matters that much, what with how much you save in comparison to G3D. Overall it's ~8x smaller with the bonus of animation sharing which could save even more space with similar models. With my mod I'm struggling to handle to factions, with md5 I should be able to handle 8 no problem and I'm only going for 5.
Title: Re: MD5 Model Format (or variant)
Post by: will on 11 March 2011, 08:54:31
excellent stats!  Any chance of seeing the numbers after 7zipping those two versions - MD5 and G3D - in separate archives?
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 11 March 2011, 12:24:14
excellent stats!  Any chance of seeing the numbers after 7zipping those two versions - MD5 and G3D - in separate archives?
Ok so all the Werewolf md5 files (mesh and anims) compressed into 7zip with ultra compression are 128KB.
The g3d files are make up a much bigger 7zip of 2.47MB! That's nearly 20x bigger  :o!
Title: Re: MD5 Model Format (or variant)
Post by: ChupaReaper on 13 February 2012, 14:16:38
I been away from these forums for a while, any news on a new model format? I'm redoing my project to use less models so the file sizes are less of a problem now, it's the skeletal animation that would be the most useful.