GAE 0.3 planning

[daniel.santos]

Hello all,
I can't seem to figure out where to start with Blender or the import/export scripts, so I thought I would ask for community help. I'm working out the specs for the 0.3 branch of GAE and I've decided that there's a few things that are defiantly requirements.

Improved Death Sequences
I need static models (i.e., one frame animations) of various items that are owned by current magitech (as well as FPM) characters: weapons, hats/helms, shields or anything else that they can easily drop. I would like to improve the battle ground scene by having these objects fly about based upon the direction of the attack that killed them (and probably how hard they were hit when they died). I did this in a Quake mod once and it turned out really nice. Hats/helms shouldn't always come off, but swords and probably even shields should (perhaps some unit's shields might have a chance of staying on) . In short, we need the physics for objects to fly about, bounce off of the ground, splash, sink, etc.

For this to work, we would need to chop up the existing death animations. First off, we remove said objects. For hats/helms, we use the original animation and remove the person. In most cases, this will also require a hair animation that is separate. This way, if the helm is staying on, the helm and person dying animation can be played out and they will stay perfectly in sync (since they are all interpolated the same way). Since removing the helm or hat from units will expose their heads, this means we'll need hair. But just adding it to the death animation can cause problems with it bleeding through the hat/helm. Thus, the hair animation will have to be separate, but work just like the hat/helm animation, following the path of the dying unit. Then, when a unit dies and we decide that that time the head covering should come off, we start the animations of the hair and unit dying, but we'll just use the 1st frame of the head covering animation, treating it as a static model and use physics to send it flying about. When the unit dies and we decide that they keep their hat, we'll just play the unit & hat dying animations (again, at the same time), but without the hair.

If we decide to do the same for shields that we do for head coverings, it would be the same. The unit's death animation will not have the shield. The shield would them have a death animation that follows the unit and when we want to have their shield fall, we treat it as a static model and toss it about with physics.

Animated Tilesets
My thought here was to create separate tilesets for the two existing in Glest, "Animated Forrest" and "Animated Winter Forrest" leaving the original "Forrest" and "Winter Forrest" in it's original state. Animation would be optional (as most features in GAE). I have not yet decided for sure on the design for the “swaying in the wind” mechanism of tileset objects (trees, etc.), but these are my current thought. Each tree should have an "animation" with 4 or more frames. The 1st frame would be the normal position and the remaining frames would be the furthest "swaying" position, starting with north. The minimum required frames (or separate g3d files, whichever makes more sense) for this to work would be 4, a normal, and one each for swaying the maximum amount at 0, 120 and 240 degrees. The number of actual frames required to keep it from looking funky would be based upon the object. You can have 9 frames, a center and one for each of 0, 45, 90, 135, 180, etc. Rendering the tree when being effected by wind would require 2 interpolations, one for the angle (using the two angular models nearest the actual angle) and another for magnitude (between the result of the 1st interpolation and the center model). World objects could still have other animation sequences caused by other environmental factors (e.g., the next item).

Each tree type should also have leaves that we can blow around. This will probably work with the existing particle system, or at least with minimal modifications/additions to it (I haven't learned that part of the engine yet).

Tileset "Whatsit"s
A "whatsit" is a usually small feature of the landscape that is animated and has a limited set of Ai functions, but does not effect game play. Whatsits interact with the environment in a preset fashion and interact with units as well. An example of this would be a bird. There should be several varieties of birds (at least 3 or 4), they should each have their own songs and animation sequences, although they may share the same behavior. The animations should be 20-50 polygons (very simple) and include a sitting in tree or bush singing, flying and possibly a pecking stuff on the ground. Whatsit behavior is tied to tilesets, so you would only see birds pecking on the ground under a bush or tree. Each tree model should have locations for a bird to execute it's "singing" action. Thus, multiple birds could be in the same tree, but they would only sit in various places. A bird would interact with units by being shy and flying away if a unit comes near them.

Another example would be a water snake. They would frequent bushes and trees in the water, but would only travel in the water unless cornered. A watersnake would probably only have two actions, "move" and "hang around and slither"

A final example is a squirrel. A squirrel can jump from tree to tree and interact with the tree by causing it to have a branch twitch.

Further, various whatsits can have different interactions with different unit types. For example, a bird might not be at all afraid of a sprite and stay put even if the sprite is hovering directly above it. Whereas a bird would flee quite early at the rumbling sound of a catapult or other huge seige weapon traveling across the landscape. I'm not %100 sure how to make that tech tree neutral, except to define “factors” that whatsits are sensitive to at various levels. Examples of factors could be noise, potential for harm (attack units scoring higher here), sensitivity towards nature, etc.

In a less pleasant landscape, a tileset may have imp whatsits that like noise and harmful units. Maybe mosquitoes. These two are just some random ideas to consider for the design, not necessarily what I think would add significantly to the game.

While none of these proposed features do anything to improve the actual gameplay, they will add an extraordinary amount to the feel of the game, which is a significant part of the game experience. If there are anybody out there who would like to contribute with animation sequences, I would be very grateful.

EDIT Nov 5, 2008: Changing title of this thread
EDIT Jan 12, 2009: Restored original text from Google cache, as it seems to have gotten truncated in the forum conversion

[weedkiller]

Hi daniel.santos,
if you are really willing to do this i will try anything to support you .
Means, i know you want to do it and i will try making what you need but my skills are kind of limited...
Well the part of the add_hair is some sort of complex, i think we need a texture which has the missing hair-parts for each unit. for example the guard needs even a complete new head..
Ok the birds, i can try some and snake too, only problem: are you aware of the size you will have to use? for example my northmen-archer has a bow with a "sehne"(?englishword)[the thing which throws the arrow]. And because its often littler that one pixel of screen it looks weard, normaly antialasing is needed for something like this.
Ant the trees, do you want to edit the standart trees? because birds can only sit "on" them i think as they have no "wood connecting the leave".
But nevertheless i will try doing something if you tell me what you need at first.

[ZaggyDad]

The units should keep the hair they've got. the ones that have their helmets and whatever attached to them prolly wouldn't have them come off anyway.

And it would be nice to have a projectile physics engine to make large rocks have a weight and then they'd bounce, and then stop and sink into the ground like a regular unit. And make it so that it can stick in the ground (in the case of an arrow)and fade away. And you should make it have a accuracy thing for archers. (That would be especially useful in the case of my mod, since the archers should be a lot worse shots.

~Zaggy

[weedkiller]

Ok, i think i agree with you zaggy.
I will start separating the modells weapons and shields and their helmets. If i can fix hairpart with ~5 polygons i will do so.

[daniel.santos]

awesome weedkiller!  It occurred to me that stuff like a snake slithering about could probably be done with 2 polygons, by just using a flat surface on which we render an animated image.  Then again, it's probably more worth it to just make a small 20-30 polygon snake.  Either way, I don't mind taking short cuts on this type of thing, as long as the effect is impacting.

[Duke]

Well that would be the question, if I remeber correctly Glest doen't support animated textures yet, so that would have to be implemented in for the one option.

[daniel.santos]

Yes, you're correct.  So at this point, just making a very simple model would be easier.

[Dr.Dixie]

If the catapult type rock objects hit the ground, it ought to stay there long(er) than others, if not just stay there. Since when does the ground swallow rocks?? Arrows are different. They aren't huge.


!Dr.Dixie!

[daniel.santos]

I've been studying the Glest code and learning.  Apparently, the Glest engine doesn't interpolate the models at play time.  When the game is loading (or specifically, when it's loading the tech tree) it loads the models into memory.  When an "animation" has more than one frame (i.e., is an actual animation) it performs the interpolation, at 40 frames per second, and stores that mesh data in memory.  So doing this interpolation (for the trees and such) at runtime will be something that isn't done now.  I'm not smart enough on this stuff yet to know realistically what the performance impact will be, but I can already think of several ways to optimize it (caching, pre-rendering, etc., hand-coded SMID operations, etc).

I'm going to create the 0.3.x branch today.  I've already got a few of the basic physics data structures setup.  I know that there are open source physics engines out there, but I don't want to get very complicated and I also would like to keep the project (and resulting executable) as small as possible.

EDIT:  Further digging into the code revealed that my initial assessment was wrong.  Indeed interpolation is done every time graphics are rendered.

[Duke]

Well if the interpolation is only done before the actual game wouldn't it be possible to do it once per techtree change and save that data on the disk?
That would probably quicken the loading process.

Another thing that slightly anoys me in that context:
If you have a techtree with severel factions, but only juse some of them in a given game Glest still loads all of the faction and not just those which are needed.

[weedkiller]

Hi all,
now i started something i need some info of you needs:
You want to animate shield and swords themselfes as seperate objects, but how do you want do that? If the Objects fly around and maybe rotate a bit when falling on groung they woud need to be centered in the g3d. If they are supposed to be laing in the ground at the end they should have one axis to "top" so it is placed in the g3d that it fits to the end. and of course the center of the object should be at the g3d-model-center since you need to rotate it ingame around this point. Ok, now we have the problem that the object starts from the position and with the right orientation like it wat at first frame in death animation... so this is tricky since i "removed" this data from the swords geometry since i placed at modellspace-origin... Ok, i think you need a place where you save position and rotation of the sword-modell by which you know its orientation at start. But i dont know how you would like to save this in the g3d, ther is now way i think. The only tricky thing you could do is as you dont use animation of the objects you can use say second frame with standart set poligons which describe orientation and position so a mathematic function could read this ingame...

OK, anyone understood what i want to say? I will continue work so that i make the isolated falling dead people at first and keep weapons as simple models in the blend, but we need to solve this problem if you want to create this funcionable in game.

Or you define a new entry in objets xml which describes this data in this way as the firingpoint for ranged units is defined in it.

[daniel.santos]

weedkiller,
Thanks for your work and thoughtfulness.  You certainly appear to understand the issues fairly well.  The objects need to be centered in the g3d file.  I haven't dug *too* deeply into this code yet, so I'm not certain how the code knows what the model's boundaries are, or even if it does.  It may only care about where it's feet are.  None the less, please center it based upon what appears to be the center of gravity rather than the center of the object from a dimensional standpoint, this will make the physics calculations easier.  I may still end up calculating the center of gravity based upon the calculated mass, but I don't plan on that for a 1st implementation.

You have a lot of questions and points, so I think I'll just quote them all for clarity's sake.

Hi all,
now i started something i need some info of you needs:
You want to animate shield and swords themselfes as seperate objects, but how do you want do that?

My idea was to have two separate g3d files for each object: one that is animated, exactly as before, just without the person in the animation and one that is static (you may already understand that, but just making sure).  The reason is so we can arbitrarily decide rather or not they drop that particular object in a particular death sequence or not. We may omit this for things that will almost always get dropped (swords perhaps).

For this question, I presume you are talking about the animated version of the objects.  If they can remain in the same location as before you removed the person, that'll make this easier.  If not, I plan to have to feed some manually-generated location & orientation data anyway, so don't worry about it.  As long as it still follows the same range of motion, they can be aligned later.

If the Objects fly around and maybe rotate a bit when falling on groung they woud need to be centered in the g3d. If they are supposed to be laing in the ground at the end they should have one axis to "top" so it is placed in the g3d that it fits to the end. and of course the center of the object should be at the g3d-model-center since you need to rotate it ingame around this point.

Here, I presume you are talking about the static models of the objects.  For the animated version, we don't worry about it, since we're going to follow the original death sequence anyway.  Yes, they do need to be centered.  Again, please try to center based upon what appears to be the center of gravity.

As far as grounding, my plan at this point is to use the verticies from the model its self to determine if it collides with the world.  However, I will be cheapening this, I will not attempt to detect collisions with any other models and I will abbreviate each surface cell into a pair of triangles.  This will allow management of proper slope and keep calculations to a minimum.  I'll have some static value for the physical properties of the ground (used for determining bouncing, etc), but each object will get these defined on a per-object basis.

Thus, we wont have to worry about what is the top or bottom of the object since we're just going to iterate through it's verticies to see if it's made any collisions.

Ok, now we have the problem that the object starts from the position and with the right orientation like it wat at first frame in death animation... so this is tricky since i "removed" this data from the swords geometry since i placed at modellspace-origin... Ok, i think you need a place where you save position and rotation of the sword-modell by which you know its orientation at start. But i dont know how you would like to save this in the g3d, ther is now way i think. The only tricky thing you could do is as you dont use animation of the objects you can use say second frame with standart set poligons which describe orientation and position so a mathematic function could read this ingame...

Yea, don't worry about this.  There is no easy way to do this.  I plan on hacking up the g3d viewer to align these and get the offset and orientation needed to make it look good.  I will probably also go through the original animations and determine a good base initial velocity.  Then, I can further alter that based upon the direction of the attacker that killed them and how hard they were hit.  I can't go too crazy on that though because their body won't follow them and it would look funny to have a sword flying off over the horizon but the guy sorta just fall down.

OK, anyone understood what i want to say? I will continue work so that i make the isolated falling dead people at first and keep weapons as simple models in the blend, but we need to solve this problem if you want to create this funcionable in game.

Or you define a new entry in objets xml which describes this data in this way as the firingpoint for ranged units is defined in it.

hehe, yea, you're making sense  Maybe an "objects.xml" file could be stuck in the "models" directory of each unit that has a fancy death and we'll define all of that crap there.  That way, in the unit.xml file, we just reference what we define in objects.xml and it doesn't get too messy.

Well if the interpolation is only done before the actual game wouldn't it be possible to do it once per techtree change and save that data on the disk?
That would probably quicken the loading process.

Another thing that slightly anoys me in that context:
If you have a techtree with severel factions, but only juse some of them in a given game Glest still loads all of the faction and not just those which are needed.

Actually, I think that saving all of this to disk would slow things down since the the data calculations are probably faster than the IO it would take to load it from disk, which is why Martino is interpolating them in the 1st place instead of having the g3d contain 40 frames per second of animiation.

Sadly, interpolation only happens 40 times persecond, regardless of your actual rendering framerate.  This is part of the price you pay for pre-interpolating, but maybe I can hack this so you can set this in the .ini file.  Then, when rendering, it can just show the frame that matches based upon rendering time.  This will take a lot of changes though, because I think the model is set in the world's update method and rendering is done on a different time frame.

As to the tech tree loading factions you aren't using, that's a good point.  Perhaps I can put that on my to do list, although I don't think it's going to be one of the 1st things I work on, I have a lot that I want to get done right now and I may be starting a "real" (i.e., paying) project soon, so I'll have less time to play with this.

[hailstone]

As to the tech tree loading factions you aren't using, that's a good point. Perhaps I can put that on my to do list, although I don't think it's going to be one of the 1st things I work on, I have a lot that I want to get done right now and I may be starting a "real" (i.e., paying) project soon, so I'll have less time to play with this.

Perhaps that's something I can work on. I have looked at doing this before but never got around to it. It would be good to have a list of smaller things like this so newbie programmers like me can have something easier to practise on.

[daniel.santos]

Perhaps that's something I can work on. I have looked at doing this before but never got around to it. It would be good to have a list of smaller things like this so newbie programmers like me can have something easier to practise on.

w00t! send me patches

[weedkiller]

ok, i think i understood the way you want it.
so this is my first try and you can test if it could work this way.
swordman_split

To the boundaries: I think glest has as most games no real physik as you already assumed. ther is the field-grid on witch a unit has its space ( 1*1 or 2*2 or buidings sort of 4*6) and sometimes a field is blocked by something else or not.
So the unit is displayed at its field origin (ok, they model has a more detaild position and origin because of "smooth" walking but "physically" thats it) and the model is drawn there. Because a unit reaches out sometimes over this little field (weapons and stuff) there are overlappings but thats simply ignored. The Drawing is nearly without physcs understanding exept that opengl saves the "deep" of the screen pixels and if it drawes the next unit over the old stuff it can sort out if the unit is visible at that point or not (bec it is behind something).
But this is only how far i understand this, i played with opengl some time ago under delphi...

For your physics you will need to enhance that someway... . But i think its worth a try just to remember old stuff.

//EDIT:

My idea was to have two separate g3d files for each object: one that is animated, exactly as before, just without the person in the animation and one that is static

Ok, i think i misunderstood that till now , i made now:
- animated unit
- animated stuff
- static stuff (in original position)
- each a static g3d for sword and shield where they are positioned and orientated so that they should be best to be rotated and moved by physics
so the zip is now an updated one

[daniel.santos]

sweet man!

[weedkiller]

Wow holy shit, you are awesome daniel!
Dont get a workoholic just because i post here something, every time you spend in glest is enough for it (ok, that is egoistic, because you are of no use for me with "fried brain" .

To the network thing, now we now only need binarys for the two systems or? I dont think the mass of players is able to compile themselves (or me on my part am not able to do it and i never had problems with precompiled binarys so far).
Ok, you gave windows compilations all the time, so only linux will be needed, perhaps titi can do this again ?

The i will look out more often on IRC.. I got a bit lazy on that last days, on the one hand i work in a firm at the moment so i need my sleep on the other hand i was often no fun to play because of my bad connection.

[ZaggyDad]

Dr. Dixie is compiling it for linux. You just need to download his binaries.

~Zaggy

[wciow]

I,ve started working on a small set of animal 'whatsits' for use with the new code.

The first batch will include:
Wolf
Bear
Snake
Bird(s)
Spider
Deer
Heron

Each one will have a simple walking animation so that they can move about between tiles. Also some like the wolf and bear will have a larger animation set since they will be available as druid pets in the FPM.

[daniel.santos]

wciow, sweet!  I'm back from my trip and I'll have some time to put towards this again.  I'm working out my plans for putting all of this together in the code and in the subversion.

wciow, I would like to get you up to speed on using subversion so you can get them checked into the right places.  Ideally, the way source control works is that the actual file you are working on "belongs" in the source control so that you can change it, check it in, change it more, decide you liked it better the way you had it and roll it back to a previous "revision", etc.  It gets pretty complicated at the fancy end of things (some aspects of which I'm only now beginning to master myself), but at the basic end of things, it's usually very easy once you get setup.

I'm going to create an 0.3 branch so we can begin getting all of this checked in, but I'm at an impasse and I would like to get some opinions.  Should I continue keeping the modified data separate from the core Glest data or just modify the existing data, breaking compatibility with the original Glest engine & game?  My original thought was to create an "animated_forest" tileset under data/glest_game/tilesets (next to "forest" and "winter_forest") keeping them separate.  But this may also require either an "improved_magitech" tech tree with the improved death sequences & such or further hacking-up the existing magitech so that the original Glest engine ignores the advanced animations, but gae picks them up.  This isn't a problem for the FPM mod, because it explicitly depends upon gae, but I've been building gae to preserve compatibility with the Glest engine, magitech tree (and ideally, all other tech tree mods as well).  Another aspect to consider is the addition of patrol and guard commands to the magitech tree.  If I added those skill types/commands to units in the magitech tree (as in FPM), the original Glest engine reject it complaining that  "patrol" isn't a valid skill type.  I could add a <gae-skill> tag that gae would pick up, but Glest would ignore, thus solving the problem,  this dirties the water and will add a bit more ugliness when/if gae and Glest are merged (since it wont be needed anymore).

So I would very much like to hear you guy's opinion and thanks.

[orion]

This is only my opinion.

At the current moment, I think you should keep the original Glest engine and the GAE seperate. My reasons for saying that are that even though you have made great progress with the GAE, Daniel.Santos, the GAE isn't completely stable. Unless I'm misunderstanding what you are saying, if you merged the original Glest engine with the GAE, one would have to reinstall Glest again just to play the normal version. In Windows, I think it's cool that you have the two executable files (the original glest.exe and the "gae".exe) seperate. That way, one can easily switch between the GAE and the standard Glest. I think merging should be done a little later and when more bugs have been knocked out. Concerning the Patrol and Guard commands, they should be done when the GAE and the standard Glest engine are merged. That way, the old Glest engine won't complain about the new skills.

PS: I'm not entirely sure I understood what you meant by merging so correct me if the above paragraph is crazy.

PSS: I'll probably be emailing you about the Key for your server and for the website this weekend. Sorry I've been taking long with the website, but I've been trying to soak up as much PHP as possible.

[hailstone]

A positive point for breaking compatibility could be the opportunity to add new features that Glest is currently holding back.

A negative is the separate install of Glest and GAE that would be needed and possibly a separation of the community.

[daniel.santos]

thanks guys, I think I agree with keeping it compatible.  I've been able to find elegant solutions to these issues thus far, and adding "gae-skill" and "gae-command" doesn't seem too unreasonable of a solution (these would be ignored by Glest).  I'm sure that I can find elegant solutions to the death animations/object tossing as well.  I'm not too certain about animated tilesets though, so this may end up a separate tileset that just wont work in Glest.

orion, no problem on the delay, we all have other stuff we're doing  As far as merging gae and Glest, I certainly think that this is no time to consider it at all unless I work on stabilizing one of the branches (0.1 or 0.2) and hope Martino and the others aren't turned off by the number of changes.

In 0.2.5 you have a goof in your xml, you get an error message in game, except that it's empty.  This is because the text box component is very simple and wasn't made to handle gobs of text, so it ends up printing large amounts of text that contain carriage returns, off screen.  I've got it so that it also spits it out to standard error now, but this is an example of a stabilization issue.  I modified the crash & exception handling mostly to be fancy (display in game instead of to the console, if possible at least).  Crash handling is rather cute because it iteratively attempts crash handling starting from the most sophisticated to the least until one of them doesn't crash themselves (text box in game, OS-level text box, printing to the console or just giving up and existing).

[wciow]

wciow, I would like to get you up to speed on using subversion so you can get them checked into the right places.  

ATM I am using Tortoise SVN for transferring stuff. I just about got the hang of how to upload and download stuff but I still find it quite confusing since I've never used an SVN before (only an FTP server without any of the revisions stuff).

As for compatibility with Glest, breaking compatibility is not a bad thing. I personally see GAE as a standalone engine anyway since it could run without Glest.exe. If Martinho sees any part of the GAE which he wants to implement into stock Glest then he is free to do so, but my personal opinion is that backwards compatibility is not necessary. GAE should move on and be a great engine on its own merits rather than another way to play the original Glest.

[weedkiller]

I think the backwardcompatiblity is only a problem in the xmls,or?
So since GAE needs new entries in it, i think its ok if that breaks compatiblity. I also think it is possible to create a little converter tool that can split a GAE xml to a glest-original since you are mostly adding new features and the unit should work with the glest-original entries too.
Ok, you already found the solution with the GAE-skill .And i dont think that it creates dump for next versions since you could use such a converter tool to easily clean out the xmls.
// EDIT:
for death animation, is the problem that original glest doesnt know how to handle the g3d file? Perhaps you can keep one g3d that is good for original and another with other ending (g3d2) so glest hopefully ignores it and your engine knows which file to load...