Future plans for LUA

[zombiepirate]

Hey, Just curious what the future plans for the LUA are. What new features are going to be added in the next release and is there a list of feature requests anywhere? Also, are there any scenarios out there that use any significant amount of LUA? They aren't well advertised if there are.

I'd like to start doing some work with the LUA code (including fixing a couple rather large bugs). I also wrote a (very) little demo scenario demonstrating some AI micro management. I could post it if anyone is interested.

[jda]

Hey, Just curious what the future plans for the LUA are. What new features are going to be added in the next release and is there a list of feature requests anywhere?

I assume you searched the tracker for the "LUA" keyword already...?

Also, are there any scenarios out there that use any significant amount of LUA? They aren't well advertised if there are.

I haven't actually looked at them (the code) but I'd guess omega's Military should have quite a bit of Lua in...? 

I'd like to start doing some work with the LUA code (including fixing a couple rather large bugs). I also wrote a (very) little demo scenario demonstrating some AI micro management. I could post it if anyone is interested.

Please do post the demo!  
And also please do fix those bugs too!

[zombiepirate]

XML: http://www.mediafire.com/?f2qnmgmknxj
Maze map I was using to test: http://www.mediafire.com/?mwzmzzindnx

It's a 9v9 battle, horsemen vs drake riders. I managed to get 8/9 drake riders to survive a wave of horsemen with some micro-management strategies. The actual position of the units during the battle is extremely touchy though so in order to generalize the strategy significantly more code would be needed. As well as functionality that the GAE LUA currently doesn't support.

[silnarm]

Hey, Just curious what the future plans for the LUA are. What new features are going to be added in the next release and is there a list of feature requests anywhere? Also, are there any scenarios out there that use any significant amount of LUA? They aren't well advertised if there are.

I was only planning some small things for scenarios, Dialogue sequences and better camera control mainly. If you wish to work on something else though, by all means do so!

I'd like to start doing some work with the LUA code (including fixing a couple rather large bugs). I also wrote a (very) little demo scenario demonstrating some AI micro management. I could post it if anyone is interested.

Haven't checked out the scenario as yet, will do so later.  There are some concerns over performance, and I'm not sure doing 'tactical' AI in Lua will be feasible, but we will need to run some tests to determine that.  Regarding the large bugs, by all means fix them, but I'd ask that you create tickets for them first.  The new Lua has not been extensively tested yet and there are bound to be a few problems, any help reporting or fixing them is most welcome

Cheers.

[jda]

Thanks, zombiepirate!
I'll test and look at the XML later.

EDIT: I just played it. I intended to check the XML afterwards but there doesn't seem to be an end-game condition in there.
So I spent a very long time looking for a remaining Drake rider or Magic base to kill after my horsemen returned to the base.
Yep, that did look like an endgame move but... there was no endgame dialog and there were far more farms than actually needed so I guessed they were meant to serve as targets for a "second" surprise attack.

But, til it got to a boring search for nothin', it was actually very very fun to play!
I liked the "here's your target!" effect. 
I thought the "follow that horseman!" thing was rather pointless in this situation but, scripting-wise, it was very interesting and might be very useful in another scenario.  


There was an actual "giveaway" of the AI plan change whenever there was one: you'd get a warning on the screen about unit # not having SetPosition or GetPosition or something of the sort.
I thought it would get typed to the terminal I ran it in so I didn't bother to pay attention.
But ... this may be something to look at, either in your script or in GAE. 

In case it's useful to you, this is what got dumped to my terminal:

Code: [Select]

Created Unit (ID, Faction): 0 0
Created Unit (ID, Faction): 1 0
Created Unit (ID, Faction): 2 0
Created Unit (ID, Faction): 3 0
Created Unit (ID, Faction): 4 0
Created Unit (ID, Faction): 5 0
Created Unit (ID, Faction): 6 0
Created Unit (ID, Faction): 7 0
Created Unit (ID, Faction): 8 0
Created Unit (ID, Faction): 9 0
Created Unit (ID, Faction): 10 0
Created Unit (ID, Faction): 11 0
Created Unit (ID, Faction): 12 0
Created Unit (ID, Faction): 13 0
Created Unit (ID, Faction): 14 1
Created Unit (ID, Faction): 15 1
Created Unit (ID, Faction): 16 1
Created Unit (ID, Faction): 17 1
Created Unit (ID, Faction): 18 1
Created Unit (ID, Faction): 19 1
Created Unit (ID, Faction): 20 1
Created Unit (ID, Faction): 21 1
Created Unit (ID, Faction): 22 1
Sending units on attack
1 true
2 true
3 true
4 true
5 true
6 true
7 true
8 true
9 true
10 true
11 true
12 true
13 true
0 true
spreading out 5
14 true
15 true
16 true
17 true
18 true
19 true
20 true
21 true
22 true
Unit Killed (ID, Faction):  5 0
Unit Critical ID, Faction: 14 1
done maneuver
jumping back in 14 117
Unit Killed (ID, Faction):  14 1
Unit Killed (ID, Faction):  10 0
Unit Critical ID, Faction: 16 1
Unit Killed (ID, Faction):  16 1
done maneuver
jumping back in 14 117
jumping back in 16 120
Unit Critical ID, Faction: 15 1
Unit Killed (ID, Faction):  15 1
done maneuver
jumping back in 14 117
jumping back in 16 120
jumping back in 15 122
Killing Invaders 11
14 false
15 false
16 false
17 true
18 true
19 true
20 true
21 true
22 true
Unit Critical ID, Faction: 17 1
Unit Killed (ID, Faction):  6 0
Unit Critical ID, Faction: 18 1
done maneuver
jumping back in 17 112
jumping back in 15 122
jumping back in 18 119
jumping back in 14 117
jumping back in 16 120
Unit Killed (ID, Faction):  17 1
done maneuver
jumping back in 17 113
jumping back in 15 122
jumping back in 18 119
jumping back in 14 117
jumping back in 16 120
Unit Killed (ID, Faction):  18 1
Unit Critical ID, Faction: 20 1
Unit Killed (ID, Faction):  20 1
done maneuver
jumping back in 20 116
jumping back in 17 113
jumping back in 15 122
jumping back in 18 119
jumping back in 14 117
jumping back in 16 120
Unit Killed (ID, Faction):  9 0
Unit Critical ID, Faction: 19 1
done maneuver
jumping back in 20 116
jumping back in 17 113
jumping back in 15 122
jumping back in 18 119
jumping back in 14 117
jumping back in 16 120
jumping back in 19 115
Unit Killed (ID, Faction):  19 1
Unit Critical ID, Faction: 22 1
done maneuver
jumping back in 20 116
jumping back in 17 113
jumping back in 15 122
jumping back in 18 119
jumping back in 22 117
jumping back in 14 117
jumping back in 16 120
jumping back in 19 115
Unit Killed (ID, Faction):  22 1
Unit Critical ID, Faction: 21 1
Unit Killed (ID, Faction):  21 1
Sending Units home
1 true
2 true
3 true
4 true
5 false
6 false
7 true
8 true
9 false
10 false
11 true
12 true
13 true
0 true
done maneuver
jumping back in 14 117
jumping back in 15 122
jumping back in 16 120
jumping back in 17 113
jumping back in 18 119
jumping back in 19 115
jumping back in 20 116
jumping back in 21 119
jumping back in 22 117
P.s.: The map is an excelent maze!!! Congrats!

[zombiepirate]

Thanks for testing.

There isn't an end game condition unless the horsemen all die. The scenario was just meant to demonstrate some basic micromanagement rather then to be playable to an end. If anyone wants to use parts of the script or the map to make a fully playable scenario then by all means do so.

Did you manipulate the horsemen when you tested or did you just let them do their own thing? The drake riders are the ones that are actually micromanaged by the LUA while the horsemen are given a blanket "attack" command at the magic's home location. If everything is left alone then 8/9 drake riders should usually survive the attack.

[jda]

There isn't an end game condition unless the horsemen all die. The scenario was just meant to demonstrate some basic micromanagement rather then to be playable to an end. If anyone wants to use parts of the script or the map to make a fully playable scenario then by all means do so.

I didn't assume it was playable to an end, but I did think it might be. But I really only mentioned that as a warning for anyone who might want to play it too. And I actually only played it before looking at the code because I don't really know LUA so I wanted to see what happens in the game to understand better what is scripted in the XML.
And thank you for the permission to reuse the script and the very nice map. I won't be doing any such thing soon myself but it's good to know for the future.  
Thanks.

Did you manipulate the horsemen when you tested or did you just let them do their own thing? The drake riders are the ones that are actually micromanaged by the LUA while the horsemen are given a blanket "attack" command at the magic's home location. If everything is left alone then 8/9 drake riders should usually survive the attack.

Yes, I certainly did manipulate the horsemen, that's how I beat and killed all the drake riders!
I did assume that though it might not be fully playable it would certainly not be fully hands-off demo-mode! That's interesting. That got me curious to look at the script (haven't yet) and check whether that "Here's the target!" thing I mentioned earlier acutally is necessary to give that order or if you added it as to give that very effect.

I also didn't realize what you just said, that the horsemen attack was automatic; I sent one horseman to scout around and almost immediatelly after all the other horsemen "followed" the first one. I selected them all (including the scout) and gave them the Stop command. Then I sent the first one out again and the others followed it again! By my second Stop order they stopped that for good. So I guess that was just coincidence?... (I think that, on my second atempt to stop them,  they might be stuck in a narrow which they couldn't cross because the first one was in front of them.)

I did notice the drake riders micromanagement on two levels:
1. Ingame as they all of a sudden "disappeared" all at once (the fog of war that was lifted before also came back). When I got to the location they were supposed to be in, they weren't there.
2. But it was specially on the output to terminal I posted before, I noticed some interesting stuff:
- There was a Unit Critical condition that, made a call to be made for  "jumping back in 14" (I assume that's either a retreat command or more likely a "replace said unit in that position with another fresher one", which would be very nice! ). I'll look at it.
- There was also a "done maneuver" command being issued from time to time I'm curious about.
- I also assume the one time only issued "killing invaders" is the start for micromanagement of the expected slaughter of the enemy.

Heck, I didn't mean to do it today as I have no time but you got me curious to watching the demo hands off and looking at the XML... Darn you!

[zombiepirate]

I fixed three bugs in the LUA, and committed to the trunk (which still claims to be 0.2.12; has 0.2.13 not been fully merged into the trunk yet?). If anyone wants to do more extensive testing then I did i'd be grateful for any feedback.

• The 'attacked' trigger now fires (I put the call in World::hit(...) if that's ok?).
• The 'command_callback' trigger hopefully fires correctly now.
• The user_data variable is passed.

[jda]

I fixed three bugs in the LUA, and committed to the trunk (which still claims to be 0.2.12; has 0.2.13 not been fully merged into the trunk yet?). If anyone wants to do more extensive testing then I did i'd be grateful for any feedback.

• The 'attacked' trigger now fires (I put the call in World::hit(...) if that's ok?).
• The 'command_callback' trigger hopefully fires correctly now.
• The user_data variable is passed.

Regarding your demo scenario, I started a new thread for discussing the scenario on the Maps, Tilesets and Scenarios board, here: https://forum.megaglest.org/index.php?topic=5358.0
This way we can focus on the LUA on this thread.

Now I'm not sure this relates to LUA but the outcome you predicted (8/9 drake riders surviving) was almost completelly reversed: I got 7/9 horsemen surviving on two different runs (both without any intervention from me).
So... maybe you last ran the scenrio on GAE release other than current 0.2.13 and there are changes between my currently installed version (official 0.2.13 32 bit Linux binaries and data) and whatever you might be running (earlier, SVN?)?
Or ... for a different hipothesis... Do floating point calculations actually affect the drake rider's actually delievered damage? If so, maybe different builds will give different results?
Am I just being silly? 

[zombiepirate]

Posted a reply in the other thread as well.

I'm still curious though how many people can script a bit of LUA, and would be willing to compile from the trunk to test or comment on the bug fixes. Also I am hoping to add a few more features, more unit control and queries should be fairly easy to implement I think. If anyone has any requests I'd be glad to hear them.

[jda]

Part of the discussion that was moved to the other thread, I think it belongs here:

Interesting. I'll re-test it on a few different builds and try to find out why...
EDIT: Tested on the 64 bit linux binaries, 5/9 drake riders survived. Tested on my latest build from the trunk, 8/9 survived. I don't think floating point calculations on just damage would account for a difference as extreme as that.

Interesting. Well, if it's not the LUA implementations themselves (you did use the same ones, only 64 bit) and it's not the floating point calculations, something else?  

EDIT: Forgot to write this before posting!  

I'm still curious though how many people can script a bit of LUA, and would be willing to compile from the trunk to test or comment on the bug fixes.

I guess I could learn and do that but certainly won't have the time before a couple weeks from now. Sorry.

Also I am hoping to add a few more features, more unit control and queries should be fairly easy to implement I think. If anyone has any requests I'd be glad to hear them.

So this really is going the LUA controlled AI coding, huh? Nice!

[wciow]

I'm still curious though how many people can script a bit of LUA

I'm willing to spend a bit of time scripting an actual scenario to showcase the upgraded Lua functions of GAE including all of the stuff Silnarm has added.

I tried the scenario and it worked well, 5/9 Drakes survived.

I would be interested to see exactly how much cpu time this micro management takes up and whether its actually workable on a larger scale? It would also be nice to have a sort of two teir approach, having the LUA functions split into reusable chunks so people can just add an AI block at the beginning of their script and then call the functions for a unit/group, or LUA experts can actually dig down into the functions and change how they work.

[hailstone]

I was thinking about having a script called for each command type. With added lua functions and move command you would be able to do swarming.

[zombiepirate]

Yes, larger scale performance is the big thing. As far as I know LUA is a really fast scripting language though so it may be able to take a lot before it slows anything down. I like the two tier idea too.

I might run some tests to see how the LUA performs later.

[silnarm]

I fixed three bugs in the LUA, and committed to the trunk (which still claims to be 0.2.12; has 0.2.13 not been fully merged into the trunk yet?). If anyone wants to do more extensive testing then I did i'd be grateful for any feedback.

Sweet!

I haven't had a chance to actually test it myself as yet, indeed I haven't even updated my working copy, but looking at the changeset, I was slightly concerned by the command callback fix, it seems likely (but I could be wrong), that setting a command callback on a unit with no commands would crash the game...  Having Unit::anyCommand() return true when there is in fact no command could be disaster, did you try this (admittedly ridiculous) scenario (setting one on an energy_source for example)? While it might seem a silly scripting error, we still don't want the game crashing

The 'last minute' fixes from 0.2.13 haven't been merged yet, but everything else has. Trunk should now report to you the version that is set in CMake, I think Yggdrasil might have added a 'dummy' projectConfig.h for people still using Jam, or using the 'hand maintained' windows project files, that's probably where the 0.2.12 is coming from.  I'd recommend making the move to CMake, it rocks (fully setup Code::Blocks project files, anyone?).  In fact, we may remove the Jamfiles at some stage, and I'm removing those windows project files very soon, unless hailstone wants to maintain them

• The 'attacked' trigger now fires (I put the call in World::hit(...) if that's ok?).

That is the perfect place for such a check

Also I am hoping to add a few more features, more unit control and queries should be fairly easy to implement I think. If anyone has any requests I'd be glad to hear them.

I can't think of any 'fancy' things, but we still need upgrade queries, which I never got around to, as well as 'gifting' upgrades.

Something like,

Code: [Select]

   if hasUpgrade(myFaction, "myUpgrade") then
      --[[ do stuff ]]
   else
      --[[ do other stuff ]]
      if not isUpgrading(myFaction, "myUpgrade") then
         giveUpgrade(myFaction, "myUpgrade");
      end
   end

And of course, all the triggers will need to be 're-wired' using sigslot soon, and all the 'junk' I added to the Unit class will need to be moved into the TriggerManager. Not the most exciting work, but any help would be appreciated. More on that soon, once I've finished addressing sigslot's performance issues

Ps: If you want windows binaries built at any time, PM or e-mail me, I'll get 'snapshots' up for you so more people can try things out.

[zombiepirate]

I haven't had a chance to actually test it myself as yet, indeed I haven't even updated my working copy, but looking at the changeset, I was slightly concerned by the command callback fix, it seems likely (but I could be wrong), that setting a command callback on a unit with no commands would crash the game...  Having Unit::anyCommand() return true when there is in fact no command could be disaster, did you try this (admittedly ridiculous) scenario (setting one on an energy_source for example)? While it might seem a silly scripting error, we still don't want the game crashing

...

Ps: If you want windows binaries built at any time, PM or e-mail me, I'll get 'snapshots' up for you so more people can try things out.

Yes, the game does crash, my test were actually setting command_callback on a horseman after it had been attacked. The horseman already had an auto attack command so it appeared to be functional. I'll see what I can do about it.

If you could get windows binaries up once i get command_callback re-fixed that'd be great. 

[silnarm]

Maybe we can just reject the request for a command callback if the unit is not executing a command? Then the || commandCallback != NULL in Unit::anyCommand() can be removed, and replaced with a check in SimulationInterface::doUpdateUnitCommand() if there is no command,

Code: [Select]

if (unit->anyCommand()) {
    // as per now, including the extra checks you put in
} else {
    if (unit->getCommandCallback()) {
        ScriptManager::commandCallback(unit);
        unit->clearCommandCallback();
    }
}

I think that should cover all possibilities.

[zombiepirate]

Tested, no luck, it doesn't seem to make any difference.

From what I've gathered from my tests; what I think is happening is that the unit never actually ends its current command in the way SimulationInterface::doUpdateUnitCommand() is expecting it to. I'm still not sure why changing Unit::anyCommand() fixes this though.

I printed the values of unit->getCommandCallback() and unit->getCurrCommand() in my terminal every update. Without the edit in Unit::anyCommand() the values often don't change even when the unit has very obviously finished it's command. The current command does, however, change when an explicit stop command is given. A stop command should be given in doUpdateCommand(), but I don't think it ever is. With the edit to anyCommand(), the value of getCurrCommand() changes properly when the unit finishes its command.

What you suggested doesn't work, i believe, because unit->anyCommand() still returns true and the unit updates it's command even when it should be returning false.

If I'm right then the bug is a bit deeper then I had originally thought and may take some work to find. If I'm wrong/being stupid I might actually be happier.

[jda]

Too bad the command_callback isn't working. In your test demo, I'd need it to implement a horsemen win endgame condition without breaking the nice "return home" routine.
Yeah, I did give in to looking at your script and though it's just an unplayable testcase, playing with it is a fun way to start the learning.

[zombiepirate]

I just committed a new fix for the 'command_callback' trigger.

It works around what I believe were actually memory allocation issues. The new commands were often being loaded into the same memory location as the old commands. Since doUpdateCommand() only checked pointers it thought the command was the same.

I added an Id to the commands so they could be referenced by number rather then memory location.

[silnarm]

Yes, using memory addresses was a poor choice

But there was actually no need to encumber Commands with Ids, you should just use the command's type id, unit->getCurrCommand()->getType()->getId()

But nice work, hope this is all works as advertised now. Although, in my own defence, I did previously advertise that command callbacks weren't reliable

[silnarm]

I've posted some snapshot builds, @r598.

Windows: http://sourceforge.net/projects/glestae/files/snapshots/gae_win32-r598.zip/download
Linux (64 bit): http://sourceforge.net/projects/glestae/files/snapshots/gae_linux64-r598.tar.gz/download

If anyone is interested in some advanced scripting, feedback on any of the new features would be most welcome, in particular any problems with command callbacks

[zombiepirate]

Hey all. I just committed a slightly more powerful LUA camera control function if anyone who compiles from svn (and can do a little bit of LUA) wants to test it.

setCameraMotion(position, angle, position frame count, angle frame count, position frame delay, position frame delay)

position is a two element table specifying map coordinates.
angle is a two element table specifying camera angle. Horizontal angle first then Vertical (the default camera angle on entering a game is (0, -60).
position and angle frame count are the number of frames it should take for the camera to reach its destination.
position and angle frame delay are how many frames the camera should wait before starting to move.

Example:

Code: [Select]

setCameraMotion(startLocation(1), {0 , -85}, 240, 160, 0, 240)This should move the camera to start location 1 in 240 frames and then tilt the camera until it is looking nearly straight down.

[Omega]

Looks totally awesome Zombie! Of course, I haven't tried it, but your description has me thinking of great things. Combine this with Silnarm's new method of NPC dialogue with colored text and that could make some pretty cool cutscenes. Now if only there was a precise method of moving units (and killing em off).

[silnarm]

Hey all. I just committed a slightly more powerful LUA camera control function if anyone who compiles from svn (and can do a little bit of LUA) wants to test it.

Forgive me for waiting nearly a month to try this out myself, but I finally had a play with it.

This is awesome
"slightly more powerful" lol!
 
I'm removing my hacky setCameraAngles and setCameraDest functions, I'll leave ticket #39 open for now, would you like to do the honours ??