AI/Balancing Strategies and Debate

[will]

What we really need is a tool that does a simple "if you have 2 factions, and this much time, who wins?" Doing a monte carlo sim.  Useful for balance within existing mods, useful for the idea of a scifi tree for all.

(Simple as in inexact, not simple as in trivial to implement.  I imagine it actually being easiest accomplished by a headless ai-only game played as fast as possible and recording the results of a few gazillion such games.  This is as inexact as the ai is bad.  Hmm, bonus is helpful in developing more human-feeling ai?)

To my mind, different resource structures are compatible.  It is all about hp at any particular part of a timeline.

[Mr War]

monte carlo? I'm not building that sir, that's a lot of effort

[Omega]

Just do a few random games with different settings each time. That's basically the Glest equivalent.

[Psychedelic_hands]

What we really need is a tool that does a simple "if you have 2 factions, and this much time, who wins?" Doing a monte carlo sim.  Useful for balance within existing mods, useful for the idea of a scifi tree for all.

(Simple as in inexact, not simple as in trivial to implement.  I imagine it actually being easiest accomplished by a headless ai-only game played as fast as possible and recording the results of a few gazillion such games.  This is as inexact as the ai is bad.  Hmm, bonus is helpful in developing more human-feeling ai?)

To my mind, different resource structures are compatible.  It is all about hp at any particular part of a timeline.

That would all be pointless in a multiplayer match, the AI could play the faction in a terrible way when really with a Human playing it could be far over powered. The best thing is just play test, play test, play test.

[will]

That would all be pointless in a multiplayer match

What we really need is a tool that does a simple "if you have 2 factions, and this much time, who wins?" Doing a monte carlo sim.
...
This is as inexact as the ai is bad.

1) players are not balanced either.  You winning 100% against me does not mean your choice of side is more powerful than mine, it means you are cleverer than me.

2) we want to improve AI and this is both a quantitative and qualitative help with that too.

[Psychedelic_hands]

That's why you play test with lots of different people. It's usually obvious when something needs to be toned down.
But opinions from a playtester will be more valuable than a computerised test.
Really, balance is quite subjectiveas long as the game is still fun I consider it balanced!  

Something maybe that could be useful is a program the pits X amount of  YZ units together in a fight, and you get to watch what happens.

[Zoythrus]

Something maybe that could be useful is a program the pits X amount of  Y units together in a fight, and you get to watch what happens.

have you heard of this? https://forum.megaglest.org/index.php?topic=6254.0

[ultifd]

For now, the best and technically the only way to balance techtrees or mods well would be to just play/test multiplayer, it's usually very effective. Besides, both of the teams have enough on their hands...

[-Archmage-]

Test in multiplayer for loopholes, then in AI vs AI to make sure the AI can handle it.

[ultifd]

Well, that's somewhat true,although I think it would be somewhat boring in observer mode. Usually during multiplayer, if there are any obvious issues it's seen, but I guess for more specific/thorough testing observer mode would be better.

[will]

Making the game headless (and running as fast as possible) would make it possible to run the "AI-only arena" much faster and more often, with results recorded for study.  A timeline of unit births and deaths would help too.

Yes player games are the very best way to decide balance.  But players are a rare commodity and we have as few of them as we do developers

AI-only games will give valuable quantity to any attempt at balancing.

[-Archmage-]

Those of us who have made mods, or are mod developers, know that the AI testing determines the main balance, while the player testing(even if it's not multiplayer) is to make sure the mod isn't too hard for a person to play or too easy(loopholes), such as the battle-machine spam in tech.

You need a mod to be playable by people and AI, you need it to be balanced AI vs AI, AI vs Human, and Human vs Human. The AI uses mathematics, while the Human player uses intelligence. You have to have mathematical balance, and you have to make sure that you seal up loopholes, so that Human players don't exploit them and ruin the mathematical balance. Some one person may think that a unit is too weak, but that's generally meaningless unless a group agrees, because their problem could be a result of their play-style.

Anyway, I sound like a doctor(Dr.Arch)....and I'm gonna head to bed.

[titi]

Is the idea to have something automated to test or a half automated way?

When I start to balance things I do the following:
1) Set speedloops in glestuser.ini to something high
2) Setup a 1 on 1 game in map conflict for the cpus techs to test and add you as a observer
 ( It turned out that the map conflict is very good to do this )
3) Watch them play and speedup things if needed ( which will be very fast if you set speedloops to 10 for example )

After this kind of balance its like Archmage says: You have to play with humans ....

[will]

So titi, if it ran in a loop - after one side wins, it starts a new game with the same settings - and it wrote the scores (and perhaps unit birth and death timelines) to a log file...

sure its nice if its 'headless', but if its manual to run but you don't have to watch it, then you can leave it running all night...

[Mr War]

I have been working on balancing maths based on comparing two units

I could build a lightweight Monte Carlo macro in excel but would have to install open office or something

[titi]

@will: If you let it play one game, the next game will be played exactly the same way( at least in MG ) because the randoms are calculated....

[will]

so who ever adds the code to record the outcome and restart will have to generate a new seed.  Seems insurmountable

[Ishmaru]

@will: If you let it play one game, the next game will be played exactly the same way( at least in MG ) because the randoms are calculated....

I've seen this its quite strange. IS there any better way to calculate random like using system clock?

[Omega]

@will: If you let it play one game, the next game will be played exactly the same way( at least in MG ) because the randoms are calculated....

I've seen this its quite strange. IS there any better way to calculate random like using system clock?

It could easily be changed to use a different seed every game. As it is, though, it keeps the same seed through the session.

[ultifd]

Those of us who have made mods, or are mod developers, know that the AI testing determines the main balance, while the player testing(even if it's not multiplayer) is to make sure the mod isn't too hard for a person to play or too easy(loopholes), such as the battle-machine spam in tech.

Yep, that's mostly true...although most of the time, as I said,  for obvious/semi-obvious player testing is fine too. This comes from a tester of engines, games, and mods.

(click to show/hide)

Good analysis, doctor. (Well, archmage=magician=somewhat of a doctor)

On a related note, it seems if you have a pack of factions which all have an air unit, and one doesn't...it's hard to balance. Just is currently how it is with the Romans faction... 

[-Archmage-]

Those of us who have made mods, or are mod developers, know that the AI testing determines the main balance, while the player testing(even if it's not multiplayer) is to make sure the mod isn't too hard for a person to play or too easy(loopholes), such as the battle-machine spam in tech.

Yep, that's mostly true...although most of the time, as I said,  for obvious/semi-obvious player testing is fine too. This comes from a tester of engines, games, and mods.

Without some AI to AI testing you can't really lock down mathematical balance.

Good analysis, doctor. (Well, archmage=magician=somewhat of a doctor)

[Ishmaru]

The fault of AI to AI testing is that Humans have the ability to plan and understand things such as exploits while AI just does what its programed to do. For example you have a Tank unit that can spawn infantry that carry C-4 explosives. The AI will simply follow whats its coded to do for example to drive near enemy units and unload. A human player could try to bring apc right next to hq and spawn anti structure units constantly from the heart of another player base, or constantly spawn infantry right next to a resource patch, and player would know that this is so effective that he would only need to build just 3 of these tanks and nothing more. The ai will build in glest case based on random. So player has many benefits as well.

[ultifd]

Without some AI to AI testing you can't really lock down mathematical balance.

For "advanced" units, yes, but for normal or simple units it's usually it's not to hard to balance it out.

The fault of AI to AI testing is that Humans have the ability to plan and understand things such as exploits while AI just does what its programed to do. For example you have a Tank unit that can spawn infantry that carry C-4 explosives. The AI will simply follow whats its coded to do for example to drive near enemy units and unload. A human player could try to bring apc right next to hq and spawn anti structure units constantly from the heart of another player base, or constantly spawn infantry right next to a resource patch, and player would know that this is so effective that he would only need to build just 3 of these tanks and nothing more. The ai will build in glest case based on random. So player has many benefits as well.

Yep, both have it's own benefits. Though with player testing you can have fun too.

[Gabbe]

Something like AI vs AI, speed up and let it use the whole processor. Then when it has completed about 100 matches, it gives you a log file wich contains win loss ratio for the factions?

[Mr War]

If people want to do this type of modeling as a separate tool, like in excel, Olaus has some cool tools to extract info from xmls