Author Topic: Merging Glest 3.2.2  (Read 16792 times)

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Merging Glest 3.2.2
« on: 25 June 2009, 01:42:15 »
This topic is to organise the merging of Glest 3.2.2 with latest GAE trunk. Programmers and testers wanting to participate should post their interest.

@silnarm: Were you wanting to lead this project?

Perhaps start at shared_lib and note any interface changes. Also it might be good to have a diff of Glest 3.2.2  and an earlier version of GAE (to get a reference) with latest GAE trunk.
« Last Edit: 25 June 2009, 01:51:30 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #1 on: 25 June 2009, 05:31:37 »
@silnarm: Were you wanting to lead this project?
Ok, I'll 'lead the charge' :)

Quote from: hailstone
Perhaps start at shared_lib and note any interface changes. Also it might be good to have a diff of Glest 3.2.2  and an earlier version of GAE (to get a reference) with latest GAE trunk.

shared_lib would indeed be the place to start, and I've already cleaned up the diffs.  There really isn't too much new in shared lib, lua of course, but we can just 'drop' that in.

Good idea on diffing against an earlier GAE and current trunk... I'll set up some local repositories and do some more diffs.

Happily, I have today and tomorrow off, and then the weekend :)

Unfortunately, upon discovering just days ago Glest's modified A* is really just a (Greedy) Best First Search and thoroughly undeserving of the name A*, I've implemented a proper A*, and it needs a tiny bit more tweeking...

I have however already decided on the ideal data-structure for it, a 'Treap' which is nice and simple (relatively, perhaps). So I should be 'done' with that tonight or tomorrow.

If you wanted to get started on the merge, here's the cleaned up and annotated diffs for shared lib,

http://www.mediafire.com/download.php?mz3dzznu3u2

Cheers.
Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #2 on: 25 June 2009, 08:12:59 »
There really isn't too much new in shared lib, lua of course, but we can just 'drop' that in.
Nice. I don't think glest_game will be as easy though.

I have however already decided on the ideal data-structure for it, a 'Treap' which is nice and simple (relatively, perhaps). So I should be 'done' with that tonight or tomorrow.
That's the first time I've heard of that data structure. I've studied binary search trees and heaps before but not as one data structure.

If you wanted to get started on the merge, here's the cleaned up and annotated diffs for shared lib,
Should we use another branch or just commit to trunk?

And for a record current revision is 56
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #3 on: 25 June 2009, 11:07:44 »
Nice. I don't think glest_game will be as easy though.
I can assure you it wont ... the diff is huge, but I haven't pruned it properly yet... I made sure line endings matched, but there was still a lot stuff that was identical that I stripped out of the shared lib diffs... not much fun :(

Quote from: hailstone
That's the first time I've heard of that data structure. I've studied binary search trees and heaps before but not as one data structure.
It sure is a nifty one, and with a very simple modification we can send the tree out of balance deliberately, favouring a low height 'to the left' where the 'good' nodes are, and letting the tree balloon out to the right, where the crappy nodes are kept. Worth a little effort I reckon :)

Quote from: hailstone
Should we use another branch or just commit to trunk?

Yeah, the path finder doesn't need to be merged now, it's fairly well self contained anyway, so it should be easy to merge later anyway.
LUA is embedded in the XML, so maybe we should get it working with xerces so you have a 'reference' for any changes made to the xml parser.

So go ahead and branch if you want to get started, I'll be in there come Saturday  :)
Glest Advanced Engine - Code Monkey

Timeline | Downloads

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Merging Glest 3.2.2
« Reply #4 on: 25 June 2009, 16:33:21 »
I'll be happy to test it once you are finished.

Because 3.2.2 has lua, I assume this means you are adapting lua into GAE (at last)? Would this make GAE version 3 now?

Also, will you consider adding a few new lua features into GAE while you do the Lua?

What all is new in 3.2.2 that isn't in GAE? I can only think of menu layout (please show IP like 3.1.2 did), lua, and dump files for errors. Oh, and maybe a few bug fixes, but not much else...
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #5 on: 25 June 2009, 23:06:08 »
I'll be happy to test it once you are finished.
Thanks!

Quote from: omega
Because 3.2.2 has lua, I assume this means you are adapting lua into GAE (at last)? Would this make GAE version 3 now?
Also, will you consider adding a few new lua features into GAE while you do the Lua?

I'd be happy to add new features for scripting, but only after the merge... we don't want to be changing things as we integrate them. After we've got it all working again, and it appears to be stable, then we can look at extending it.  Your request for timers has not fallen on deaf ears, it will be straight forward to implement, but it will have to wait until the merge is complete.

Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #6 on: 26 June 2009, 00:30:31 »
Would this make GAE version 3 now?
Once the merge is done then I think we could do a tag with 0.2.12. I'd like to merge tinyxml and pathfinder branches and either remove access to multiplayer or revert it back to vanilla Glest multiplayer before becoming 0.3.

I can only think of menu layout (please show IP like 3.1.2 did), lua, and dump files for errors.
If multiplayer stays for 0.3 it would depend how much work it is not forcing people to bind the port and such. I've left the dump files for now but it should be ok to add later. What menu organisation do you prefer?

I've committed the shared_lib changes and modified the project to include lua. I'm curious if virtual void renderSystemLineAlpha(ParticleSystem *ps); in particle_renderer_gl.h and graphics/particle_renderer.h should be added.

Edit: I noticed Daniel checks if a pointer has memory before deleting. Is there any reason for this.

Code: [Select]
NetworkManager::~NetworkManager() {
if(gameNetworkInterface) {
delete gameNetworkInterface;
}
}
« Last Edit: 26 June 2009, 00:33:58 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Merging Glest 3.2.2
« Reply #7 on: 26 June 2009, 02:43:50 »
I'd be happy to add new features for scripting, but only after the merge... we don't want to be changing things as we integrate them. After we've got it all working again, and it appears to be stable, then we can look at extending it.  Your request for timers has not fallen on deaf ears, it will be straight forward to implement, but it will have to wait until the merge is complete.
Fantastic! I tried to simulate a timer by using the os.time() lua function, but for some reason, Glest only seems to get the start time, and will never update. Is lua run real time or statically at the start? Guess that's why its necessary for a command for timing. As well as a grid coordinates (x,y) for placing units, attack/move commands, etc; Those are the two that could be most important, though I agree that first merge, then extra lua later.

Concerning menu, should the menu be exactly like 3.2.2? I found nothing wrong with 3.1.2's menus. If one does decide to adapt the 3.2.2 menu, we will get more room for buttons though, which may be good. You're coding, you decide, just once again I ask that you show the IP, because my ISP does not give me one straight IP, but seems to like to hook me with different ones, forcing me to do a quick google search for an IP site. Works, but glest had such a feature, and martino never gave a reason for removing it. Inexperienced players have no clue their IP!

Dump files can wait, but it is just one of the very few things separating GAE from Glest. I've never seen what they look like, because I never upgraded my glest from the beta, which only had the dump files and bug fixes, but they supposedly have better crash logs (?).
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #8 on: 26 June 2009, 04:57:30 »
Edit: I noticed Daniel checks if a pointer has memory before deleting. Is there any reason for this.

Code: [Select]
NetworkManager::~NetworkManager() {
if(gameNetworkInterface) {
delete gameNetworkInterface;
}
}

That's because modern operating systems don't take well to you trying to delete memory you didn't allocate ;)
You should always do this if there is a chance it doesn't point at something you allocated...
In short, if you're using them locally (in a function) and you know you allocated the mem, no need to test.
if it's a member variable, probably best to test it.

Fantastic! I tried to simulate a timer by using the os.time() lua function, but for some reason, Glest only seems to get the start time, and will never update. Is lua run real time or statically at the start?

Sounds like it's not hooked up... either way, wont be hard :)
Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #9 on: 26 June 2009, 08:18:57 »
That's because modern operating systems don't take well to you trying to delete memory you didn't allocate ;)
You should always do this if there is a chance it doesn't point at something you allocated...
In short, if you're using them locally (in a function) and you know you allocated the mem, no need to test.
if it's a member variable, probably best to test it.
I did some more research into this. The reason why it works is because he sets it to NULL in NetworkManager::end() otherwise it would be testing a hex value which from what I tested is a true expression (maybe not for all though) and then double-deleting results in undefined behaviour. Turns out that you don't need to check for NULL (see http://www.parashift.com/c++-faq-lite/freestore-mgmt.html#faq-16.8 ).
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #10 on: 26 June 2009, 09:50:58 »
Yeah, I should have realised that argument was flawed... the only way the test is going to be false is if it points to NULL, and deleting NULL is apparently perfectly safe.  And you are quite correct about logical tests, in C/C++ a zero is false, anything else is true... so if you didn't initialise the pointer to NULL, didn't allocate anything to it, and then did this, the test would be true (the pointer will have a 'junk' value) and you proceed to delete the memory from a 'junk address', then the OS would crack it. Of course without the test you get the same result... the real lesson here is, Always initialise pointers!

Edit: another top link hailstone, I've already found something else I've been doing wrong ...
« Last Edit: 26 June 2009, 10:16:48 by silnarm »
Glest Advanced Engine - Code Monkey

Timeline | Downloads

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #11 on: 28 June 2009, 02:48:00 »
Ok, the shared lib is looking good. Compiles cleanly and the game runs.

Now for the fun part  :-\
As with shared lib, there will be more stuff in the diffs than is actually missing. Unfortunate perhaps, but at least we don't have to add more code that could break things ;)

Here's what I'm thinking, do the files one at a time (or two, if there are changes in header and module file).

Upon adding some code, tag it with a merge comment...
Code: [Select]
//MERGE ADD START
addedNewCodeHere ();
//MERGE ADD END

Where deletion of code is called for, just comment out the code and tag it thus
Code: [Select]
//MERGE DELETE START
/*
   someOldCode ();
   thatWeDontWant ();
*/
//MERGE DELETE END

Edit: for one line additions/deletions, use //MERGE ADD, or //MERGE DELETE, immediately before the line, omit the START and corresponding END.

Once you've done the file, try to compile it. If it compiles move on, if not then find the offending undeclared function call(s) or variables or whatever they are, and comment it(/them) out...

Code: [Select]
//MERGE ADD START
//addedNewCodeHere ();
//butItWontCompileYet ();
//MERGE ADD END

Probably best to keep
Code: [Select]
//MERGE ADD START
//MERGE ADD END
//MERGE DELETE START
//MERGE DELETE END
in the clipboard, the idea is changes will be easy to find later with a grep/find all. So, no typos!
It'll be a bit tedious, but I think a bit of care here will help a lot later.
We'll do the "bit's and pieces" first, and the 'world' and 'game' directories last,
I'm going to start on the 'facilities' and 'menu' directories now, let me know when you're ready and I'll mail some patches through...

PS: diff just pulled the files in alphabetically, so afile.cpp comes before afile.h, make sure to check for header changes before you try to compile a *.cpp.
« Last Edit: 28 June 2009, 03:38:26 by silnarm »
Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #12 on: 28 June 2009, 09:36:14 »
That's good. I'm ready.

I'm not sure that it's necessary to use all the comments because a diff will tell you the changes that were made (although that might be tedious too). We'll start out doing the comments and see how we go.

EDIT: In source/glest_game/graphics/renderer.cpp you have //MERGE ADD then //MERGE DELETE on the next line. Did you forget to add the new line or is not meant to be there?
« Last Edit: 28 June 2009, 11:09:53 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #13 on: 28 June 2009, 11:41:15 »
I'm not sure that it's necessary to use all the comments because a diff will tell you the changes that were made (although that might be tedious too). We'll start out doing the comments and see how we go.
Yeah, I actually stopped doing it for some stuff, if I was sure it wasn't going to cause problems...

Quote from: hailstone
EDIT: In source/glest_game/graphics/renderer.cpp you have //MERGE ADD then //MERGE DELETE on the next line. Did you forget to add the new line or is not meant to be there?
Good question! Don't happen to know the answer do you ?

Anyway, it's all in there now... and I'm down to 49 errors...
I'll mail through all the patches so you've got a reference for investigations, but at this stage it's probably easier to just check the 3.2.2 files.  There are a few spots where the code-bases had diverged a bit... so there's going to be plenty of investigative work to be done :)

Will commit everything within an hour or two... it wont compile ;)

Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #14 on: 30 June 2009, 00:19:44 »
From the latest commit you say there is only three related errors left. Were you wanting me to do anything?

Quote
Will commit everything within an hour or two... it wont compile Wink
I'll let you off this time  :P
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #15 on: 1 July 2009, 10:41:38 »
Sorry, didn't notice the message before the quote...

Got rid of the compilation errors now, and the first couple of runtime errors...

names and directories of tilesets/maps/ etc are handled differently though... the current crash is in Checksum::addFile(), if you want to set a break point there and start investigating, I think I'm done till at least tomorrow some time :)
Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #16 on: 2 July 2009, 02:26:53 »
How are you using Lua?

I'm getting this error using the Windows binary when trying to run:
LDR: LdrpWalkImportDescriptor() failed to probe H:\GAE_0.2.11\lua5.1.dll for its manifest, ntstatus 0xc0150002

UPDATE: I downloaded the source and compiled it myself and now it's working. Only the static lib worked though. The dll says there is a bug or something, I forgot to record the message.
« Last Edit: 2 July 2009, 03:55:16 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #17 on: 2 July 2009, 03:50:03 »
How are you using Lua?

I'm getting this error using the Windows binary when trying to run:
LDR: LdrpWalkImportDescriptor() failed to probe H:\GAE_0.2.11\lua5.1.dll for its manifest, ntstatus 0xc0150002

Hmmm... I installed 'lua for windows'
http://luaforge.net/frs/?group_id=377

and am linking against 'lua5.1.lib'

Gave me no issues at all...
Glest Advanced Engine - Code Monkey

Timeline | Downloads

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #18 on: 2 July 2009, 04:00:04 »
That's the third download page for Windows Lua libs I've seen now.

EDIT:
The path is wrong in World::loadTileset.
Code: [Select]
tileset.load( "tilsets/" + game.getGameSettings().getTileset(), checksum);
Trying to figure out why the tileset isn't loading now.

EDIT 2:
It now runs as expected. There were some paths missing from MenuStateNewGame::loadGameSettings(). Might also be an issue with MenuStateScenario::loadGameSettings(). Would it be better to put the strings in game settings like?
Code: [Select]
const string &getMap() const {return map;}
const string &getTileset() const         {return tileset;}
const string &getTech() const {return tech;}
const string &getMapPath() const {return "maps/" + map + ".gbm";}
const string &getTilesetPath() const {return "tilesets/" + tileset;}
const string &getTechPath() const {return "techs/" + tech;}

- Still need to test the Lua scripts.
- The exit window asks if you want to quit and then only one button saying No. (Probably need changes in GraphicMessageBox::init(... , ... , ...) )
« Last Edit: 2 July 2009, 08:58:18 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #19 on: 2 July 2009, 09:34:19 »
It now runs as expected...
Sweet...

Quote from: hailstone
There were some paths missing from MenuStateNewGame::loadGameSettings(). Might also be an issue with MenuStateScenario::loadGameSettings(). Would it be better to put the strings in game settings like?
GameSettings got changed somewhere along the way... the diffs obviously didn't help us much on that one. The 'new way' is to store just the name, then append path and extension when needed.  That scheme you propose gives us the best of both worlds, nice one!

Quote
- Still need to test the Lua scripts.
- The exit window asks if you want to quit and then only one button saying No. (Probably need changes in GraphicMessageBox::init(... , ... , ...) )
If it's up and running, there is the menus issue to sort out... I didn't want to change the menu sequence so I skipped the diffs for it...  the sequence of menus to there and the normal new game screen has changed in 3.2.2. I didn't want to mess the gui much because it might be next on our hit-list anyway...

Edit: I 'dodged up' the rest of the path problems, scenarios load, but we're not getting any messages.
« Last Edit: 2 July 2009, 11:42:28 by silnarm »
Glest Advanced Engine - Code Monkey

Timeline | Downloads

wciow

  • Behemoth
  • *******
  • Posts: 968
    • View Profile
Re: Merging Glest 3.2.2
« Reply #20 on: 2 July 2009, 15:52:43 »


Quote
If it's up and running, there is the menus issue to sort out... I didn't want to change the menu sequence so I skipped the diffs for it...  the sequence of menus to there and the normal new game screen has changed in 3.2.2. I didn't want to mess the gui much because it might be next on our hit-list anyway...

Any chance you could leave the old style unit loading menu the way it is. I Prefer it to the new one.
Check out my new Goblin faction - https://forum.megaglest.org/index.php?topic=9658.0

hailstone

  • Local Moderator
  • Battle Machine
  • ********
  • Posts: 1,568
    • View Profile
Re: Merging Glest 3.2.2
« Reply #21 on: 3 July 2009, 02:09:32 »
I fixed up the paths but it's a bit different to what I posted before. There are still literals where it doesn't suite using gameSettings though.

I have the message boxes loading but the lng files don't seem to be loading. I haven't committed anything yet.

I think we should stick with the menu structure we have now (ie no submenus). I can finish off the menu stuff if you want.

Instead of having tutorials in a separate folder I think we should have category folders then the scenarios in them.
eg scenarios/tutorials/TheScenario, scenarios/battle/TheScenario

This way, when I add in the new GUI, we can use a drop down box for categories and a list box for the scenarios.

Quote from: wciow
Any chance you could leave the old style unit loading menu the way it is. I Prefer it to the new one.
This has already been changed but it can be changed back later.

UPDATE: Changes have been committed

EDIT:
- Scenario lng files are working now.
- showMessage('intro', 'mission') shows the message but when you click OK it doesn't let you select using the rectangle.

UPDATE: added quick fix to use rectangle select with showMessage. Needs more researching.

EDIT:
When Lua script is read it processes them all at once so it doesn't wait for the first message box to be responded to before the next one is processed. So when I set the first one not enabled the second also became not enable but is still shown on the screen.
« Last Edit: 3 July 2009, 04:56:22 by hailstone »
Glest Advanced Engine - Admin/Programmer
https://sourceforge.net/projects/glestae/

silnarm

  • Local Moderator
  • Behemoth
  • ********
  • Posts: 1,373
    • View Profile
Re: Merging Glest 3.2.2
« Reply #22 on: 3 July 2009, 07:55:41 »
Quote from: wciow
Any chance you could leave the old style unit loading menu the way it is. I Prefer it to the new one.
This has already been changed but it can be changed back later.

I was careful to leave everything to do with this 'commented out' not deleted, I want it back too! Scenarios use the same mechanism to show messages in game though, so I've just changed it for now. I will separate them at some point though, so we can have our old loading screen back.

Quote from: hailstone
When Lua script is read it processes them all at once so it doesn't wait for the first message box to be responded to before the next one is processed. So when I set the first one not enabled the second also became not enable but is still shown on the screen.

The 'MessageBox' is more functional than it used to be, and we appear to missing some of that functionality :)

I'll leave it with you for now, if you don't mind... almost got the pathfinder ready... abandoned the treap, just using the STL heap algorithms for the open list, but I've introduced a most annoying bug, which only manifests itself in release build...

Glest Advanced Engine - Code Monkey

Timeline | Downloads

titi

  • MegaGlest Team
  • Airship
  • ********
  • Posts: 4,240
    • View Profile
    • http://www.titusgames.de
Re: Merging Glest 3.2.2
« Reply #23 on: 3 July 2009, 11:03:54 »
All this merging sounds very promising! Hopefully we will end up with a stable GAE version!
What about multiplayer? Which one did you choose for now? The original one from glest 3.2.2 or the one from daniel ?
Try Megaglest! Improved Engine / New factions / New tilesets / New maps / New scenarios

Omega

  • MegaGlest Team
  • Dragon
  • ********
  • Posts: 6,167
  • Professional bug writer
    • View Profile
    • Personal site
Re: Merging Glest 3.2.2
« Reply #24 on: 3 July 2009, 21:11:24 »
Sounds like work is going well. Once completed, I'll put together a lua script that will test every function in a variety of ways to make sure that it works.

BTW: an idea for the future: should lua be able to control some settings like day-night time, fog of war, weather, etc; Of course, don't worry about it now. Focus on finishing the merger. Good work being done!
Edit the MegaGlest wiki: http://docs.megaglest.org/

My personal projects: http://github.com/KatrinaHoffert