Paths

[hailstone]

I was thinking that there should be a paths file that specifies the path either relative to the executable or absolute path. That way it doesn't matter if we move folders and files around since the paths can easily be changed, one time, with the file.

Example paths.ini:

Code: [Select]

BaseDir = /usr/games/glestadv
UserDir = ~/.glestadv

Config = %UserDir%/glestadv.ini; %BaseDir%/glestadv.ini
Keymap = %UserDir%/keymap.ini; %BaseDir%/keymap.ini
Servers = %UserDir%/servers.ini; %BaseDir%/servers.ini

LogDir =
DataDir = gae/data
LangDir = %DataDir%/lang
ScenarioDir = gae/scenarios
TilesetDir = tilesets
MapDir = maps
TechDir = techs

Semicolons for alternative paths.

[daniel.santos]

I'm pretty tired, but I don't want this thread lost. In short, I kinda like the way wesnoth does add-ons.  Each add-on gets it's own directory and from there, you can have additional maps, eras (their tech tree), campaigns & scenarios -- I like this approach.  This allows you to modularize add-ons so you can download Bill's "xyz" add-on that contains a tech tree, maps, scenarios and a campaign that glues them all together.  Then Jack can write a "pdq" add-on that re-uses Bill's tech tree and a few of his maps, but then adds a few more maps and his own scenarios & campaigns and simply marks Bill's "xyz" add-on as a dependency.

I'm not sure I like the paths.ini exactly as you specified, I think we need default options that are specified in the configure script because different linux distributions put these things in different places.  Also, you generally can't write the /usr directory.  So maybe we'll have pre-processor values with some sane defaults, but also accept command line overrides like
glestadv --userdir=~/.glestretarded
or
glestadv --config=~/.glest-super-advanced/g.ini
or
glestadv --datadir=/usr/local/share/games/pitbull-casino/dancing-poodles/macaroni-and-cheese/

OK, I need sleep now

[daniel.santos]

Well, to follow up on this, I'm starting to think about packaging a little bit more.  If we're going to finally make installers, we should maybe modify the build system to create packages for the basic Linux distros, that is, .deb and .rpm.  Like Windows installers, Debian & Redhat packages can be installed & uninstalled -- very important if you're going to run as root and write to system-wide directories.  So whatever we come up with should be nice analogous to windows and it's installer.  Also, it should retain the ability to run from a local directory, if somebody wants to install the game into their own home directory (i.e., they don't want to install it globally or they don't have the privs to do so).

As far as build system, I've had more experience with autotools since the last time this subject came up, so maybe early next year I can redo the build system using autotools if nobody else steps up to the plate with that or something comparable.  I'm not opposed to CMake or SCons as long as whomever does it leaves some documentation to tell us how to use it properly, because I don't know much about either of those.   The main problem I have with the current build system is that it's a non-standard, non-popular mechanism that Matze came up with.  Not that that is bad in and of its self, there's a lot of very clever stuff in it that we're not even using (like the ability to create MS Visual C++ project files!), but Matze isn't available to help with changes and I have found it very difficult to modify & maintain.

[Yggdrasil]

Problem with the path.ini approach: We need to have a fixed place where the path.ini file is located, relative to the binary. I really dislike to have a config file next to the binary, e.g. in /usr/bin.

I'm not sure I like the paths.ini exactly as you specified, I think we need default options that are specified in the configure script because different linux distributions put these things in different places.  Also, you generally can't write the /usr directory.  So maybe we'll have pre-processor values with some sane defaults, but also accept command line overrides like
glestadv --userdir=~/.glestretarded
or
glestadv --config=~/.glest-super-advanced/g.ini
or
glestadv --datadir=/usr/local/share/games/pitbull-casino/dancing-poodles/macaroni-and-cheese/

I agree. There should be defaults set at compile time.

useful paths:
datadir - root dir for the installed data, for all users of the system
userdir - not really needed but nice to have, user specific data directory for e.g. mods, looked before datadir ("overwrites")
configdir - directory for configs (*.ini) and logs, not set at compile time, getenv("HOME")/.glestae for unix, getenv("UserProfile")/.glestae for windows

datadir and configdir are essential. datadir is really easy to achieve: Just run chdir to this directory right at the start of glest. configdir needs some more changes to Logger and the other config files.

Well, to follow up on this, I'm starting to think about packaging a little bit more.  If we're going to finally make installers, we should maybe modify the build system to create packages for the basic Linux distros, that is, .deb and .rpm.

This would be nice but is not really needed. We just need good install routines. The distributors make the rest.

As far as build system, I've had more experience with autotools since the last time this subject came up, so maybe early next year I can redo the build system using autotools if nobody else steps up to the plate with that or something comparable.  I'm not opposed to CMake or SCons as long as whomever does it leaves some documentation to tell us how to use it properly, because I don't know much about either of those.

As i wrote on the mailinglist, i have a more or less functional cmake build system (currently not working on windows). The extension cpack is able to generate installers for windows, deb and rpm packages from the install routines and some additional settings.

[daniel.santos]

As i wrote on the mailinglist, i have a more or less functional cmake build system (currently not working on windows). The extension cpack is able to generate installers for windows, deb and rpm packages from the install routines and some additional settings.

awesome!!!

I posted to the mailing list, so I hope somebody replies to let me know it went through.  If not, I need to hassle with it more until I figure out how to get the *$)#@(* mailing list to accept my emails.

Actually, userdir will be very important, as I'm darn sure not going to sudo to root and write mods into the /usr directory.   It's not uncommon on *nix for userdir=~/.glestae and configdir=~/.glestae/config or some such.  For *nix systems (which includes Mac now) , I would certainly prefer there to be only one directory for this.  The same can probably hold true for windows, although windows pukes (in various ways) on directories and files that start with a dot.  Surprising?  Accident?  Unlikely.  So for Windows, the "%USERDIR%/Application Data/GlestAE/" directory could probably be used (where %USERDIR% should translate into something like "C:\Documents And Settings\your_user_name").  We don't currently support searching in multiple directories for things, but this is something I've been discussing on & off (possibly via physfs or maybe some SDL_ filesystem library?).

Add-ons
So this is a bit of a tangent, sorry.  Basically, I want to have the concept of add-ons that will have their directory.  So if I install both a "Jack" and a "Jill" add-on, the directories might look something like this:

Code: [Select]

~/.glestae/addons/Jack
~/.glestae/addons/Jack/maps
~/.glestae/addons/Jack/tilesets
~/.glestae/addons/Jack/tilesets/hilltop
~/.glestae/addons/Jack/tilesets/hilltop/models
~/.glestae/addons/Jack/tilesets/hilltop/sounds
~/.glestae/addons/Jack/tilesets/hilltop/textures
~/.glestae/addons/Jill
~/.glestae/addons/Jill/scenarios
~/.glestae/addons/Jill/scenarios/buckets_and_pregnancy

I want some type of standard file that must be in the root directory of each add-on for GAE to even install it.  This file should specify version & dependencies (of other add-ons).  Each dependency entry should specify an add-on name and a version mask.  Example, the file ~/.glestae/addons/Jill/info may contain something like:

Code: [Select]

<addon name="Jill" description="Jill's XXX Adventure" version="0.3a">
    <dependencies>
        <dependency name="Jack" min-version="0.2"/>
    </dependencies>
</addon>

Ideally, however, it should accept version ranges with exclusions for known bad versions, etc. But initially, we can skip the version specification (for add-on dependencies) entirely.

Each add-on can optionally have any of the directories maps, tilesets, scenarios, lang, techtree and core (and any other add-on-able objects we come up with later).  The "core" directory will only be used if GAE is started with an option that tells it to use an alternate set of core data -- this will eventually facilitate the "entire game modification" we've been talking about, where the 3d background, screens, music, GUI screens, look & feel, etc. can be altered so that you don't even have to know that it's running on the GAE engine.  I would like for GAE to eventually be able to "become" a different game just by installing an add-on and specifying to use its core data instead of the default "Glest" game.

[hailstone]

The paths.ini file doesn't need to be in the same dir as the executable if you use the command line args to specify where it is. Although I don't  mind how it's done as long as there is something for all supported platforms.

[Yggdrasil]

Add-ons
-- Snip --

This looks similar to the mod specification in spring.
http://springrts.com/wiki/Mod_specification

spring itself is just an engine, all games are mods. I think they support 2 data directories, system wide and in home.

Will have a look how they do it because they have also cmake (and scons).

[Yggdrasil]

Sorry for doublepost, but otherwise nobody would notice.

configdir and datadir are now implemented in trunk. Both is set at compile time through the header projectConfig.h.

spring has its own virtual filesystem implementation. Did not dive in the code because i think using physfs would be much better. It supports zip and 7z archives, and other engine specific archives like quake stuff. The interface is quite easy. We would need to map all file operations to equivalences in physfs. Not sure how much work this would be. Will have a look and upload a branch if i get something to work.

[Yggdrasil]

triplepost ftw.

I uploaded a new branch called physfs with an initial working physfs integration. It only builds on linux with cmake currently and needs some more cleanup. So it might crash now and then.

Nevertheless i was able to play a game with a zipped megapack in ~/.glestadv/addons. Need to repack it, so that only maps, scenarios, techs and tilesets folders are in it at toplevel. You can also unpack it in it's own folder.

lzma aka 7z is currently broken. Seems to be a problem in physfs itself.

[Omega]

It only builds on linux with cmake currently and needs some more cleanup. So it might crash now and then.

Shame on you.

What is this even needed for?!?

[Yggdrasil]

It only builds on linux with cmake currently and needs some more cleanup. So it might crash now and then.

Shame on you.

no comment...

What is this even needed for?!?

It's an attempt to implement the addon concept daniel was talking about. It's also a more flexible way to get separate directories for configs, normal data and mods. E.g. it's easy to add/remove a new faction to magitech without touching the normal data directory. This is important for linux users because they usually don't have write access to this directory. It makes mod management also easier for other platforms (e.g. replace files without physically overwriting files).

[Omega]

Sounds handy, but the source MUST always be linux AND windows. I don't mind a linux function that still compiles and works on windows (but the function only executes on linux). But the source must ALWAYS be OS independant.

[Yggdrasil]

I will not invest any time to ifdef all the physfs stuff because it's meant to also work on windows and it's in a separate branch. So it can break without interupting other people. I just had not find the time to adjust the build systems for windows and have a look if it builds. It's work in progress.

[silnarm]

Broken builds on windoze or linux are an unfortunate fact of life on this project

I would not be concerned, especially in a branch no one else is working in. If you get lucky, someone else may well come along and fix it all up for you anyway, laziness can pay-off

[wciow]

TBH Physfs should only be built into the linux builds anyway.

If I understand correctly (not being a coder), the point of Physfs is to make the linux file system of Glest easier to use. This is understandable because of the way that linux distrubutes its program files around the system.

However the windows build really doesn't need this since its file structure is very clear and easy by design. If a windows mod it set up correctly the zipped file can just be extracted directly to the correct folder techs/factions/tilesets etc.

Just my 2p 

[Omega]

Yes, but my question is if the executable would still work on windows with a bit of linux code on it? Ideally, it wouldn't use that linux code on windows so that it could be compatible. Has anyone tested?

[titi]

@wciow and omega:
thats all wrong, Physfs is needed to use zipped files easily. For example it would be possible to have a complete techtree in a zipfile and you can use it directly without unpacking! This zip-file is much smaller on the disk than the unpacked techtree. This is how other games handle the things too.

[Omega]

Really? That's what it does? That really sounds quite handy. However, only if it is OS-independant. Can it be?

[Yggdrasil]

Yes, of course. I fixed today all build systems, so it works now also on windows if one gets physfs itselfs and stores it in the deps folder.

btw: physfs 2.0 is required.

[wciow]

So mods can now be packaged into a single mod file like on the spring engine?

How does this work for tilesets and maps? What if I want to play two magitech compatible factions from different mod files?

Can someone explain this in newbie terms because I'm a little confused   ??? 

[Yggdrasil]

At the moment it just (virtually) merges the directories.

Assuming we have 2 mods, one zipped, the other just in it's folder:

Code: [Select]

~/.glestadv/addons/mod.zip
  |-- techs/magitech/factions/mybrandnewone/...
  |-- maps/dark_forest.gbm
  `-- maps/greatmap.gbm

~/.glestadv/addons/anothermod
  |-- maps/coolnewmap.gbm
  `-- tilesets/wonderland/...
And the normal data:

Code: [Select]

/usr/share/glestae
  |-- maps/dark_forest.gbm
  |-- tilesets/dark_forest/...
  |-- techs/magitech/factions/magic/...
  `-- techs/magitech/factions/tech/...

what we get in the end is:

Code: [Select]

techs/magitech/factions
  |-- mybrandnewone/...
  |-- magic/...
  `-- tech/...
maps
  |-- dark_forest.gbm (from mod.zip, virtually overwritten)
  |-- greatmap.gbm
  `-- coolnewmap.gbm
tilesets
  |-- dark_forest/...
  `-- wonderland/...

Overwriting between mods is a bit undefined yet. We would need dependency checking to get this. Not sure if modders need this.

[wciow]

Thanks Yggdrasil, I get it now 

[me5]

So you download a mod put it in "~/.glestadv" (hidden folder in your home directory) and then you can just play that mod without any permission issues? ->AWESOME