Messages

776

Feature requests / Re: integrated mod installer

« on: 18 November 2010, 16:42:34 »

yeah I was expecting to author the zip or 7z archive directly to turn a range request into a full archive.

my python script was not intended in any way to be the codebase that gets adopted, but rather I simply wanted to know what fraction of a mod altered for each iteration and how big the diffs might be; not find a quick way to determine that, I wrote that throw-away python script.

I feel uncomfortable with a tar+xdelta approach and would prefer an approach where the xml of a mod was parsed in the same way the game does, all dependencies located, and all the appropriate bits getting zipped up along with meta information.  It might be that the end result is an archive, but the way to get there is quite different and more involved.

The appeal of HTTP range requests is the simplicity of hosting.

So rather than a script that just thinks about directories, a tool that actually parses and packages the mod.  That's my general outlook.

777

Feature requests / Re: integrated mod installer

« on: 18 November 2010, 09:05:53 »

I added rename-checking and got these numbers for MRise 1.0 -> 1.6:

245 added:       39.8 MB
9   renamed:      1.6 MB
300 unchanged:   91.0 MB
11  changed:      7.6 MB
87  deleted:     17.1 MB

471 previously: 117.3 MB -> 60.2 MB zipped -> 50 MB 7z/rar
556 now:        148.7 MB -> 78.1 MB zipped -> 64 MB 7z/rar
    delta:               -> 27.0 MB zipped -> 19 MB 7z

My script, if anyone wants to run it on their mods (requires Python): https://gist.github.com/704809

9 of the files were found to be renamed.

With zip (my format of choice) the 1.0 mod archive was 60 MB compressed; it was originally distributed as a 7zip archive using RAR compression (and, I imagine, solid), and was only 50 MB.

The 1.6 mod archive compresses to 78 MB using zip, and was distributed as a 7zip archive with RAR too, taking just 64 MB.

Using the 'diffing', I calcuate you would only need to download 27 MB of of the 1.6 zip file in order to patch a copy of 1.0 that you already had; if we used 7zip (non-solid; actually, solid didn't improve compression dramatically) then this would be only around 19 MB.

So you can imagine two archives sitting on public web hosting, e.g. Google Sites.  There is also a 'diff' meta-data file.  The updater checks the meta-data file to see if its changed, using HTTP If-Modified-Since etc.  If its modified, it gets the meta-data and either fetches the full new archive, or uses a range request to download the delta (that has been conveniently packaged at the end of the new archive).  You can imagine a central site with scripts and things for the mod makers to be using to 'publish' the availability of updates etc, but I think it useful that the actual GBs of mods are stored elsewhere, and Google Sites seems to be a good choice.

Now 7z is supported in the latest GAE, but I feel that zip is an acceptable format and will be easier to work with.

There are zip archivers (e.g. 7z, info-zip but especially kzip) that do a better job of compressing zip archives than the normal 'zip' tools (but take much longer).

Working out how to diff mods was an interesting distraction but the real meat of the problem is now learning how the virtual FS works in GAE, and how to mod Glest and MG.

Keep feature requests and insights rolling in!

778

Feature requests / Re: integrated mod installer

« on: 17 November 2010, 23:56:14 »

Some stats about diffing two versions of a mod: Malevolent Rising 1.6 vs 1.0 (yes I know that's a big jump, but it was about the only two files I could find):

245 files added:     41,706,266 bytes
300 files unchanged: 95,469,241 bytes
11  files changed:    7,958,481 bytes
96  files deleted:   19,597,478 bytes

Is this fairly typical?  Also, is there ever much renaming of files without changing their contents?  (I did not check for this in the script I wrote to get those numbers)

It looks like I have a very simple format for the partial downloads of new versions in mind....

779

Feature requests / Re: integrated mod installer

« on: 17 November 2010, 21:56:58 »

Gathering advice on the mechanics of the packaging here: http://encode.ru/threads/1164-recommended-formats-for-game-data-and-partial-updates

My thought is that two tools are needed:

1) a packaging tool that generates the archive, some patch archives for moving cheaply from sequential version to version, and accurately recording the dependencies

2) the front-end in GAE itself, probably via the tech tree options and fraction choices in the new game screens; I can see that an external tool is more immediate to develop, but I favour a deeper integration as giving more customer satisfaction longer term

And this cascades into what to do if playing a multi-player game and ensuring all players have the same (e.g. most recent) version...

780

Tools / Re: Code::Blocks

« on: 17 November 2010, 21:18:31 »

I am not a CB user, but you point out there is a wchar file there; you should ensure that your sourcecode directories are 'user includes' and not 'system includes'?

781

Feature requests / Re: integrated mod installer

« on: 17 November 2010, 20:50:39 »

I have found this thread on the mailing list:

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikA%3Ddn8ESu_LyBREJMV1rObnOnp1oiv8txTU1yt%40mail.gmail.com&forum_name=glestae-devel

Is this already going somewhere?

782

General discussion / making font sizes adjustable

« on: 17 November 2010, 18:58:22 »

a tiny contribution, hopefully the first of many:

I admit I couldn't locate where RenderFontMenu and kin in the config are used; it looks like they are unused, right?

So anyway, I added a new config setting, RenderFontScaler, which is a multiplier to make the size of all fonts bigger or smaller globally; e.g. if you specify "RenderFontScaler 0.5" in the glestadv.ini, then your fonts are all half as tall as the default size (and I find this works much better in the New Game screen and so on).

What is the appropriate route and place for these tweaks?

https://github.com/williame/Glest/commit/bfd4bfb0456c9fc42b4738725b7f8723a7b362a9

783

Feature requests / integrated mod installer

« on: 17 November 2010, 06:43:57 »

https://forum.megaglest.org/index.php?topic=6002.msg62656#msg62656

Apologies if this has previously been discussed to death (although I couldn't find a thread on it in this forum), but I'm volunteering to try and implement it.  I have plenty of experience programming this downloading kind of thing, but it will be a fun and gentle introduction to coding glest...

The first step is get a good idea of the requirements so please reply with your assumptions and desires.

Oh, and explain how we manage mods that add fractions vs mods that use the engine for alternative games...