Big Endian support - let's do it

[eddyp]

Hello,

Together with Giuseppe Borzi I have made a Debian package for Glest (version 2.0.0) and realised that there is no big endian support. As I own a big endianmachine I would like to be able to play the game on it.

AFAIK there have been some patches for the endian issue, but no steady development for it.

I want to make this work and I would like to know if there are people which would want to help at this work.


I have read that some version in the past was endianess agnostic (0.9 ?)  but 2.0.0 is definetly not.

[eddyp]

AFAIK there have been some patches for the endian issue, but no steady development for it.

I want to make this work and I would like to know if there are people which would want to help at this work.


I have read that some version in the past was endianess agnostic (0.9 ?)  but 2.0.0 is definetly not.

I have started the work on this. I am working my way into the game, starting from the point where the game starts.

What I have done already is to implement normalize*Header for PixmapIO* classes.


here is a snippet:

Code: [Select]

+void PixmapIoBmp::normalizeFileHeader(struct BitmapFileHeader* header){
+
+#if WORDS_BIGENDIAN=1
+
+       swap32(header->size);
+       swap16(header->reserved1);
+       swap16(header->reserved2);
+       swap32(header->offsetBits);
+
+#endif // WORDS_BIGENDIAN=1
+
+}


what changes are things like fread and fwrite:

Code: [Select]

--- old-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp       2006-09-21 12:38:59.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp       2006-09-21 12:39:01.000000000 +0300
@@ -97,7 +97,8 @@
        //read header
        TargaFileHeader fileHeader;
        fread(&fileHeader, sizeof(TargaFileHeader), 1, file);
-
+       normalizeFileHeader(&fileHeader);
+
        //check that we can load this tga file
        if(fileHeader.idLength!=0){
                throw runtime_error(path + ": id field is not 0");
@@ -182,6 +183,7 @@
        fileHeader.height= h;
        fileHeader.imageDescriptor= components==4? 8: 0;

+       normalizeFileHeader(&fileHeader);
        fwrite(&fileHeader, sizeof(TargaFileHeader), 1, file);
 }


I will keep you posted.

If anybody has any suggestions, hints, objections... please say so.

[eddyp]

I will keep you posted.

If anybody has any suggestions, hints, objections... please say so.

I have worked on saturday on the patch. I can see the menu now and I can get to the point that things start to load, but at some point I get a segfault while trying to load a g3d file.

I am not sure, but at that point the game member should be initialized.

I will test the same code on a i386 machine, when I'll have the time and opportunity.


This is the current form of the patch, but is still incomplete and still ugly: http://http://pkg-games.alioth.debian.org/devel/glest-be-patch/glest-cvs20070106-incomplete-be-enabling.patch. I will make it more modular by moving all normalize function in the agnosticity.* files. Currently the patch should be safe for LE machines since every code modification is enabled by WORDS_BIGENDIAN==1


2006-Jan-06 : Updated link to make it easier to merge changes back; the Glest CVS hasn't changed one bit since my initial check-out, so I took the liberty to use a current date as a CVS timestamp.

[eddyp]

I will keep you posted.

If anybody has any suggestions, hints, objections... please say so.

I have worked on saturday on the patch. I can see the menu now and I can get to the point that things start to load, but at some point I get a segfault while trying to load a g3d file.

I am not sure, but at that point the game member should be initialized.

I will test the same code on a i386 machine, when I'll have the time and opportunity.


This is the current form of the patch, but is still incomplete and still ugly: http://http://pkg-games.alioth.debian.org/devel/glest-be-patch/glest-cvs20070106-incomplete-be-enabling.patch. I will make it more modular by moving all normalize function in the agnosticity.* files. Currently the patch should be safe for LE machines since every code modification is enabled by WORDS_BIGENDIAN==1


2006-Jan-06 : Updated link to make it easier to merge changes back; the Glest CVS hasn't changed one bit since my initial check-out, so I took the liberty to use a current date as a CVS timestamp.

Update: I will no longer have access to the ppc machine I was using. Thus, I will not be able to work on big endian support.

I didn't managed to do the tests on a i386 machine, but probably will be able to do that on my new laptop, when I'll buy it. you never know! I might discover the game doesn't work well on amd64 machine in 64 bit mode, so I might do some changes in that direction.

The complete status is at http://pkg-games.alioth.debian.org/deve ... TATUS-TODO and will be posted here too.

I will also post the patch here, too, for more redundancy.

[eddyp]

This file is about adding big endian support to the Glest game.
Upstream homepage for the game is http://www.glest.org/
CVS repository (anonymous access):
   cvs -d:pserver:anonymous@glest.cvs.sourceforge.net:/cvsroot/glest login
   cvs -z3 -d:pserver:anonymous@glest.cvs.sourceforge.net:/cvsroot/glest co -P glest
Forum: http://www.glest.org/board_456/




These notes were written on the 6th of January 2007, but the tests and coding
were done in september 2006, so some of the libabries might have changed,
although is highly unlikely since Debian was preparing for the freeze of Etch
at that time.



Current status (tests on a PowerBook5,2 - PowerBook G4 15' - Debian Sid system):
   - http://www.glest.org/board_456/viewtopic.php?t=1426
   - the application starts to the point it should start the game
      - menu mainly works
      - there were many segfaults that lead to bad data structures
        being read; some swap funtions for 16 and 32 bits were created and
        were added near file reads
      - still the music doesn't sound right (in spite of that, I
        don't think is the game, but the ogg/vorbis library since
        another app, rezound which uses the same pair played the music in
        the same broken way) - library versions are now libogg0 1.1.3-2
        and libvorbis0a 1.1.2.dfsg-1.2; mplayer plays the music properly
   - the music issue needs to be invesigated on BE machines to see if its
     origin is in the game or the libs.
   - the game doesn't start and reaches some kind of endless loop or some
     long calculus from where it never gets out - I didn't had the
     time/patience to see what happens
   - the Debian Games team repo does NOT contain the patch attached (if you
     didn't got this patch, it should be accessible:

- http://pkg-games.alioth.debian.org/deve ... ling.patch
- on the glest forum - http://www.glest.org/board_456/viewtopic.php?t=1426
- via the Debian Games Team <pkg-games-devel @ alioth.debian.org>
- from me: Eddy Petrișor <eddy.petrisor @ gmail.com> ; contact me by email if
  you want it and you can't find it.

   - the upstream CVS repo seems to be mostly inactive, which is kind of sad
     and is one reason/pretext for my lack of activity on the patch
   - I am not sure if the patch would apply cleanly aginst the released 2.0.0
     version instead of the CVS version
   - [2006-jan-06] the upstream forum seems to have went through a hardware
     change and should be active; not sure how this relates to actual
     development

Unfortunately I will no longer have access to the PowerBook I used for
debugging, thus I will not be able to continue this work. Anyone willing to
make the thing work is welcome.


 6 ian 2006 - Eddy Petrișor <eddy.petrisor @ gmail.com>

[eddyp]

(25th of April 2007 : update - make it harder for mail harvesters to find my email)

Changelog is:

Mon Sep 25 00:50:26 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * fix the way variables are referenced so it works; implement normalizeMeshHeader* functions and place them in the proper place; isolate normalizeMapFileHeader in a distinct function

Mon Sep 25 00:43:28 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * every call to swap functions should be conditioned by the fact we are running on a BE machine

Mon Sep 25 00:41:02 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * fix shifting amounts (these are not bytes, are bits!)

Mon Sep 25 00:40:30 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * fix OR expressions - OH MY GOD, too much scripting will inject || in your head

Sun Sep 24 03:12:45 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * try to fix endian issues for reads and writes in the rest of the game

Sun Sep 24 03:11:20 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * add the implementation of the swap functions which are needed in order to achieve endianness independence

Sat Sep 23 21:15:35 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * test program to see how LE and BE machines interpret data from streams

Fri Sep 22 02:31:36 EEST 2006  Eddy Petrișor <eddy.petrisor @ gmail.com>
  * added code to swap bytes (if needed) in image headers, when loaded from a file
  - only the shared_lib code was touched
  - not sure if data placed in the file is correctly read

Code: [Select]

--- old-bigendian/glest-cvs/source/glest_game/menu/main_menu.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/glest_game/menu/main_menu.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -23,6 +23,8 @@
 #include "faction.h"
 #include "metrics.h"
 #include "leak_dumper.h"
+#include "maps.h"
+#include "agnosticity.h"
 
 using namespace Shared::Sound;
 using namespace Shared::Util;
@@ -492,16 +494,6 @@
 
 void MenuStateGameSettings::loadMapInfo(string file, MapInfo *mapInfo){
 
- struct MapFileHeader{
- int32 version;
- int32 maxPlayers;
- int32 width;
- int32 height;
- int32 altFactor;
- int32 waterLevel;
- int8 title[128];
- };
-
  Lang &lang= Lang::getInstance();
 
  try{
@@ -511,6 +503,7 @@
 
  MapFileHeader header;
  fread(&header, sizeof(MapFileHeader), 1, f);
+ normalizeMapFileHeader(&header);
 
  mapInfo->size.x= header.width;
  mapInfo->size.y= header.height;
diff -rN -u old-bigendian/glest-cvs/source/glest_game/world/map.cpp new-bigendian/glest-cvs/source/glest_game/world/map.cpp
--- old-bigendian/glest-cvs/source/glest_game/world/map.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/glest_game/world/map.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -20,6 +20,8 @@
 #include "tech_tree.h"
 #include "config.h"
 #include "leak_dumper.h"
+#include "agnosticity.h"
+#include "maps.h"
 
 using namespace Shared::Graphics;
 using namespace Shared::Util;
@@ -102,18 +104,6 @@
 }
 
 void Map::load(const string &path, TechTree *techTree, Tileset *tileset){
-
- struct MapFileHeader{
- int32 version;
- int32 maxPlayers;
- int32 width;
- int32 height;
- int32 altFactor;
- int32 waterLevel;
- int8 title[128];
- int8 author[128];
- int8 description[256];
- };
 
  try{
  FILE *f= fopen(path.c_str(), "rb");
@@ -122,6 +112,7 @@
  //read header
  MapFileHeader header;
  fread(&header, sizeof(MapFileHeader), 1, f);
+ normalizeMapFileHeader(&header);
 
  if(next2Power(header.width) != header.width){
  throw runtime_error("Map width is not a power of 2");
@@ -146,7 +137,13 @@
  for(int i=0; i<maxPlayers; ++i){
  int x, y;
  fread(&x, sizeof(int32), 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32(&x);
+ #endif // WORDS_BIGENDIAN==1
  fread(&y, sizeof(int32), 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32(&y);
+ #endif // WORDS_BIGENDIAN==1
  startLocations[i]= Vec2i(x, y)*cellScale;
  }
 
diff -rN -u old-bigendian/glest-cvs/source/glest_game/world/map.h new-bigendian/glest-cvs/source/glest_game/world/map.h
--- old-bigendian/glest-cvs/source/glest_game/world/map.h 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/glest_game/world/map.h 2006-09-25 16:18:00.000000000 +0300
@@ -18,6 +18,7 @@
 #include "logger.h"
 #include "object.h"
 #include "game_constants.h"
+#include "maps.h"
 
 #include <cassert>
 
@@ -229,6 +230,7 @@
  void smoothSurface();
  void computeNearSubmerged();
  void computeCellColors();
+
 };
 
 
diff -rN -u old-bigendian/glest-cvs/source/glest_map_editor/map.cpp new-bigendian/glest-cvs/source/glest_map_editor/map.cpp
--- old-bigendian/glest-cvs/source/glest_map_editor/map.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/glest_map_editor/map.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -1,4 +1,5 @@
 #include "map.h"
+#include "agnosticity.h"
 
 #include <cmath>
 #include <stdexcept>
@@ -404,6 +405,7 @@
  //read header
  MapFileHeader header;
  fread(&header, sizeof(MapFileHeader), 1, f1);
+ normalizeMapFileHeader(&header);
 
  altFactor= header.altFactor;
  waterLevel= header.waterLevel;
@@ -415,7 +417,13 @@
  resetPlayers(header.maxPlayers);
  for(int i=0; i<maxPlayers; ++i){
  fread(&startLocations[i].x, sizeof(int32), 1, f1);
+ #if WORDS_BIGENDIAN==1
+ swap32(&(startLocations[i].x));
+ #endif // WORDS_BIGENDIAN==1
  fread(&startLocations[i].y, sizeof(int32), 1, f1);
+ #if WORDS_BIGENDIAN==1
+ swap32(&(startLocations[i].y));
+ #endif // WORDS_BIGENDIAN==1
  }
 
  //read Heights
@@ -473,11 +481,18 @@
  strncpy(header.author, author.c_str(), 128);
  strncpy(header.description, desc.c_str(), 256);
 
+ normalizeMapFileHeader(&header);
  fwrite(&header, sizeof(MapFileHeader), 1, f1);
 
  //write start locations
  for(int i=0; i<maxPlayers; ++i){
+ #if WORDS_BIGENDIAN==1
+ swap32(&(startLocations[i].x));
+ #endif // WORDS_BIGENDIAN==1
  fwrite(&startLocations[i].x, sizeof(int32), 1, f1);
+ #if WORDS_BIGENDIAN==1
+ swap32(&(startLocations[i].y));
+ #endif // WORDS_BIGENDIAN==1
  fwrite(&startLocations[i].y, sizeof(int32), 1, f1);
  }
 
diff -rN -u old-bigendian/glest-cvs/source/glest_map_editor/map.h new-bigendian/glest-cvs/source/glest_map_editor/map.h
--- old-bigendian/glest-cvs/source/glest_map_editor/map.h 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/glest_map_editor/map.h 2006-09-25 16:18:00.000000000 +0300
@@ -3,23 +3,12 @@
 
 #include "util.h"
 #include "types.h"
+#include "maps.h"
 
 using Shared::Platform::int8;
 using Shared::Platform::int32;
 using Shared::Platform::float32;
 
-struct MapFileHeader{
- int32 version;
- int32 maxPlayers;
- int32 width;
- int32 height;
- int32 altFactor;
- int32 waterLevel;
- int8 title[128];
- int8 author[128];
- int8 description[256];
-};
-
 namespace Glest{ namespace MapEditor{
 
 // ===============================================
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/include/graphics/maps.h new-bigendian/glest-cvs/source/shared_lib/include/graphics/maps.h
--- old-bigendian/glest-cvs/source/shared_lib/include/graphics/maps.h 1970-01-01 02:00:00.000000000 +0200
+++ new-bigendian/glest-cvs/source/shared_lib/include/graphics/maps.h 2006-09-25 16:18:00.000000000 +0300
@@ -0,0 +1,26 @@
+#ifndef _SHARED_LIB_INCLUDE_MAP_H_
+#define _SHARED_LIB_INCLUDE_MAP_H_
+
+#include "types.h"
+
+using Shared::Platform::int32;
+using Shared::Platform::int8;
+
+typedef struct {
+ int32 version;
+ int32 maxPlayers;
+ int32 width;
+ int32 height;
+ int32 altFactor;
+ int32 waterLevel;
+ int8 title[128];
+ int8 author[128];
+ int8 description[256];
+} MapFileHeader;
+
+namespace Shared{ namespace Util{
+
+void normalizeMapFileHeader(MapFileHeader *header);
+
+}}//end namespace
+#endif // _SHARED_LIB_INCLUDE_MAP_H_
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/include/graphics/model.h new-bigendian/glest-cvs/source/shared_lib/include/graphics/model.h
--- old-bigendian/glest-cvs/source/shared_lib/include/graphics/model.h 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/include/graphics/model.h 2006-09-25 16:18:00.000000000 +0300
@@ -116,6 +116,10 @@
  void load(const string &dir, FILE *f, TextureManager *textureManager);
  void save(const string &dir, FILE *f);
 
+ //endianness
+ void normalizeMeshHeader(MeshHeader *header);
+ void normalizeMeshHeaderV3(MeshHeaderV3 *header);
+ void normalizeMeshHeaderV2(MeshHeaderV2 *header);
 private:
  void computeTangents();
 };
@@ -165,6 +169,8 @@
 
 private:
  void buildInterpolationData() const;
+
+ void normalizeModelHeader(struct ModelHeader*header);
 };
 
 }}//end namespace
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/include/graphics/pixmap.h new-bigendian/glest-cvs/source/shared_lib/include/graphics/pixmap.h
--- old-bigendian/glest-cvs/source/shared_lib/include/graphics/pixmap.h 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/include/graphics/pixmap.h 2006-09-25 16:18:00.000000000 +0300
@@ -71,6 +71,8 @@
 
  virtual void openWrite(const string &path, int w, int h, int components);
  virtual void write(uint8 *pixels);
+
+ virtual void normalizeFileHeader(struct TargaFileHeader *header);
 };
 
 // =====================================================
@@ -91,6 +93,9 @@
 
  virtual void openWrite(const string &path, int w, int h, int components);
  virtual void write(uint8 *pixels);
+
+ virtual void normalizeFileHeader(struct BitmapFileHeader *header);
+ virtual void normalizeInfoHeader(struct BitmapInfoHeader *header);
 };
 
 // =====================================================
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/include/util/agnosticity.h new-bigendian/glest-cvs/source/shared_lib/include/util/agnosticity.h
--- old-bigendian/glest-cvs/source/shared_lib/include/util/agnosticity.h 1970-01-01 02:00:00.000000000 +0200
+++ new-bigendian/glest-cvs/source/shared_lib/include/util/agnosticity.h 2006-09-25 16:18:00.000000000 +0300
@@ -0,0 +1,37 @@
+// ==============================================================
+// This file is part of Glest Shared Library (www.glest.org)
+//
+// Copyright (C) 2006 Eddy Petris,or
+//
+// You can redistribute this code and/or modify it under
+// the terms of the GNU General Public License as published
+// by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version
+// ==============================================================
+
+#ifndef _SHARED_UTIL_AGNOSTICITY_H_
+#define _SHARED_UTIL_AGNOSTICITY_H_
+
+#include "types.h"
+#include "util.h"
+
+using Shared::Platform::uint8;
+using Shared::Platform::uint16;
+using Shared::Platform::int16;
+using Shared::Platform::uint32;
+using Shared::Platform::int32;
+
+namespace Shared{ namespace Util{
+
+//endian agnosticity
+void swap(uint8 size, void *data);
+void swap16(uint16 *data);
+void swap16(int16 *data);
+void swap32(uint32 *data);
+void swap32(int32 *data);
+void swap32x(uint32 *data, uint8 count);
+void swap32x(int32 *data, uint8 count);
+
+}}//end namespace
+
+#endif // _SHARED_UTIL_AGNOSTICITY_H_
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/graphics/maps.cpp new-bigendian/glest-cvs/source/shared_lib/sources/graphics/maps.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/graphics/maps.cpp 1970-01-01 02:00:00.000000000 +0200
+++ new-bigendian/glest-cvs/source/shared_lib/sources/graphics/maps.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -0,0 +1,36 @@
+// ==============================================================
+//      This file is part of Glest Shared Library (www.glest.org)
+//
+//      Copyright (C) 2006 Eddy Petris,or
+//
+//      You can redistribute this code and/or modify it under
+//      the terms of the GNU General Public License as published
+//      by the Free Software Foundation; either version 2 of the
+//      License, or (at your option) any later version
+// ==============================================================
+
+#include "maps.h"
+#include "agnosticity.h"
+
+using Shared::Util::swap32;
+
+namespace Shared{ namespace Util{
+
+void normalizeMapFileHeader(MapFileHeader *header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->version));
+ swap32(&(header->maxPlayers));
+ swap32(&(header->width));
+ swap32(&(header->height));
+ swap32(&(header->altFactor));
+ swap32(&(header->waterLevel));
+
+#endif // WORDS_BIGENDIAN=1
+}
+
+
+
+}}//end namespace
+
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/graphics/model.cpp new-bigendian/glest-cvs/source/shared_lib/sources/graphics/model.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/graphics/model.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/sources/graphics/model.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -19,9 +19,11 @@
 #include "conversion.h"
 #include "util.h"
 #include "leak_dumper.h"
+#include "agnosticity.h"
 
-using namespace Shared::Platform;
 
+using namespace Shared::Platform;
+using namespace Shared::Util;
 using namespace std;
 
 namespace Shared{ namespace Graphics{
@@ -95,6 +97,7 @@
  //read header
  MeshHeaderV2 meshHeader;
  fread(&meshHeader, sizeof(MeshHeaderV2), 1, f);
+ normalizeMeshHeaderV2(&meshHeader);
 
 
  if(meshHeader.normalFrameCount!=meshHeader.vertexFrameCount){
@@ -138,13 +141,16 @@
  fread(&opacity, sizeof(float32), 1, f);
  fseek(f, sizeof(Vec4f)*(meshHeader.colorFrameCount-1), SEEK_CUR);
  fread(indices, sizeof(uint32)*indexCount, 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32x(indices, indexCount);
+ #endif // WORDS_BIGENDIAN==1
 }
 
 void Mesh::loadV3(const string &dir, FILE *f, TextureManager *textureManager){
  //read header
  MeshHeaderV3 meshHeader;
  fread(&meshHeader, sizeof(MeshHeaderV3), 1, f);
-
+ normalizeMeshHeaderV3(&meshHeader);
 
  if(meshHeader.normalFrameCount!=meshHeader.vertexFrameCount){
  throw runtime_error("Old model: vertex frame count different from normal frame count");
@@ -185,12 +191,16 @@
  fread(&opacity, sizeof(float32), 1, f);
  fseek(f, sizeof(Vec4f)*(meshHeader.colorFrameCount-1), SEEK_CUR);
  fread(indices, sizeof(uint32)*indexCount, 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32x(indices, indexCount);
+ #endif // WORDS_BIGENDIAN==1
 }
 
 void Mesh::load(const string &dir, FILE *f, TextureManager *textureManager){
  //read header
  MeshHeader meshHeader;
  fread(&meshHeader, sizeof(MeshHeader), 1, f);
+ normalizeMeshHeader(&meshHeader);
 
  //init
  frameCount= meshHeader.frameCount;
@@ -238,6 +248,9 @@
  fread(texCoords, sizeof(Vec2f)*vertexCount, 1, f);
  }
  fread(indices, sizeof(uint32)*indexCount, 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32x(indices, indexCount);
+ #endif // WORDS_BIGENDIAN==1
 
  //tangents
  if(textures[mtNormal]!=NULL){
@@ -267,11 +280,15 @@
  texture->getPixmap()->saveTga(dir+"/"+texName);
  }
 
+ normalizeMeshHeader(&meshHeader);
  fwrite(&meshHeader, sizeof(MeshHeader), 1, f);
  fwrite(vertices, sizeof(Vec3f)*vertexFrameCount*pointCount, 1, f);
  fwrite(normals, sizeof(Vec3f)*normalFrameCount*pointCount, 1, f);
  fwrite(texCoords, sizeof(Vec2f)*texCoordFrameCount*pointCount, 1, f);
  fwrite(colors, sizeof(Vec4f)*colorFrameCount, 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap32x(indices, indexCount);
+ #endif // WORDS_BIGENDIAN==1
  fwrite(indices, sizeof(uint32)*indexCount, 1, f);*/
 }
 
@@ -313,6 +330,58 @@
  }
 }
 
+void Mesh::normalizeMeshHeader(MeshHeader *header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->frameCount));
+ swap32(&(header->vertexCount));
+ swap32(&(header->indexCount));
+/*
+ float32 diffuseColor[3];
+ float32 specularColor[3];
+ float32 specularPower;
+ float32 opacity;
+*/
+ swap32(&(header->properties));
+ swap32(&(header->textures));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
+void Mesh::normalizeMeshHeaderV3(MeshHeaderV3 *header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->vertexFrameCount));
+ swap32(&(header->normalFrameCount));
+ swap32(&(header->texCoordFrameCount));
+ swap32(&(header->colorFrameCount));
+ swap32(&(header->pointCount));
+ swap32(&(header->indexCount));
+ swap32(&(header->properties));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
+
+void Mesh::normalizeMeshHeaderV2(MeshHeaderV2 *header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->vertexFrameCount));
+ swap32(&(header->normalFrameCount));
+ swap32(&(header->texCoordFrameCount));
+ swap32(&(header->colorFrameCount));
+ swap32(&(header->pointCount));
+ swap32(&(header->indexCount));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
 // ===============================================
 // class Model
 // ===============================================
@@ -452,6 +521,8 @@
  //model header
  ModelHeader modelHeader;
  fread(&modelHeader, sizeof(ModelHeader), 1, f);
+ normalizeModelHeader(&modelHeader);
+
  meshCount= modelHeader.meshCount;
  if(modelHeader.type!=mtMorphMesh){
  throw runtime_error("Invalid model type");
@@ -468,6 +539,9 @@
  else if(fileHeader.version==3){
 
  fread(&meshCount, sizeof(meshCount), 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap(sizeof(meshCount), (void *)&meshCount);
+ #endif // WORDS_BIGENDIAN==1
  meshes= new Mesh[meshCount];
  for(uint32 i=0; i<meshCount; ++i){
  meshes[i].loadV3(dir, f, textureManager);
@@ -478,6 +552,9 @@
  else if(fileHeader.version==2){
 
  fread(&meshCount, sizeof(meshCount), 1, f);
+ #if WORDS_BIGENDIAN==1
+ swap(sizeof(meshCount), (void *)&meshCount);
+ #endif // WORDS_BIGENDIAN==1
  meshes= new Mesh[meshCount];
  for(uint32 i=0; i<meshCount; ++i){
  meshes[i].loadV2(dir, f, textureManager);
@@ -512,6 +589,7 @@
 
  string dir= cutLastFile(path);
 
+ normalizeModelHeader(&modelHeader);
  fwrite(&modelHeader, sizeof(ModelHeader), 1, f);
  for(int i=0; i<meshCount; ++i){
  meshes[i].save(dir, f);
@@ -520,4 +598,14 @@
  fclose(f);*/
 }
 
+void Model::normalizeModelHeader(struct ModelHeader*header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap16(&(header->meshCount));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
 }}//end namespace
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp new-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/sources/graphics/pixmap.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -18,6 +18,7 @@
 #include "util.h"
 #include "math_util.h"
 #include "leak_dumper.h"
+#include "agnosticity.h"
 
 using namespace std;
 
@@ -97,7 +98,8 @@
  //read header
  TargaFileHeader fileHeader;
  fread(&fileHeader, sizeof(TargaFileHeader), 1, file);
-    
+ normalizeFileHeader(&fileHeader);
+
  //check that we can load this tga file
  if(fileHeader.idLength!=0){
  throw runtime_error(path + ": id field is not 0");
@@ -182,6 +184,7 @@
  fileHeader.height= h;
  fileHeader.imageDescriptor= components==4? 8: 0;
 
+ normalizeFileHeader(&fileHeader);
  fwrite(&fileHeader, sizeof(TargaFileHeader), 1, file);
 }
 
@@ -201,6 +204,21 @@
  }
 }
 
+void PixmapIoTga::normalizeFileHeader(struct TargaFileHeader *header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap16(&(header->colourMapOrigin));
+ swap16(&(header->colourMapLength));
+ swap16(&(header->xOrigin));
+ swap16(&(header->yOrigin));
+ swap16(&(header->width));
+ swap16(&(header->height));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
 // =====================================================
 // class PixmapIoBmp
 // =====================================================
@@ -224,6 +242,7 @@
  //read file header
     BitmapFileHeader fileHeader;
     fread(&fileHeader, sizeof(BitmapFileHeader), 1, file);
+ normalizeFileHeader(&fileHeader);
  if(fileHeader.type1!='B' || fileHeader.type2!='M'){
  throw runtime_error(path +" is not a bitmap");
  }
@@ -231,6 +250,7 @@
  //read info header
  BitmapInfoHeader infoHeader;
  fread(&infoHeader, sizeof(BitmapInfoHeader), 1, file);
+ normalizeInfoHeader(&infoHeader);
  if(infoHeader.bitCount!=24){
         throw runtime_error(path+" is not a 24 bit bitmap");
  }
@@ -281,13 +301,14 @@
  }
 
  BitmapFileHeader fileHeader;
-    fileHeader.type1='B';
+ fileHeader.type1='B';
  fileHeader.type2='M';
  fileHeader.offsetBits=sizeof(BitmapFileHeader)+sizeof(BitmapInfoHeader);
  fileHeader.size=sizeof(BitmapFileHeader)+sizeof(BitmapInfoHeader)+3*h*w;
 
-    fwrite(&fileHeader, sizeof(BitmapFileHeader), 1, file);
-    
+ normalizeFileHeader(&fileHeader);
+ fwrite(&fileHeader, sizeof(BitmapFileHeader), 1, file);
+
  //info header
  BitmapInfoHeader infoHeader;
  infoHeader.bitCount=24;
@@ -302,6 +323,7 @@
  infoHeader.xPelsPerMeter= 0;
  infoHeader.yPelsPerMeter= 0;
 
+ normalizeInfoHeader(&infoHeader);
  fwrite(&infoHeader, sizeof(BitmapInfoHeader), 1, file);
 }
 
@@ -313,6 +335,39 @@
     }
 }
 
+void PixmapIoBmp::normalizeFileHeader(struct BitmapFileHeader* header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->size));
+ swap16(&(header->reserved1));
+ swap16(&(header->reserved2));
+ swap32(&(header->offsetBits));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
+void PixmapIoBmp::normalizeInfoHeader(struct BitmapInfoHeader* header){
+
+#if WORDS_BIGENDIAN==1
+
+ swap32(&(header->size));
+ swap32(&(header->width));
+ swap32(&(header->height));
+ swap16(&(header->planes));
+ swap16(&(header->bitCount));
+ swap32(&(header->compression));
+ swap32(&(header->sizeImage));
+ swap32(&(header->xPelsPerMeter));
+ swap32(&(header->yPelsPerMeter));
+ swap32(&(header->clrUsed));
+ swap32(&(header->clrImportant));
+
+#endif // WORDS_BIGENDIAN==1
+
+}
+
 // =====================================================
 // class Pixmap1D
 // =====================================================
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/sound/sound_file_loader.cpp new-bigendian/glest-cvs/source/shared_lib/sources/sound/sound_file_loader.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/sound/sound_file_loader.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/sources/sound/sound_file_loader.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -16,9 +16,12 @@
 
 #include "sound.h"
 #include "leak_dumper.h"
+#include "agnosticity.h"
 
-using namespace Shared::Platform;
-using namespace std;
+using Shared::Platform::uint32;
+using Shared::Platform::uint16;
+using Shared::Util;
+using std;
 
 namespace Shared{ namespace Sound{
 
@@ -72,10 +75,16 @@
 
     //first sub-chunk (header) - Data (nChannels)
     f.read((char*) &size16, 2);
+ #if WORDS_BIGENDIAN==1
+ swap16(&size16);
+ #endif // WORDS_BIGENDIAN==1
  soundInfo->setChannels(size16);
 
     //first sub-chunk (header) - Data (nsamplesPerSecond)
     f.read((char*) &size32, 4);
+ #if WORDS_BIGENDIAN==1
+ swap32(&size32);
+ #endif // WORDS_BIGENDIAN==1
  soundInfo->setsamplesPerSecond(size32);
 
     //first sub-chunk (header) - Data (nAvgBytesPerSec)  - Ignore
@@ -86,6 +95,9 @@
 
     //first sub-chunk (header) - Data (nsamplesPerSecond)
     f.read((char*) &size16, 2);
+ #if WORDS_BIGENDIAN==1
+ swap16(&size16);
+ #endif // WORDS_BIGENDIAN==1
  soundInfo->setBitsPerSample(size16);
 
  if (soundInfo->getBitsPerSample() != 8 && soundInfo->getBitsPerSample()!=16){
@@ -106,6 +118,9 @@
 
         //second sub-chunk (samples) - Size
         f.read((char*) &size32, 4);
+ #if WORDS_BIGENDIAN==1
+ swap32(&size32);
+ #endif // WORDS_BIGENDIAN==1
  dataSize= size32;
  soundInfo->setSize(dataSize);
     }
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/util/agnosticity.cpp new-bigendian/glest-cvs/source/shared_lib/sources/util/agnosticity.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/util/agnosticity.cpp 1970-01-01 02:00:00.000000000 +0200
+++ new-bigendian/glest-cvs/source/shared_lib/sources/util/agnosticity.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -0,0 +1,57 @@
+// ==============================================================
+// This file is part of Glest Shared Library (www.glest.org)
+//
+// Copyright (C) 2006 Eddy Petris,or
+//
+// You can redistribute this code and/or modify it under
+// the terms of the GNU General Public License as published
+// by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version
+// ==============================================================
+
+#include "agnosticity.h"
+
+namespace Shared{ namespace Util{
+
+void swap16(uint16 *data){
+ *data = (( (*data) & 0x00ffu) << 8) | ( (*data) >> 8 ) ;
+}
+void swap16(int16 *data){
+ *data = (( (*data) & 0x00ffu) << 8) | ( (*data) >> 8 ) ;
+}
+void swap32(uint32 *data){
+ uint32 tmp;
+ tmp = *data;
+ *data = ( ( (tmp & (uint32)0x000000ff) << 24 ) |
+  ( (tmp & (uint32)0x0000ff00) << 8 ) |
+  ( (tmp & (uint32)0x00ff0000) >> 8 ) |
+  ( (tmp & (uint32)0xff000000) >> 24 ) ) ;
+}
+void swap32(int32 *data){
+ int32 tmp;
+ tmp = *data;
+ *data = ( ( (tmp & (int32)0x000000ff) << 24 ) |
+  ( (tmp & (int32)0x0000ff00) << 8 ) |
+  ( (tmp & (int32)0x00ff0000) >> 8 ) |
+  ( (tmp & (int32)0xff000000) >> 24 ) ) ;
+}
+void swap32x(uint32 *data, uint8 count){
+ for ( uint8 i = 0 ; i < count ; i++ ) swap32(data+i);
+}
+void swap32x(int32 *data, uint8 count){
+ for ( uint8 i = 0 ; i < count ; i++ ) swap32(data+i);
+}
+void swap(uint8 size, void *data){
+ if (size == 2){
+ swap16((int16 *)data);
+ }
+ else if (size == 4){
+ swap32((int32 *)data);
+ }
+ else
+ {
+ // nothing to do
+ }
+}
+
+}}//end namespace
diff -rN -u old-bigendian/glest-cvs/source/shared_lib/sources/util/util.cpp new-bigendian/glest-cvs/source/shared_lib/sources/util/util.cpp
--- old-bigendian/glest-cvs/source/shared_lib/sources/util/util.cpp 2006-09-25 16:18:00.000000000 +0300
+++ new-bigendian/glest-cvs/source/shared_lib/sources/util/util.cpp 2006-09-25 16:18:00.000000000 +0300
@@ -159,4 +159,5 @@
  return min+(((max-min)/RAND_MAX)*rand());
 }
 
+
 }}//end namespace



[chascar]

It is a pity you can't continue working on it. I hope someone else takes over it, I'd love to play on my Power Book.

Thanks anyway for you contribution.

[eddyp]

It is a pity you can't continue working on it. I hope someone else takes over it, I'd love to play on my Power Book.

Thanks anyway for you contribution.

I hope they apply the patch on the CVS version since the patch shouldn't do any harm to the current code and is better for people to merge it as soon as possible to avoid later code to written in the same way as it was until now.

Martinho, could you apply this patch over the CVS HEAD?

[eddyp]

It is a pity you can't continue working on it. I hope someone else takes over it, I'd love to play on my Power Book.

Thanks anyway for you contribution.

I hope they apply the patch on the CVS version since the patch shouldn't do any harm to the current code and is better for people to merge it as soon as possible to avoid later code to written in the same way as it was until now.

Martinho, could you apply this patch over the CVS HEAD?

Is any of the people with CVS access interested in this patch?
Altough incomplete, it was written in such a way that it shouldn't interfere in any way with the code on little endian machines, so it should be safe to commit.

If any issues occur, I can be contacted by mail and I can assist.
I am also available to anyone desiring to continue my work, also by mail.

my address is   < eddy . petrisor  @  gmail . com >

PLEASE, PLEASE, commit this!

It would be a shame to loose this patch, even if it is incomplete, because I am sure that with about a week worth of work, this patch could be completed.

[eddyp]

Hello,

Going through the forum I found a BE patch for glest; I have merged this in the development version of the debian package and I got these comments from a fellow developer which had a PowerBook G4:

<Rhonda> eddyp: It doesn't segfault for me anymore  \o/
<Rhonda> Though I have to admit, it's quite a bit misty in the game ...
<Rhonda> I mean, everything grey.  
<Rhonda> Couldn't process event: Can't open TGA file: screens/screen0.tga
<Rhonda> Maybe that's the reason why it's all grey?
<eddyp> with my patch the sounds was all crackly
<eddyp> s/sounds/music/
<Rhonda> Couldn't process event: Error creating texture 3D
<Rhonda> Music works in the menu.
<Rhonda> In the game I hear nothing.
<Rhonda> And the view-triangle rotates only when I press m and then a (counter-clockwise) or d (clockwise)
<Rhonda> p writes to the bottom that game was paused, so I think in principle it doesn't look too bad.  PowerBook G4 this is.
<eddyp> okie, r300 driver, right?
<Rhonda> Yoda your master was?
<Rhonda> File r300_render.c function r300Fallback line 393
<eddyp> version of the driver/X
<eddyp> ?
<eddyp> just for documentation purposes
<Rhonda> Btw., the rotation of the camera doesn't stop when I release a or d, is that on intention?
<Rhonda> xserver-xorg-video-ati version 1:6.6.3-2
<Rhonda> Using the ati driver at the moment.  Never found any difference between using ati or radeon ...
<Rhonda> RV350 [Mobility Radeon 9600 M10] is my graphic's card

The patch was downloaded from http://http://kozz.org/files/games/glest/glest-cvs20070102-endian.patch.gz; explanations about the patch are at:    http://http://www.powerdeveloper.org/forums/viewtopic.php?p=6675


Note: I would have added a comment in the powerdeveloper forum, but I get an error when I try to register.

[Auria]

Hey thanks your work, i would really hope it comes true! i hope they'll apply it

[eddyp]

I tested the patch on a little endian machine to test the inrusion level of the patch (or better said, if it breaks anything); currently I think is ok, although it seems that there are some crackling sounds in the audio; I am not 100% if these are the patch's fault since I didn't had time to test without the patch.

[afxgroup]

Hello guys...
i'm interesting also in this patch since i've compiled the last glest on AmigaOS4 PPC.. well, it starts but very soon i receive:

Bits per sample must be 8 or 16:

this seems related to the endianess of the machine..

Am I wrong?

Andrea

[eddyp]

Hello guys...
i'm interesting also in this patch since i've compiled the last glest on AmigaOS4 PPC.. well, it starts but very soon i receive:

Bits per sample must be 8 or 16:

this seems related to the endianess of the machine..

Am I wrong?

Andrea

No, but Glest developers are not interested at all in this patch and they even violently opposed to this patch.

If you want to give his patch a try, you must be aware that it wasn't updated to match the 3.x.y code and you might want to try to start from this patch (which is reported to be incomplete and should apply cleanly to 2.0.0):
http://http://svn.debian.org/wsvn/pkg-games/packages/trunk/glest/debian/patches/be-support-from-kozz?op=file&rev=0&sc=0

[afxgroup]

i've just patched 3.x.x code and now that error has gone away..
now i have an error when trying to use glTexImage2D function..
must investigate. maybe pixmap.getW() and pixmap.getH() are wrong for an endianess problem

[eddyp]

i've just patched 3.x.x code and now that error has gone away..
now i have an error when trying to use glTexImage2D function..
must investigate. maybe pixmap.getW() and pixmap.getH() are wrong for an endianess problem

You mean you updated the patch to work over 3.x.x? Are you willing to maintain this patch?

[afxgroup]

i can if only i manage the game to work at least the first time.. and now i haven't.. i get that error i wrote above. must check what could be the problem.

[softcoder]

I have added code in megaglest that may finally address the big / small endian issues. I do not have a system to test on, but feel free to compile the svn version of megaglest and test.

[eddyp]

I have added code in megaglest that may finally address the big / small endian issues. I do not have a system to test on, but feel free to compile the svn version of megaglest and test.

I recommend you contact the Debian maintainer of megaglest pkg-games-devel@lists.alioth.debian.org and suggest an experimental package to be uploaded to get some feedback from more powerpc users.