angelscript and xcode 2.4

It would be best if someone that already uses AngelScript on Mac OS X could take a look at this.

I do not have access to Mac OS X myself so I cannot help you out on this.

However, you should be able to use the library in portability mode, by definining the AS_MAX_PORTABILITY flag when compiling the library. You'll have to write wrapper functions using the asCALL_GENERIC calling convention, but at least it works.

Let me know if you figure out the solution, so that I can include it in the official version.

Regards,
Andreas

i've cleaned up and maede that piece of code compile in macosx some time ago.
i think i've used gcc 3.x to compile it. and at least it was compiling and linking
on my machine (ppc g4 - xcode 2.3), but there was errors trying to make it running...
dunnow why you are encoutering some errors in linking, maybe try switch your compiler
to a previous version (i.e. not use 4.x)...

Quote:

Original post by kunitoki i've cleaned up and maede that piece of code compile in macosx some time ago. i think i've used gcc 3.x to compile it. and at least it was compiling and linking on my machine (ppc g4 - xcode 2.3), but there was errors trying to make it running... dunnow why you are encoutering some errors in linking, maybe try switch your compiler to a previous version (i.e. not use 4.x)...

Tried that already.. Tried compiling with gcc3.3 and same thing occured. And this is a fresh clean install of Xcode 2.4.

Did you compile yours w/ PIC or without PIC?

Quote:

Original post by kunitoki i've cleaned up and maede that piece of code compile in macosx some time ago. i think i've used gcc 3.x to compile it. and at least it was compiling and linking on my machine (ppc g4 - xcode 2.3), but there was errors trying to make it running... dunnow why you are encoutering some errors in linking, maybe try switch your compiler to a previous version (i.e. not use 4.x)...

Update..

I figured out why the "gnuc" makefile project compiles.. the ppc source file hadn't been added to it.

The error I'm getting is Gcc3.3 AND 4.0 not liking the way the assembly is done.. as if I comment it out,it compiled (though it obviously wouldn't work then :-D )

i don't know exactly, but i think i was compiling without pic. anyway i was using xcode 2.3, and the ppc file was currently in the project...
some time passed from when i had checked this, but since i needed to focus on
other stuff, i leaved it for a while.

OK.. I got it compiling w/ PIC enabled now.

And you'll laugh at the fix.

I added ".text\n" to the beginning of the asm ("") block.

Also, the as_config.h needs to be updated to check for __i386__ and NOT __INTEL__
as __INTEL__ isn't defined in gcc4.0.1 (at least), however __i386__ is defined when compiling -arch i386.

next to see if it works and do further debugging (already went through and read up on the ABI.. pretty icky).

nice... i was really really stuck when i tried to debug that. it seems that the bytecode generated is correct,
but when running it some strange things happens, since i'm not that guru of ppc asm programming...

kunitoki, were you at aLL successful in getting things running on PPC? even in PORTABILITY mode?

As right now I've compiled up the test_features app and it's segfaulting in the asCContext::ExecuteNext function. and while stepping through that function in GDB on the mac and comparing it w/ walking it through my linux box (x86) I am seeing some nice discrpencies of which AngelScript instructions it's trying to run.

example..

the first 2 words are as follows:

00000062 00000040

And on the PPC Mac it calculates the first instruction as
BC_POP, where as the linux box iterprets it as BC_PGA. So the VM (ExecuteNext) and/or compile is not taking into account the endianess of the PPC.

So either the compiler needs to take into account the endianess and swap the bytes there for the interpreter, or the interprater needs to swap the bytes as it's interpreting (probably having the compiler adjust them would be faster)

I know AngelScript is working on PPC in portability mode. It is already being used in some projects.

I wish I could help. It's frustrating to see that you're having so much trouble getting my library to work.

[edit]Apparently I didn't know so well ;)[/edit]

[Edited by - WitchLord on October 4, 2006 7:38:54 PM]

I guess I need to see how someone patched angel script to get it running on PPC, or maybe they were using an old version? but 2.7.0 does not want to correctly run.

as going through the main executenext function it handles the instruction different in regular code and the "Debug" code.

The main issue I'm running into right now is the PPC encodes the first 2 instruction when running the saveload test as

00 00 00 62 00 00 00 40

the INTEL encodes it as

62 00 00 00 40 00 00 00

due to Endianess

And in your debugging code you (w/ AS_DEBUG) you do
(*l_bc) & 0xFF

Which works correctly on both architectures since the value is treated as the full DWORD.

where as
*(asBYTE *)l_bc == (*l_bc) & 0xFF only in little endian systems (as you treat the l_bc array as bytes and grab the first one, which on PPC is 00)

I've changed that decoding in the switch to (*l_bc) & 0xFF, however it then blows up when fetching the WORDARG0 since you are making assumptions about the order of the data in the memory. (for the BC_PGA instruction)

When I left the instruction decoding as *(asBYTE *)l_bc and instead swapped the data in asCByteCode::Output (just the main instruction) things progressed slightly further except when it tried building the dynamic module initiated line 152 of test_saveload.cpp when it tried finding labels in asCByteCode::FindLabel as it tried finding a label w/ a number somewhere in the neighborhood of 0x30000000 (after a first calling of that function finding a label of 0x01).

Now, any assistance you can give on how things are encoded in the byte code would be helpful so I can decipher this and get it running on OS X ppc (and eventually OS X x86). I am being contracted to port a Win32 game over to Linux and Mac OS X and game author chose angelscript due to it's ease of setup and use and portability. And so far I'm impressed with the design and would rather NOT have to gut the scripting engine in this game with another and would like to try to get angelscript running on all platforms, thus adding another platform for this game that I am porting AND adding value to your great scripting language

http://www.outoforder.cc/ is my personal website (with many open source projects of my own)
http://www.frictionalgames.com/ is the home page of the game Penumbra that I am porting.

Witchlord,
Would you have some time for one on one IM/Chat sometime during the week or over the weekend that we can go over things as I decipher more of angelscript in tracking down these issues? Let me know if and when you may have time? My Timezone is at GMT-0500 (Indiana, US).

Regards,
Edward Rudd

What you're saying makes sense. I'm starting to think that perhaps others are not really using AngelScript on PPC after all. Maybe it's just Mac on the new Intel machines.

I'll have to review all the code that interacts with the bytecode directly to make sure they use macros that can be adapted to both little and big endian.

I think SourceForge.net has a PPC system on their compile farm, and I'll give it a try.

I don't know how much time I'll have this month for this though, as I'm currently in the middle of another project. So if you want to give it a go to get a start on it it would be great.

You'll have to start with a couple of source files:

as_bytecodedef.h - this is where the bytecodes are defined and their general structure
as_bytecode.cpp - this is where the bytecodes are constructed
as_context.cpp - this is where the bytecodes are interpreted
as_restore.cpp - this is where bytecode is saved to/loaded from disk (this is less importat to start with)

All bytecodes are packed into DWORDs. The least significant bytecode stores the instruction number. The most significant word may store a WORD argument.

A second DWORD may be used for more arguments, e.g. an additional two WORDs, or a DWORD.

Here's a table showing the layout of bytecodes (least significant byte to the right):



I don't use IM. Send me an e-mail. I check it almost every hour. My timezone is GMT -0300 (Brasilia, Brazil).

I'll do my best to help you out on this problem. Penumbra is a really cool looking game, and I would be very honored if it still uses AngelScript when it gets released.

Regards,
Andreas

More exactly we need to make sure the way the bytecodes are packed in asCByteCode::Output() is the same way they are interpreted in asCContext::ExecuteNext().

I haven't done any porting between little and big endian systems before, but I guess the biggest problem is the pointer casts? I mean, shifts and bitwise-and should work just the same way independently of endianess, right?

It should be possible to rewrite the macros on lines 1080 to 1091 in as_context.cpp to be endian independent, using shifts and bitwise-and instead of pointer casts.

What I've done thus far is add some macros into as_config.h to set some ENDIAN swap macros up.
and then I've wrapped those around the appropriate pieces in the asCByteCode::Output function to encode them in the correct place. and converted ALL placed in ExecuteNext to use a consistent *(asBYTE*) cast for the instruction.

I another idea of encoding the data in a consistant way without the need for swapping bytes and shifting that I'm going to try today (for the sake of speed).

The main issue I'm having now is the FindLabel function failing.. I'm going to investigate that today as well and report back here w/ what I find.

Shifts could be different, if you're doing them with ASM instructions. The architecture might provide two different kinds of shift instructions. One that shifts bits how they actually appear in memory, and the other how they are 'logically'. The 'logical' shift should behave the same with both endians.

Quote:

Original post by Deyja Shifts could be different, if you're doing them with ASM instructions. The architecture might provide two different kinds of shift instructions. One that shifts bits how they actually appear in memory, and the other how they are 'logically'. The 'logical' shift should behave the same with both endians.

More fun:)

I opted to change the encoding function to using pointer arithmatic (my version w/o shifts/byte swapping) and it works.. Which it should as that's how the datais being decoded.

Now just stepping through the test apps and finding everywhere else where the assumption is made that the bytes are in a different order. And there are a few.. . This is going to take a while :(

I'll help you. Can you send me the code changes that you've made so far?

I was going to try compiling angelscript on sourceforge.net's PPC today, but unfortunately it seems that all three of the PPC machines are out of order at this moment :(

They have a Power5 machine as well, so I gave it a try. I'm not familiar with this CPU, but it looks to be big endian, so I might be able to do some work with it until the PPC machines are available again.

Does anyone have any idea on how to know which preprocessor macros are defined by default using gnuc? I tried:

touch test.cc
g++ -dD test.cc

But it doesn't give me any output.

Regards,
Andreas

Quote:

Original post by WitchLord I'll help you. Can you send me the code changes that you've made so far? I was going to try compiling angelscript on sourceforge.net's PPC today, but unfortunately it seems that all three of the PPC machines are out of order at this moment :( They have a Power5 machine as well, so I gave it a try. I'm not familiar with this CPU, but it looks to be big endian, so I might be able to do some work with it until the PPC machines are available again. Does anyone have any idea on how to know which preprocessor macros are defined by default using gnuc? I tried: touch test.cc g++ -dD test.cc But it doesn't give me any output. Regards, Andreas

g++ -E -dM - < /dev/null > gcc-predefines.h

That will list all the predefines to gcc-predefines.h
(I googled for that one myself many times)

unified Diff based on SVN trunk
http://www.outoforder.cc/downloads/patches/PPCFix.diff

This includes restructuring of the asCByteCode::Output, asCRestore read and write byte fixes as well as many fixes to ExecuteNext (haven't yet HIT all instructions so not everything is tested).

I also re-tested things on my x86 system to be sure things were still working the same.

I also fixed the as_debug.h so it properly declared _mkdir for GCC compilers.

Suggestion:
set the svn:eol-style property on all the source files to the value 'native', that way I won't have line ending h*** problems when I create my diffs :-D.

the callfunc_ppc function hasn't been tested AT all (obviously) and I'm currently still running in MAX_PORTABILITY mode (and DEBUG mode)

I got past the FindLabel issue (wrong encoding of instructions in ::Output)
and the SaveLoad test runs perfectly now.
TestAny, however fails miserably w/ a script assert of

--- Assert failed ---
func: void TestAny()
mdle:
sect: TestAny
line: 11
---------------------

I'm done for the night.. sleep calls me.

Am I correct that the bytecode produced by the PPC version is different from the one produce by the x86 version? Are you byteswapping during the save/restore phase so the on-disk representation is identical?

No, I have not done that yet. That is going to require a good amount of rewriting of the save/restore functions to get them to save/restore with byte swapping.