SRL Archive - Development

I finally got Simba to compile under 64-bit linux, then I went through the extremely long, hard and barely documented steps to set up an environment where I can cross-compile Simba binaries for linux-i386, win32, win64, and linux-x86_64. After hours of pain it now works perfectly, but I noticed something strange, all of the executables are around 30 MB, and they aren't even statically linked:

mopar@killer-linux:~/simba/simba/Projects/Simba$ ls -lah Simba.i386-* Simba.x86_64-*
-rwxr-xr-x 1 mopar mopar 37M 2010-09-03 14:52 Simba.i386-linux
-rw-r--r-- 1 mopar mopar 30M 2010-09-03 15:09 Simba.i386-win32.exe
-rwxr-xr-x 1 mopar mopar 24M 2010-09-03 14:56 Simba.x86_64-linux
-rw-r--r-- 1 mopar mopar 31M 2010-09-03 15:15 Simba.x86_64-win64.exe
mopar@killer-linux:~/simba/simba/Projects/Simba$ file Simba.i386-* Simba.x86_64-*
Simba.i386-linux:       ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
Simba.i386-win32.exe:   PE32 executable for MS Windows (console) Intel 80386 32-bit
Simba.x86_64-linux:     ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, not stripped
Simba.x86_64-win64.exe: PE32+ executable for MS Windows (console)

They do all run as expected, so what's the problem?

I have attached all my make files to this post in an archive.

You need to strip the binaries, as Wizzup? taught me to do.

From: moparisthebest
I finally got Simba to compile under 64-bit linux, then I went through the extremely long, hard and barely documented steps to set up an environment where I can cross-compile Simba binaries for linux-i386, win32, win64, and linux-x86_64. After hours of pain it now works perfectly, but I noticed something strange, all of the executables are around 30 MB, and they aren't even statically linked:

mopar@killer-linux:~/simba/simba/Projects/Simba$ ls -lah Simba.i386-* Simba.x86_64-*
-rwxr-xr-x 1 mopar mopar 37M 2010-09-03 14:52 Simba.i386-linux
-rw-r--r-- 1 mopar mopar 30M 2010-09-03 15:09 Simba.i386-win32.exe
-rwxr-xr-x 1 mopar mopar 24M 2010-09-03 14:56 Simba.x86_64-linux
-rw-r--r-- 1 mopar mopar 31M 2010-09-03 15:15 Simba.x86_64-win64.exe
mopar@killer-linux:~/simba/simba/Projects/Simba$ file Simba.i386-* Simba.x86_64-*
Simba.i386-linux:       ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
Simba.i386-win32.exe:   PE32 executable for MS Windows (console) Intel 80386 32-bit
Simba.x86_64-linux:     ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, not stripped
Simba.x86_64-win64.exe: PE32+ executable for MS Windows (console)

They do all run as expected, so what's the problem?

I have attached all my make files to this post in an archive.

ok, this may not be to do with the large files, but well done and congrats on making it compile, does this mean we can soon have a native x64 simba for windows?

~shut

I passed -Xs to fpc so it was supposed to strip them itself, regardless it did make them much smaller but they still look bigger than your guys' release:

mopar@killer-linux:~/simba/simba/Projects/Simba$ ls -lah Simba.i386-* Simba.x86_64-*
-rwxr-xr-x 1 mopar mopar 7.0M 2010-09-03 15:37 Simba.i386-linux
-rw-r--r-- 1 mopar mopar 3.8M 2010-09-03 15:37 Simba.i386-win32.exe
-rwxr-xr-x 1 mopar mopar 9.5M 2010-09-03 15:36 Simba.x86_64-linux
-rw-r--r-- 1 mopar mopar 5.1M 2010-09-03 15:38 Simba.x86_64-win64.exe
mopar@killer-linux:~/simba/simba/Projects/Simba$ file Simba.i386-* Simba.x86_64-*
Simba.i386-linux:       ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
Simba.i386-win32.exe:   PE32 executable for MS Windows (console) Intel 80386 32-bit
Simba.x86_64-linux:     ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
Simba.x86_64-win64.exe: PE32+ executable for MS Windows (console)

It is much more normalish though, thanks! :)

I've updated the make files to strip the executables after they are made, I attached the new make files to this post and am linking to an archive with all 4 binaries in it in case you guys want to test them.

http://www.moparisthebest.com/downloads/simba/simba_binaries.7z

My next step is to set up an automated build system on my server so that there can be binaries built for all supported systems on every commit or just when the devs want to. Maybe with some updated linux binaries some more linux people will get involved. :)

Awesome!

Yeah, FPC tends to generate binaries that are quite large, even after stripping. But ~4MB isn't that much of an issue I suppose.

From: Shuttleu
ok, this may not be to do with the large files, but well done and congrats on making it compile, does this mean we can soon have a native x64 simba for windows?

~shut

Simba works on x64 Windows, it just hasn't been tested well. (Read: Pascalscript hasn't been tested well)


Double post… Oh well.

E: I added the four Makefiles. I tested x86_64 and it worked, so I assume the others work as well.

We could probably make them a bit more interactive / easier to use, something I made for university:

.PHONY: default clean resetconfig x86_64 i386 configmessage config_target

arrgg = i386
all := Makefile
build = gcc -o $(1).o -c $(1).c -Wall -ansi -pedantic -ggdb -pipe -O2
binary := OS2
objects := scheduler.o
nazi := mt19937ar.o mem_alloc.o simul.o

default: config_target

.config:
    @echo "config_target: configmessage" > .config

configmessage:
    @echo "Please run '$(MAKE) x86_64' once to switch to amd64."
    @echo "Or run '$(MAKE) i386' to switch to Intel 32 bit."

#report

configclean: clean
    @echo Killing configuration
    @rm .config -f
    @echo "config_target: configmessage" > .config

x86_64: configclean
    @echo Setting config to x86_64
    @rm .config -f
    @echo "arrgg = IA64" > .config
    @echo "config_target: $(binary)" >> .config
    @echo "Now type $(MAKE) to begin build"

i386: configclean
    @echo Setting config to i386
    @rm .config -f
    @echo "arrgg = i386" > .config
    @echo "config_target: $(binary)" >> .config
    @echo "Now type $(MAKE) to begin build"

clean:
    rm $(binary) report.pdf report.log report.aux *.o -f

scheduler.o: $(all) scheduler.c schedule.h
    $(call build,scheduler)

$(binary): $(objects)
    gcc -o $(binary) $(objects) $(foreach object,$(nazi),$(arrgg)/$(object)) -lm -pipe

report: report.tex
    texi2pdf report.tex --silent

run:
    gdb ./$(binary)

-include .config

From: Wizzup?
Simba works on x64 Windows, it just hasn't been tested well. (Read: Pascalscript hasn't been tested well)


Double post… Oh well.

im gonna test x64 on windows

~shut

From: Shuttleu
im gonna test x64 on windows

~shut

You'll probably need a 64 bit SMART if you want to use SMART on 64 bit.

From: Wizzup?
You'll probably need a 64 bit SMART if you want to use SMART on 64 bit.

yeah, just found that out

Exception in Script: Error loading plugin C:\Simba\Plugins\libsmart.dll

:(

~shut

EDIT: how do i compile SMART for x64?

SMART works perfectly fine on Windows 7 64bit.
Simba works perfectly as well, but sometimes when you close SMART before stopping the script, Simba won't close. It'll take about 15 minutes before it decides to close sometimes.

From: holic
SMART works perfectly fine on Windows 7 64bit.
Simba works perfectly as well, but sometimes when you close SMART before stopping the script, Simba won't close. It'll take about 15 minutes before it decides to close sometimes.

The one your using is 32bit, Win 7 is just running it in 32 bit mode, its not the same.

So now all we need is SMART for the others?

From: Narcle
The one your using is 32bit, Win 7 is just running it in 32 bit mode, its not the same.

So now all we need is SMART for the others?

I know that, he was just asking if it ran, and I was saying it did.
Or was he asking about a native version?

From: holic
I know that, he was just asking if it ran, and I was saying it did.
Or was he asking about a native version?

64bit Simba, not 32bit

From: Wizzup?
I added the four Makefiles. I tested x86_64 and it worked, so I assume the others work as well.

That's fine for now, but I'm going to clean up the Makefile and incorporate them into one, just so you know. :)

And I personally tested both Linux binaries and the 32-bit windows one in wine, all of them ran (I didn't test scripts or anything more than a simple Writeln('bob'); ), wine crashed on the 64-bit executable, but that's probably a wine thing, I don't know if it even supports 64-bit windows executables.

And a quick question for Wizzup? before I finish writing up my automated build system, the x64 patch you gave me for RUTIS, it obviously compiles on 32-bit architectures too but is it OK to use on them? Does it work and such? Thanks. :spot:

edit: Finished up my unified Makefile, hopefully it can be included in the git repo instead of all the duplicated code in the ones I uploaded earlier? :stirthepot:

From: moparisthebest
That's fine for now, but I'm going to clean up the Makefile and incorporate them into one, just so you know. :)

And I personally tested both Linux binaries and the 32-bit windows one in wine, all of them ran (I didn't test scripts or anything more than a simple Writeln('bob'); ), wine crashed on the 64-bit executable, but that's probably a wine thing, I don't know if it even supports 64-bit windows executables.

Probably a WINE issue, yeah. Although 1.3 should support 64 quite well.

From: moparisthebest
And a quick question for Wizzup? before I finish writing up my automated build system, the x64 patch you gave me for RUTIS, it obviously compiles on 32-bit architectures too but is it OK to use on them? Does it work and such? Thanks. :spot:

Yeah, it should work on 32 architectures too. PtrUInt = Cardinal on 32 bit. I didn't apply it yet as I want a full patch first, so I can see what I changed in total.

From: moparisthebest

edit: Finished up my unified Makefile, hopefully it can be included in the git repo instead of all the duplicated code in the ones I uploaded earlier? :stirthepot:

Committed & pushed. :)

E: Any ideas if cross compiling would have any effect on the created binaries? I guess it doesn't, but it's one of the reasons I didn't auto generated them myself before, and instead use the binaries mastaraymond created.

sorry but i really want to test this 64 bit windows version, but i have no idea how to compile smart, whenever i do i get

From: cmd
C:\Users\Thomas\Desktop\New folder>make windows
"Compiling Main.cpp"
The syntax of the command is incorrect.
make: *** [build/windows/Main.o] Error 1

~shut

From: Shuttleu
sorry but i really want to test this 64 bit windows version, but i have no idea how to compile smart, whenever i do i get


~shut

Please make a different thread for this. :)

From: Wizzup?
Probably a WINE issue, yeah. Although 1.3 should support 64 quite well.



Yeah, it should work on 32 architectures too. PtrUInt = Cardinal on 32 bit. I didn't apply it yet as I want a full patch first, so I can see what I changed in total.



Committed & pushed. :)

E: Any ideas if cross compiling would have any effect on the created binaries? I guess it doesn't, but it's one of the reasons I didn't auto generated them myself before, and instead use the binaries mastaraymond created.

Just as I thought, I only have WINE 1.1 :)

Great, I was having nightmares thinking about applying the patch for the 64-bit builds and reversing it for the 32-bit builds. :p Not really, but it would have been annoying, is there any good way for me to make sure it works without running a runescape script?

About cross compiling. Honestly if we are both using the same library versions and FPC version the binaries should be identical regardless of what architecture/OS the compiler ran on, since all of the code written to produce them is identical. The only difference is the architecture the FPC code is compiled to run on, it should produce the same output. A simpler example would be a 'hello world' application, the code is the same, so it has the same output no matter what platform it is compiled for (hello world). If we could get the same version of all our libraries, the same Lazarus, and the same FPC I'd be willing to bet the sha1sums of the binaries produced would be identical. :)

edit: Now that I'm thinking about it, I don't know what effect endianess would have on this, or if you could even cross-compile for a big-endian system on a little-endian one or vice-versa, but we don't have to worry about this here because all these systems are the same.

From: Shuttleu
sorry but i really want to test this 64 bit windows version, but i have no idea how to compile smart, whenever i do i get

Toss me a link for the SMART thing (via PM) and I'll compile and ship it with the binaries (if the license allows).

From: moparisthebest
Just as I thought, I only have WINE 1.1 :)

Great, I was having nightmares thinking about applying the patch for the 64-bit builds and reversing it for the 32-bit builds. :p Not really, but it would have been annoying, is there any good way for me to make sure it works without running a runescape script?

About cross compiling. Honestly if we are both using the same library versions and FPC version the binaries should be identical regardless of what architecture/OS the compiler ran on, since all of the code written to produce them is identical. The only difference is the architecture the FPC code is compiled to run on, it should produce the same output. A simpler example would be a 'hello world' application, the code is the same, so it has the same output no matter what platform it is compiled for (hello world). If we could get the same version of all our libraries, the same Lazarus, and the same FPC I'd be willing to bet the sha1sums of the binaries produced would be identical. :)

edit: Now that I'm thinking about it, I don't know what effect endianess would have on this, or if you could even cross-compile for a big-endian system on a little-endian one or vice-versa, but we don't have to worry about this here because all these systems are the same.

As to verifying is Simba works properly, I don't know. We don't really have a lot of unit tests at this time.

Ok, so the cross compiling shouldn't cause any issues.

From: moparisthebest

Toss me a link for the SMART thing (via PM) and I'll compile and ship it with the binaries (if the license allows).

http://blanddns.no-ip.org:81/repos/public-smart

From: Wizzup?
As to verifying is Simba works properly, I don't know. We don't really have a lot of unit tests at this time.

I'm working on this one (pure scripting engine stuff though, nothing that has to do with the MML).

From: Markus
I'm working on this one (pure scripting engine stuff though, nothing that has to do with the MML).

That's really all that needs testing when swapping scripting engines. Let me know when it's done as I can test 3 of the 4 versions from this computer and linux.

From: Wizzup?
http://blanddns.no-ip.org:81/repos/public-smart

Great, and GPL'd too. I quickly got it to compile for 32 and 64-bit linux, and 32-bit windows, but found 64-bit windows gcc is hard to find and throws a strange assembler error, making me think it still has bugs. So I can't help with a 64-bit Windows SMART as of now, but I'll keep working on it. :)

From: moparisthebest
That's really all that needs testing when swapping scripting engines. Let me know when it's done as I can test 3 of the 4 versions from this computer and linux.



Great, and GPL'd too. I quickly got it to compile for 32 and 64-bit linux, and 32-bit windows, but found 64-bit windows gcc is hard to find and throws a strange assembler error, making me think it still has bugs. So I can't help with a 64-bit Windows SMART as of now, but I'll keep working on it. :)

Yeah… 64 bit windows as a pain for me too.

Here is the newest version of the Makefile with some enhancements and a few bug fixes (also makes a release package, but you don't have to include that if you don't want to)

http://www.moparisthebest.com/downloads/simba/Makefile

Also, I need to know if you are going to apply the x64-Rutis patch to the main branch, or if I will have to patch it myself to compile the 64-bit programs. That should be all I need to tidy up my automated build script. :spot:

I updated the makefile (hopefully) one last time, it now has options to make distributable archives with releases including plugin support for different architectures. It also applies and removes the rutis-x64 patch when compiling for x64 architectures.

The makefile, along with distributions for each architecture and an all-in-one release, can be found here:

http://www.moparisthebest.com/downloads/simba/

Committed the Makefile, thanks. :)

Sorry to go a bit off topic, but thanks a lot mopar!

You've really shined these past few weeks much like old times! :)

Thank you!!

From: Nava2
Sorry to go a bit off topic, but thanks a lot mopar!

You've really shined these past few weeks much like old times! :)

Thank you!!

This is the first and last time I will use this:

+1

sry for hijacking. Fawki's Leaving thread quickly and quietly.

@Wizzup?
No problem, if you're anything like me you HATE manually packaging releases, and will release more often if it's done for you automagically. :)

@Nava2 and Fakawi
Thanks! I basically lost interest in everything SRL and SCAR ever since I went to Linux years ago, and now I can't stand to boot into windows to play games, let alone program anything! So Simba is very exciting, especially with pending python support. :D

Another fix for the makefile:

http://www.moparisthebest.com/downloads/simba/Makefile

Sorry about the frequent updates, hopefully this is the last. :)

Committed.