SRL Archive - Development

Simba Automated Build System (nightlies)


moparisthebest

2010-09-14 16:12:11

This has been deprecated, look here:
http://villavu.com/forum/showthread.php?p=835249#post835249

I finally found time to finish up my automatic build system, it is set up on a cronjob to check every hour if the git repository has been updated, and, if it has, it tries to build Simba for all architectures and operating systems that are currently supported.

Developers will find this useful because they can see detailed build logs for each commit. And get test releases out faster for users to debug.

Users will find this useful because they can find cutting-edge releases of Simba, and help developers spot regressions and newly-introduced bugs faster.

That is where all the files reside:
http://www.moparisthebest.com/downloads/simba/

You can always find the last release that successfully compiled here:
http://www.moparisthebest.com/downloads/simba/current/

You can always find the last error log here:
http://www.moparisthebest.com/downloads/simba/errors/make.last_error.log
Or all of them here:
http://www.moparisthebest.com/downloads/simba/errors/

And lastly, the tools I use to run this here:
http://www.moparisthebest.com/downloads/simba/make_tools/

If you have any questions, suggestions or the like, feel free to post them here.

@Wizzup, if you want these to go to your server as well, if you set me up an account I can rsync them there, or any other way you want to go.

Woohoo! :spot:

noidea

2010-09-14 20:28:00

Very cool. This is really great. A lot of the phone development is setup with nightlies over at xda.
*Thumbs up*

moparisthebest

2010-09-14 20:32:14

Technically these are 'hourlies', since it checks every hour, but I've never heard of that so I figured I'd call them 'nightlies'. :p

DeSnob

2010-09-14 20:44:05

Oooh.. this is cool! You're doing a lot for the community.. Thanks :o)

HyperSecret

2010-09-14 20:49:40

Very nice mopar! It awesome to see you contribute to Simba like you have been.

The error logs are created from your machine that Simba is compiled on correct? And this is only useful to people who compile Simba source, right?

moparisthebest

2010-09-14 21:06:55

From: HyperSecret
Very nice mopar! It awesome to see you contribute to Simba like you have been.

The error logs are created from your machine that Simba is compiled on correct? And this is only useful to people who compile Simba source, right?


The make logs are probably only useful to developers, the curious, or people who compile it from source, but the binaries are useful to any user that wants to run Simba. :)

Wizzup?

2010-09-16 09:55:38

Awesome. ATM I'm actively reconfiguring some systems at home, but once I get some time I'll see if it's a good idea to sync them.

nielsie95

2010-09-16 14:57:09

Great! :)

moparisthebest

2010-09-16 21:29:42

From: Wizzup?
Awesome. ATM I'm actively reconfiguring some systems at home, but once I get some time I'll see if it's a good idea to sync them.


If you don't want me to push them to you, you could always setup a cronjob to just mirror that folder from my website with wget. I could write the script for you if you want and you could set it to run nightly or something.

Or if you trust me, I could push them to you with rsync, whatever. :)

(also, commit my patch so that I can compile the latest simba automatically already! :p)

moparisthebest

2010-09-22 03:52:14

I've updated it to include irokiplugin and skyplugin automatically as well as hopefully fixing SMART for 64-bit windows. I wrote a Makefile for compiling all pascal plugins automatically, you can grab the whole toolchain in a 7z file:
http://www.moparisthebest.com/downloads/simba/make_tools/plugin_builder/Makefile

It also compiles SMART as well, for all architectures of course. I just realized I didn't change the comments at the top, oh well it's too late now, I'll do it later.

Now if I can just get Wizzup? to commit my patch from here:
http://villavu.com/forum/showpost.php?p=737873&postcount=4
Then the Simba all-in-one release can work for all platforms which will make it many times easier to test multiple platforms from the same file. :)

Wizzup?

2010-09-22 05:12:08

I don't think Skyplugin is required anymore. It is in the Simba sources, but not in http://wizzup.org/static/srl/simba_plugins.tar.bz2
I'll have to look at the patch still, but I'll have some time in the evening.

moparisthebest

2010-12-17 12:35:09

I just read Wizzup?'s thread about a new stable version of Simba coming out soon:
http://villavu.com/forum/showthread.php?t=60868

So I went ahead and made some changes to my directory structure with the autobuild script to make it easier to find the stable versions (the ones with tags):
http://www.moparisthebest.com/downloads/simba/
As in you can always get the latest stable version from this URL:
http://www.moparisthebest.com/downloads/simba/stable/current/
(at the time of this writing, it is simba-0.94, which is also accessible from here:
http://www.moparisthebest.com/downloads/simba/stable/simba-0.94/ )

this is not automatic however, I have to manually add in the symbolic links for the stable tags, because I can't figure out a good way to get git to give me the tag name for the current commit, if anyone knows a good clean solution please let me know?

The closest I can come to grabbing the current tag is this:
$ git describe $(git rev-list --max-count=1 HEAD)
simba-0.94-26-g8bd3fd1
But, as you can see, it creates a 'tag' even though the current HEAD isn't tagged, I'd prefer a command that returned nothing if the HEAD wasn't tagged, or a tag if it was, or something along those lines, any ideas?

moparisthebest

2010-12-27 10:19:15

Does 64-bit windows Simba work for anyone? I can't really test them, but it would be good if someone could test newer and older ones to find one that works and where it broke:

http://www.moparisthebest.com/downloads/simba/unstable/

(wine64 seems to crash on everything for me, I need to recompile it with a newer gcc)

Dgby714

2010-12-27 10:48:46

From: moparisthebest
Does 64-bit windows Simba work for anyone? I can't really test them, but it would be good if someone could test newer and older ones to find one that works and where it broke:

http://www.moparisthebest.com/downloads/simba/unstable/

(wine64 seems to crash on everything for me, I need to recompile it with a newer gcc)


It runs, but the pascalscript is a little screwy.

moparisthebest

2010-12-27 11:24:43

Oh, so Simba itself ran, but SMART was the one with the access error?

(if so, that's good, it's probably due to the AnsiString issue)

Wizzup?

2011-01-03 17:08:39

Mopar, I tagged simba-0.95. It's unreleased, but I'm just letting you know.

moparisthebest

2011-01-04 09:32:33

From: Wizzup?
Mopar, I tagged simba-0.95. It's unreleased, but I'm just letting you know.


Ok, I've updated the stable links, as always, the most current is here:

http://www.moparisthebest.com/downloads/simba/stable/current/

which just points to:

http://www.moparisthebest.com/downloads/simba/stable/simba-0.95/

And they all have the latest SMART from ben's SVN.

Wizzup?

2011-01-04 10:47:19

It's not yet current, but fine. ;)

moparisthebest

2011-01-05 04:39:18

From: Wizzup?
It's not yet current, but fine. ;)


The commit you tagged is the one that is going to be declared 'stable' isn't it? (barring any unforeseen bugs, of course, but I doubt that will be a problem) If so, I won't have to change anything, if not I'll just have to update 'current' (and presumably you will tag another commit 0.96 or similar)

Wizzup?

2011-01-06 11:34:35

From: moparisthebest
The commit you tagged is the one that is going to be declared 'stable' isn't it? (barring any unforeseen bugs, of course, but I doubt that will be a problem) If so, I won't have to change anything, if not I'll just have to update 'current' (and presumably you will tag another commit 0.96 or similar)


It is going to be declared current, until I tag a new one. :)

moparisthebest

2011-06-25 18:27:26

The last few builds seem to have failed, can someone have a look at the error logs and tell me if it's something I need to change or install or something you guys can do?

Semi-ontopic, how is support for Linux coming along? I see you may be trying to integrate a new scripting engine, will it support linux (x86 and amd64), or is it a hodge-podge of platform dependent code as well?

Wizzup?

2011-06-25 18:39:06

From: moparisthebest
The last few builds seem to have failed, can someone have a look at the error logs and tell me if it's something I need to change or install or something you guys can do?

Semi-ontopic, how is support for Linux coming along? I see you may be trying to integrate a new scripting engine, will it support linux (x86 and amd64), or is it a hodge-podge of platform dependent code as well?


Well, the new scriping engine is coming around quite nicely, although we're not completely sure when it will arrive. Nielsie wrote it from scratch and on his own, if he gets tired from writing it, it may never come. </disclaimer> But it's certainly going the right way. There's a lape-integration branch I made just today, with the work done by Niels and Dgby714 merged into it.

Oh, yeah. It will work on any platform FPC supports because it contains no ASM/Calling hacks whatsoever. Downside is that you need to write/generate lots of wrappers. Something Niels/Ray already did for RUTIS, some time back. I've tested it on Linux x64 and it worked fine. So once that works we can stress-test the MML code for Linux, I expect to run into at least a few bugs.

What errors are you getting on your builds? I know there's a link to it, but I want to go to bed now and can look in more detail later. IIRC your lazarus was a bit outdated last time I checked, but I'm not sure if that's still the case. :)

MylesMadness

2011-06-25 19:45:03

Didn't Ben write a scripting engine? Why isn't that being used?

Dgby714

2011-06-25 19:50:05

From: MylesMadness
Didn't Ben write a scripting engine? Why isn't that being used?


Never completed.

moparisthebest

2011-06-25 23:38:20

Thanks for the info on the scripting engine, sounds promising at least. :)

https://www.moparisthebest.com/downloads/simba/current/make.2011-06-24_10_09.log
The error on 64-bit linux is this:
Compiling /home2/simba/simba/Units/Synapse/ssl_openssl_lib.pas
Fatal: Can't find unit Libc used by ssl_openssl_lib

Which is easily solved, looking into the log from compiling 32-bit linux is this:
ssl_openssl_lib.pas(88,7) Hint: Unit "libc" not used in ssl_openssl_lib

It's not even used, so if you remove 'Libc' from the uses in ssl_openssl_lib.pas, both compile just fine, I don't really want to bother with submitting a patch file for that.

The other issue is -lkeybinder cannot be found, even though I installed the package from the package manager, I'm pretty sure that is a environment error on my part though, so I'll have to look into it more.

edit:
Turns out the package 'python-keybinder' contains no native code at all, and no keybinder.h exists in there either. I downloaded the source tarball and manually compiled and moved the files to the right position for amd64 and x86 linux, then, with this patch:
diff --git a/Units/Synapse/ssl_openssl_lib.pas b/Units/Synapse/ssl_openssl_lib.pas
index fa85f70..23234de 100644
--- a/Units/Synapse/ssl_openssl_lib.pas
+++ b/Units/Synapse/ssl_openssl_lib.pas
@@ -85,7 +85,7 @@ uses
Classes,
synafpc,
{$IFNDEF MSWINDOWS}
- Libc, SysUtils;
+ SysUtils;
{$ELSE}
Windows;
{$ENDIF}
Everything works fine.

Now the only issue is probably the scripting engine on anything but windows, get that working! (or please, python interpreter!)

edit2:
Also just a quick update on Mac OSX support, I managed to get support for compiling both PPC and i386 Mac OSX executables with gcc on my Linux box, so since the libraries and such are all set up correctly, it shouldn't be too hard to convince FPC that it can also cross-compile a Mac executable for this auto-build setup, but I probably won't mess with it until a cross-platform interpreter is finished, because Simba is basically useless on all but win32 until then.

In short, I can probably make the automated build system spit out Mac OSX executables too.

Wizzup?

2011-06-26 06:15:06

From: moparisthebest
Thanks for the info on the scripting engine, sounds promising at least. :)

https://www.moparisthebest.com/downloads/simba/current/make.2011-06-24_10_09.log
The error on 64-bit linux is this:
Compiling /home2/simba/simba/Units/Synapse/ssl_openssl_lib.pas
Fatal: Can't find unit Libc used by ssl_openssl_lib

Which is easily solved, looking into the log from compiling 32-bit linux is this:
ssl_openssl_lib.pas(88,7) Hint: Unit "libc" not used in ssl_openssl_lib

It's not even used, so if you remove 'Libc' from the uses in ssl_openssl_lib.pas, both compile just fine, I don't really want to bother with


Right, I've fixed this in our main repos. We might want to file a bug at Synapse, though… Weird actually, I've never ran into an issue with this. What
is your FPC version? (I use 2.4.2 stable ATM)


From: moparisthebest
T
The other issue is -lkeybinder cannot be found, even though I installed the package from the package manager, I'm pretty sure that is a environment error on my part though, so I'll have to look into it more.

edit:
Turns out the package 'python-keybinder' contains no native code at all, and no keybinder.h exists in there either. I downloaded the source tarball and manually compiled and moved the files to the right position for amd64 and x86 linux, then, with this patch:


Yeah, different package. ;)

From: moparisthebest
T
diff --git a/Units/Synapse/ssl_openssl_lib.pas b/Units/Synapse/ssl_openssl_lib.pas
index fa85f70..23234de 100644
--- a/Units/Synapse/ssl_openssl_lib.pas
+++ b/Units/Synapse/ssl_openssl_lib.pas
@@ -85,7 +85,7 @@ uses
Classes,
synafpc,
{$IFNDEF MSWINDOWS}
- Libc, SysUtils;
+ SysUtils;
{$ELSE}
Windows;
{$ENDIF}
Everything works fine.

Now the only issue is probably the scripting engine on anything but windows, get that working! (or please, python interpreter!)


It's certainly in a better state than a few months ago. So it is coming. :-)

From: moparisthebest

edit2:
Also just a quick update on Mac OSX support, I managed to get support for compiling both PPC and i386 Mac OSX executables with gcc on my Linux box, so since the libraries and such are all set up correctly, it shouldn't be too hard to convince FPC that it can also cross-compile a Mac executable for this auto-build setup, but I probably won't mess with it until a cross-platform interpreter is finished, because Simba is basically useless on all but win32 until then.

In short, I can probably make the automated build system spit out Mac OSX executables too.


Well, that's pretty cool. We'll probably need to make quite some changes to the code though, and to be fair, I've never tried hard to make it compile for Mac, as I simply don't own a system and it runs terribly unstable in Virtual Machines.
At least, last time I tried it did.

moparisthebest

2011-06-26 11:16:57

I read all of Markus's notes regarding Mac support, but most of that appears to only be needed if you don't want to require OSX users to have an X server installed, which I don't really see as a problem as it's a built-in option on all OSX versions that I know of. I think if Simba works fine under X11 on Linux, then it should do the same under X11 on OSX with little to no changes.

As for testing things on OSX, I generally ask a few members of moparisthebest.com to help out and they are glad to do so. I also own an old PowerMac G3 ($10 @ flea market :)) that I put OSX Tiger on, which I can test PPC things on, and have OSX Leopard running fine in Virtualbox to test x86 things on. I'm not even sure if FPC supports Mac PPC as an architecture anymore, since Apple has kind of phased that out, but I can compile and test both with gcc/g++.

Wizzup?

2011-06-26 12:48:28

BTW, 0.96 (tag) is stable. I've also added some 0.97-rcX tags but they are of no real relevance yet, the next one probably will be.

Sir R. M8gic1an

2011-11-09 12:14:57

Hey, any chance you can score me a mac build?

-RM

Wizzup?

2011-11-12 07:45:52

From: moparisthebest
I read all of Markus's notes regarding Mac support, but most of that appears to only be needed if you don't want to require OSX users to have an X server installed, which I don't really see as a problem as it's a built-in option on all OSX versions that I know of. I think if Simba works fine under X11 on Linux, then it should do the same under X11 on OSX with little to no changes.

As for testing things on OSX, I generally ask a few members of moparisthebest.com to help out and they are glad to do so. I also own an old PowerMac G3 ($10 @ flea market :)) that I put OSX Tiger on, which I can test PPC things on, and have OSX Leopard running fine in Virtualbox to test x86 things on. I'm not even sure if FPC supports Mac PPC as an architecture anymore, since Apple has kind of phased that out, but I can compile and test both with gcc/g++.


The only issue will be that Simba will only be able to target X11 windows.

By the way, thanks a LOT for these autobuilds. They're wonderful and really save me a lot of time.

E: It does seem to be down now, though. :(
E: And back again, kinda.