SRL Archive - Development

I guess this won't be interesting to most of you - but I think it is important to put these problems / ideas general.

NOTE: In the context of threads -> Parent = main thread; child = non main thread; but they belong to the same process.

Introduction:

To be able to run several scripts (or even one) with Mufasa, we need to implement a way to start and stop a script. In this situation we have a Parent (The GUI), and childs. These childs can be a "Thread", or a "Process".

When we use a thread, the memory between the parent and childs is generally shared. A thread can be controlled from the main process; and dies with the Parent. Threads are generally easier to use than Processes, but also less powerful, as they fully depend on the parent. Communication is typically easier with threads, as the main thread has control over the rest of the threads, and can access it memory.

Mufasa uses threads.

Apart from the advantages I just named, there are also disadvantages.
One major disadvantage is that SMART, or rather, the Java VM can only be used once per process; not per thread. So we can't have two threads using a different SMART. I'm sure you can imagine what effect this has. This was even initially a problem; since we used to load Plugins once a script has started, and unloaded them once a script stopped; this stopping (and closing) SMART.
Currently we hacked this into Mufasa, so that plugins are loaded separately from the Script, but can still be used. Another notable fact is that FPC GUI's are only allowed to be run on the main thread, currently we have to perform a LOT of hacking to get it done; and it still allows the scripter to kill the main thread on accident.

A process on the other hand is (totally) independent of it's parent. (Well, not totally, but let's assume it is…)
Running a FPC GUI on a new process is no problem, as we will run the script on the main thread. Another notable advantage is that would could run it in the background; effectively without a GUI. (Client/Server structure anyone?)

We would also be able to run one JVM (SMART) per process, thus effectively running multiple scripts, each with their own smart. To the user this would make no difference, except for the fact that he would see multiple Mufasa processes in his Task Manager. Forking/Starting a new process isn't a problem - but communication is. We could easily start a new process, pass is the script to compile and execute, and it will compile and execute it. Additionally, it can also load libraries it needs. (SMART)
Now, if we want to stop the script, we can simply send a terminate signal, and it will close. This is will also close SMART.
This isn't dangerous, but very annoying to the user. (This is why Raymond and I made the plugins global in the first place!)

–EDIT–
We would start a new process with a "Script Loader Process". GUI -> fork() + exec('scriptloader');
– END-EDIT–

So, there are roughly two solutions.
The first one includes sharing the libraries, and certain memory maps. This can be quite complicated, and it is possibly still affected by the fact that the JVM is limited to one instance per process.

The other solution is excessive communication between the parent process and the child process. We can then make the parent "tell" the client to Stop the script. We then explicitly do not kill the child, thus keeping the libraries (SMART…) loaded. Once again, we would separate the libraries/plugins from the script. (Not a problem, since we only run one script per process now.)

However, PascalScript's .Execute method is blocking. This means we have no easy means to interrupt it's execution. (Killing it is possible, as mentioned above, but this will kill the entire process).

So, the second method is basically the only method we should use for processes. But inter process communication isn't easy, especially with blocking calls.

We could spawn a thread on the new process, that would perform all the communication, and could possibly call .Stop on the main thread. I don't know how stable this would be; as we absolutely wouldn't have any means to safely "Stop" the main the main thread running the script. Killing the main thread basically means killing the process, which we already can do, but then we will kill SMART.

Another solution would be to modify Pascal Script, so that it will perform Inter Process Communication while executing. For example, per line. (Pascal Script has an OnLine event)
This would be doable, I guess… But perhaps I am missing something…

As one can see, the major problems actually occur with SMART; since it is a special instance of a script.
We could also face the fact that SMART will close once we stop a script. But this can be very annoying. But it is definitely a possibility we should consider - as you can see the major problems aside around and with SMART.
If we ought to consider running only one script per Mufasa (GUI!) is a problem; then SCAR seems to have a big problem too.
So perhaps we don't really have a problem?

What are your thoughts?

Technical replies appreciated; but just giving your opinion on what we should do is also welcome.

Quick list:

• Stick to what you have, threads. We don't need more than one SMART. Load plugins seperately from script.
• Stick to threads, but load plugins at the start of the script; even if it means closing SMART.
• Switch to processes; and load plugins when the script starts. Don't care is SMART closes.
• Switch to processes; and try to find a way to gracefully stop the script without stopping the process, thus allowing SMART and others to live on. Don't load plugins on script start; but on process start. (Either with another thread; signals; or with OnLine)
• Added: Just do it like SCAR. (A bit like the first option)
• Other, will post

~Wizz

E: Alternatively, one could change the way SMART works, and turn it into a server, which would manage other smart instances (processes), and allow other processes to attach or detach to these smart instances. This would fully separate SMART from Mufasa. A bit like KYAB I guess. ;)

tl;dr

Sing a song if you love wizzup!

Signals/interrupts of some sort. But if we have to work this hard _AROUND_ smart, I say it just gets dropped.

I guess we wouldn't even need to change SMART; we can write a program to act as a server - it can just use the lib per process.

I say processes. If at all possible, make it so the executable can directly execute the scripts, that way the main program can just call it, along with some parameters, and let it run.
For the IO window, just use stdin/out/err, and to start and stop the script, just start it and kill it (however for pause that would need something different, so you could use debugger tactics and pause execution of the program)

If it is done this way, you would be able to run scripts without even needing a GUI at all. Of course, doing it this way would be considered advanced and a GUI should always be used unless it is better without.

Either way, separating the script into its own process allows for numerous advantages: 1) with some hacking you could run the GUI as root, then make the script run as nobody, 2) scripts can be chrooted, 3) in some cases, isolating the memory access would be a good thing, because then you couldn't screw up the GUI if some bug was found.

Just my two cents. I'm sure I went wrong somewhere, but please comment.

I think it would be best if you could open a process for each tab that's opened as the tab gets opened, that way you wouldn't have SMARTs closing and (if I understand correctly) it would allow a less hacked-together way of putting up forms (since each process won't have a GUI).

What I would recommend though, is if the scripts end up running on the main thread or something that would interfere with processing messages, rewrite your wait() so that it checks every 1000ms or so. Actually, you should do that anyway so we can pause halfway through waits.

Something like this,
[php]program new;
var
time: longint;


procedure Sleep(amount: longint);
var
time: longint;
begin
time := GetSystemTime;
wait(amount mod 1000);
repeat
wait(1000);
until (GetSystemTime - time >= amount);
end;


begin
time := GetSystemTime;
Sleep(33333);
Writeln('Slept for '+IntToStr(GetSystemTime - time)+'ms');
end.

Compiled succesfully in 16 ms.
Slept for 33337ms
Succesfully executed[/php]

Most of this has gone way over my head, but we don't want smart to close every time a script stops. Its probably not a problem for you guys, since your countries actually have good internet, but I get capped alot (down to dialup speed) and it takes ~10 minutes to fully load a smart client and start autoing. If I have to wait 10 minutes every time I want to stop and start a script, I'm sticking with SCAR.

On a slightly unrelated note, multiple instances of SMART and scripts running from the same GUI would be epic. No idea how to do it, but ibot can do it, so we can do it.

From: Widget
I think it would be best if you could open a process for each tab that's opened as the tab gets opened, that way you wouldn't have SMARTs closing and (if I understand correctly) it would allow a less hacked-together way of putting up forms (since each process won't have a GUI).

What I would recommend though, is if the scripts end up running on the main thread or something that would interfere with processing messages, rewrite your wait() so that it checks every 1000ms or so. Actually, you should do that anyway so we can pause halfway through waits.

Something like this,
[php]program new;
var
time: longint;


procedure Sleep(amount: longint);
var
time: longint;
begin
time := GetSystemTime;
wait(amount mod 1000);
repeat
wait(1000);
until (GetSystemTime - time >= amount);
end;


begin
time := GetSystemTime;
Sleep(33333);
Writeln('Slept for '+IntToStr(GetSystemTime - time)+'ms');
end.

Compiled succesfully in 16 ms.
Slept for 33337ms
Succesfully executed[/php]

We have that. :)

procedure psWait(t: Integer);
{$ifdef MSWINDOWS}
begin
  Sleep(t);
end;
{$else}
var
  EndTime : integer;
begin
  if t > 50 then
  begin;
    EndTime := GetTickCount + t;
    while (CurrThread.PSScript.Exec.Status = isRunning) and (GetTickCount < EndTime) do
      Sleep(16);
  end else
    sleep(t);
end;
{$endif} 

Anyway, now after sleeping a bit I think the best solution would probably be processes + kill after execution. We can still have smart running, if we turn it into some sort of "server" system; processes can attach and detach. We'd only need to share a bit of memory, and the other communication can go over sockets. (or pipes)

E: We wouldn't even need to modify SMART, I guess.

You can start more than one SMART, why not have a daemon that spawns SMARTs?

From: R0b0t1
You can start more than one SMART, why not have a daemon that spawns SMARTs?

That was the idea.

I think we''ll drop this until we have a stable Mufasa. Even though the plugins are global; and we can only use 1 SMART. It's too much of a hassle for now.

buuump with SMART now open source, we have some choices and more options.

Moar bumps!

i say proccesses, but as i heard, it will be much harder to impliment, so i say sort out all the major bugs enough for simba to be released, and not as a RC. then start on switching from threads to processes

but thats just my thoughts

~shut

From: Shuttleu
i say proccesses, but as i heard, it will be much harder to impliment, so i say sort out all the major bugs enough for simba to be released, and not as a RC. then start on switching from threads to processes

but thats just my thoughts

~shut

Simba is released ;)

From: NCDS
Simba is released ;)

simba is on RC5/0.9 ;)

~shut

Yeah I think the current options for Simba are like this:

• Leave it the way it is. (TTimer and other callbacks cant use CurrThread variable)
• Allow only one scriptthread. (Will fix the issue; but you can't run more than one script at the time)

Processes are something for Simba 2.0, IMO.

From: Wizzup?
Yeah I think the current options for Simba are like this:

• Leave it the way it is. (TTimer and other callbacks cant use CurrThread variable)
• Allow only one scriptthread. (Will fix the issue; but you can't run more than one script at the time)

Processes are something for Simba 2.0, iMO.

im guessing you mean you cant run more than one script at a time "per simba"

~shut

From: Shuttleu
im guessing you mean you cant run more than one script at a time "per simba"

~shut

Yes.

i vote for the second option

~shut

I don't think that many people will auto multiple scripts with one Simba right now until we can have multiple SMARTs spawned.

From: Shuttleu
i vote for the second option

~shut

Implemented.

And nava, yes, indeed.

Let me just post that this change has been reverted some time ago as it really wasn't a good idea. Everyone who posted here knows this, but people who only read this thread probably don't. It is still possible to run multible scripts in Simba.