SRL Archive - Help and Support

TPA Wrap?


Runaway

2012-05-28 21:18:38

What the heck is the difference between TPA "Wrap" functions and normal TPA functions?

I've been using:

SplitTPAWrap(TPA, 15, ATPA);
in my script, and now that I've switched over to:

ATPA := SplitTPA(TPA, 15);

am I just imagining that it's working way better, or is there actually some difference I don't know about?

Gu3st...

2012-05-28 21:26:46

It's exactly what it says it is; a wrapper.
In SplitTPAWrap it's taking ATPA as a variable, while in SplitTPA is returns an ATPA

At least that's what I think it is :p

tehq

2012-05-28 21:32:49

From http://docs.villavu.com/simba/scriptref/tpa.html

SplitTPAWrap - Splits the points with max distance Dist to their own TPointArrays and returns the result in the T2DPointArray Res.

SplitTPA - Splits the points with max distance Dist to their own TPointArrays.

Zyt3x is right on.

Runaway

2012-05-28 21:42:46

From: Zyt3x
It's exactly what it says it is; a wrapper.
In SplitTPAWrap it's taking ATPA as a variable, while in SplitTPA is returns an ATPA


From: tehq
Zyt3x is right on.


Is it possible that there is a problem with SplitTPAWrap? I have been getting weird errors lately that were making my ATPA's return a length of 0… then they randomly work again (even though I used the color picker and the colors being searched for are exactly the same still). Now that I've switched over to SplitTPA I have no abnormal TPA color-finding issues and the results are actually surprisingly more accurate.

Am I just crazy? :stirthepot:

E: By surprisingly more accurate, I mean:

Using SplitTPAWrap: 13.3 Loads/hr average
Using SplitTPA: 18 Loads/hr average

Flight

2012-05-28 21:47:00

I've heard somewhere it's better to use wrappers, something associated with memory leaks otherwise.

Runaway

2012-05-28 21:49:15

From: Flight
I've heard somewhere it's better to use wrappers, something associated with memory leaks otherwise.


Yeah, I think it was beginner5 that was talking about that. But my results seem to say otherwise… I'm so confused about this!

tehq

2012-05-28 22:22:07

Just played around with these functions and memory. The variable in the wrapper gets freed, but the result in the non wrapper isn't freed. This is because PS doesn't clean up the arrays when they're passed as a result.

Runaway

2012-05-28 22:33:26

From: tehq
Just played around with these functions and memory. The variable in the wrapper gets freed, but the result in the non wrapper isn't freed. This is because PS doesn't clean up the arrays when they're passed as a result.


So, does this mean that Simba's memory use slowly builds each time I use SplitTPA, and it doesn't if I use a wrapper?

Brandon

2012-05-28 22:59:54

Simba does not allow the overloading of functions (Functions with the same name, different parameters or return types) so instead wrappers are made inorder to keep a function with a similar name.

ClickMouse, ClickMouse2, Mouse are all basically wrappers of the same function. Memory for these functions that have the wrap in their names don't leak for me but for others they say it leaks :S

This is because PS doesn't clean up the arrays when they're passed as a result.


No clue where you get your info from. PS doesn't clean up after anyone. You free any allocated memory yourself. TPA's are just variables, they get freed just like every other variable when a function returns. Also the wrappers are the exact same as their counterparts with different parameters and return types. That's all there is to it.

Runaway

2012-05-28 23:13:15

From: Brandon
Also the wrappers are the exact same as their counterparts with different parameters and return types. That's all there is to it.


Ok. So does that mean that SplitTPA could be handling the same variable differently than SplitTPAWrap if there is already an ATPA stored in it with a random length?


SplitTPAWrap(TPA, 15, ATPA);

ATPA := SplitTPA(TPA, 15);

I get much better results with SplitTPA, could it be the fact that it clears the ATPA var before storing the data (or some difference that would be similar)? Or perhaps the memory leak in the wrapper you mentioned could be causing some issue? They are both placed in loops and I don't clear ATPA if the loop doesn't break out.

bg5

2012-05-28 23:35:37

Run this first:
program new;
var
TPA : TPointArray;
a : integer;
begin
for a := 0 to 1000 do
TPA := TPAFromBox(IntToBox(0,0,2000,2000));
end.


then restart Simba and run this

program new;
var
TPA : TPointArray;
a : integer;
begin
for a := 0 to 1000 do
TPAFromBoxWrap(IntToBox(0,0,2000,2000),TPA);
end.

to see the difference.

I get much better results with SplitTPA

Result should be the same ,except non-wrapper is leaking.

Brandon

2012-05-29 00:10:09

From: beginner5
Run this first….
Result should be the same ,except non-wrapper is leaking.


Wrappers do not leak.. wrappers and the counterpart function are one in the same..

If your going to benchmark the functions like that then take into consideration the difference between a pointer and assignment.

TPA:= TPAFromBox(…..) constantly assigns a ton of points to that TPA in your while/for-loop.

TPAFromBoxWrap(…..) just assigns a pointer to the already made TPA.

I may not have made the above clear so.. Consider the following:

Function AppendStringArrays(Texts, ToAppend: TStringArray): TStringArray; //Returns a COPY of the Resulting TPA.
var
I, J, K: Integer;
begin
J:= High(ToAppend);
K:= Length(Texts);
Setlength(Texts, J + K + 1);
For I:= 0 to J do
Texts[I + K] := ToAppend[I];

Result:= Texts; //Creates a TPA called Result, COPY Texts to the Resulting TPA and Return the Result.. This way, Texts is never modified.
end;


Procedure AppendStringArraysWrap(var Texts, ToAppend: TStringArray); //Fill Texts Directly with the results.. no Copies made.
var
I, J, K: Integer;
begin
J:= High(ToAppend);
K:= Length(Texts);
Setlength(Texts, J + K + 1);
For I:= 0 to J do
Texts[I + K] := ToAppend[I]; //Texts is thus modified and not the same as before.
end;

Pass-By-Reference vs. Pass-By-Value. Most programmers get screwed by this if they do what you just asked me to try. Yes I tried it and yes I ran out of memory AS EXPECTED for the PassByValue function whereas the PassByReference did not run out of memory. At the end of both runs, my CPU Usage returned to normal. During the time the function was ran, this behaviour was of course expected.

This will happen in ANY LANGUAGE when you use benchmark it like that. Anyway, the functions aren't leaking. They are behaving exactly how they are supposed to behave.

bg5

2012-05-29 01:22:52

At the end of both runs, my CPU Usage returned to normal.

CPU usage isn't important ,but memory usage. Memory usage doesn't return to normal after first one was crashed by error ,at least on my computer ,unless I close Simba:

TPA:= TPAFromBox(…..) constantly assigns a ton of points to that TPA in your while/for-loop.


Yes it assign tons of points to TPA. But meanwhile all not used memory should be freed. Program should take only so much memory as it need to make operation. Memory should not grow up ,because for every iteration it does same function ,so need same size of memory buffer.

You have right ,that Pass-By-Reference need less memory than Pass-By-Value, which works like:

- first create new variable ,do stuff ,and safe result to it: [ memory region of Result ]
- so there are now two memory regions: [ memory region of Result ] and [ memory region of TPA ]
- now assign label called TPA to [ memory region of Result ]

- is that all? Noo! What would you do in c++ ? You would free [ memory region of TPA ] ,because it's useless ,name TPA is now pointing to [ memory region of Result ]. But in PascalScript you can't manually free memory unless it's class object ,so you expect Simba will do it for you. But it won't! That's the issue.
So on every iteration inaccessible copy of TPA is left in memory and blows your RAM




And to be sure I tried do this in Delphi 7…no leaks.

program Project1;

{$APPTYPE CONSOLE}

uses
SysUtils;

type TPoint = record
x ,y : integer;
end;
type TPointArray = array of TPoint;

type TBox = record
x1,y1,x2,y2 : integer;
end;


var
TPA : TPointArray;
a : integer;

function IntToBox (a,b,c,d : integer) : TBOx;
begin

Result.x1 := a;
Result.y1 := b;
Result.x2 := c;
Result.y2 := d;

end;

function TPAFromBox2(const Box : TBox) : TPointArray;
var
x, y: integer;
l : integer;
begin;
SetLength(Result, (Box.x2 - Box.x1 + 1) * (Box.y2 - Box.y1 + 1));
l := 0;
For x := box.x1 to Box.x2 do
for y := box.y1 to box.y2 do
begin;
Result[l].x := x;
Result[l].y := y;
inc(l);
end;
end;

begin
for a := 0 to 10000 do
TPA := TPAFromBox2(IntToBox(0,0,2000,2000));
end.

Brandon

2012-05-29 01:54:20

No, no.. In C++ I would not free anything because it's just a variable. There is nothing to free. I've created my own versions of TPA's and Dynamic arrays in C++ Here: http://villavu.com/forum/showthread.php?t=82610

In your function above, your asking it to push back 4000 points onto a NEW variable every time..

As long as my variable isn't allocated with New or my variable isn't a Box Type then I won't need to free anything. You don't delete structs/records/classes unless allocated on the stack using the new operator or malloc or anything of the sort in C++.. Fortunately for us, Simba does not create NEW structs every time.

Also programs do not free memory immediately and they certainly do not always free it after a function returns. In this case your function isn't returning, it keeps allocating more and more space every time long before it returns. It's basically like calling SetLength 100000 times in a row. For all you know, SetLength could easily be zeroing out all the memory allocated, nulling the pointer to it and allocating a new set.

When the operating system needs memory or has free time, it goes and clears unused memory and pointers for you, providing that you didn't crash it first. When your program closes then your OS will do a sweep of the memory it used. Sometimes it may do it while it's running but that depends on a lot of things.

Example would be to declare a local variable and return a reference to it. This is undefined behaviour because sometimes the memory isn't cleared so the value it returns may be correct. Other times, it's cleared as soon as the function returns and will give you random ascii characters or NULL values.

Simba is interpreted fully. It's going to behave far differently from running a DLL or a console application.

Pass By Value:

Create a Temporary Variable on the stack.
Function runs it's crap here and constantly pushes back values onto that TPA.
Return a COPY of the TPA and assign it to the calling variable.
Temporary Variable MAY or may NOT have been cleaned yet.
Everytime this function is ran, a new set of memory is allocated to not overwrite the previous if any.

Pass By Reference:

Functions runs its crap here and pushes back values onto a variable that's already created via a pointer to the variable's memory buffer.
No cleaning necessary, no copying, no anything.

In C++ vectors do the job just right. Know what they do? They allocate MORE than the amount of memory required so they don't have to keep allocating over and over and over everytime you push_back(Point(X, Y)). This is why vector::capacity() is different from vector::size()

Imagine calling malloc in C 5000 times in a row to push back 5k values. It doesn't make sense to do that if you already know your going to push 5k values flat.

The function you just posted took 8876 milliseconds to run just once using 1000x700 for the box.. LOL! That's a long ass time.

I know my post is long but just bare with me.. Try this function (it is a bit slower due to creating a new temporary variable but it will not lag you or leak I promise):



{$I SRL/SRL.SImba}

Function TPAFromBox2(const Box: TBox): TPointArray;
var
TPA: TPointArray; //Creates a temporary variable to hold our result.. This variable has the highest chance of being destroyed when the function returns.
x, y: integer;
l : integer;
begin;
SetLength(TPA, (Box.x2 - Box.x1 + 1) * (Box.y2 - Box.y1 + 1));
l:= 0;
For x := box.x1 to Box.x2 do
For y := box.y1 to box.y2 do
begin;
TPA[l].x := x;
TPA[l].y := y;
inc(l);
end;

Result:= TPA;
SetLength(TPA, 0); //Just a safety measure, we set it's length to 0 to null out it's memory pointer in hopes that it works.. and it indeed does.. sometimes it works without this, sometimes not.
//If your going to use points that are 1000x700 and stuff that large, KEEP SETLENGTH(0) or else your going to lag like shit!
end;

var
T, I: Integer;
TPA: TPointArray;

begin
I:= 0;
MarkTime(T);
Repeat
TPA:= TPAFromBox2(IntToBox(0, 0, 1000, 700));
Inc(I);
Until(I > 200);
writeln(TimeFromMark(T));
end.

bg5

2012-05-29 03:56:56

Well, I have no idea what to say about c++ stuff ,because my knowledge is basic :P

Your function doesn't leak.And this also doesn't:
Function TPAFromBox2(const Box: TBox): TPointArray;
var
x, y: integer;
l : integer;
begin;
SetLength(Result, (Box.x2 - Box.x1 + 1) * (Box.y2 - Box.y1 + 1));
l := 0;
For x := box.x1 to Box.x2 do
for y := box.y1 to box.y2 do
begin;
Result[l].x := x;
Result[l].y := y;
inc(l);
end;
end;
var
T, I: Integer;
TPA: TPointArray;

begin
for i :=0 to 20 do
TPA:= TPAFromBox2(IntToBox(0, 0, 1000, 700));

end.

But if you delete 2 from TPAFromBox2 it will skyrocket! And it's exactly the same function: https://github.com/MerlijnWajer/Simba/blob/master/Units/MMLCore/tpa.pas#L1879

only difference is the 'level' of interpreter.

nielsie95

2012-05-29 04:12:10

It is a bug in Pascal Script which has to do with variable management. This bug should not occur in Lape, for example. (See the link in beginner5's signature).

Gu3st...

2012-05-29 04:14:40

Hmm… yes, it's clear that PS is the sinner once again…
Luckily we have Lape!

Use the wrapper for now if you want to avoid memory leaks

nielsie95

2012-05-29 04:17:58

Well, I'm not sure if it is a memory leak. It's more like memory is just building up and freed later on (like Brandon is saying), because this will work:


program new;

procedure test;
var
TPA : TPointArray;
begin
TPA := TPAFromBox(IntToBox(0,0,2000,2000));
end;

var
a : integer;
begin
for a := 0 to 1000 do
test();
end.

masterBB

2012-05-29 04:48:15

TPAs work with objects. When you use a non wrapper TPA function returning a TPA it will pass it as an instance/object instead of a value. This will not overwrite the previous TPA but instead replace it, making the previous TPA still exist. At the end of the function it is freed though cause it uses the functions memory spot.

bg5

2012-05-29 04:57:35

So you use wrapper ,or clean TPA manually before function?
     SetLenght(TPA,0); 
TPA := TPAFromBox(IntToBox(0,0,2000,2000));

I have no idea ,why i deleted previous post ,lol

Brandon

2012-05-29 05:00:46

From: masterBB
TPAs work with objects. When you use a non wrapper TPA function returning a TPA it will pass it as an instance/object instead of a value. This will not overwrite the previous TPA but instead replace it, making the previous TPA still exist. At the end of the function it is freed though cause it uses the functions memory spot.


But that's the thing.. it isn't freeing (most of the time) at the end it frees only when you close Simba even when the function has ended.

Will Leak because TPA still exists outside the function.

var
TPA: TPointArray;

Procedure Test;
begin
TPA:= TPAFromBox(IntToBox(0, 0, 2000, 2000));
end;

Nielsie's example (Does not leak because TPA is destroyed when the function returns (most of the time at least)):

Procedure Test;
var
TPA: TPointArray;
begin
TPA:= TPAFromBox(IntToBox(0, 0, 2000, 2000));
end;


So how are you supposed to assign such things to a global value? Well create a local var and assign it to result then 0 it out. Only way I see working for globals so far other than passing by reference.

Either use a wrapper for pass by reference way of assignment OR just free it manually with the setlength crap. If it's local then there's no need to worry, but global is a pain in the ass.

bg5

2012-05-29 05:05:31


Nielsie's example (Does not leak because TPA is destroyed when the function returns (most of the time at least)):


But if you make leak inside his function , it won't be freed even when function returns ,try little modification:

program new;

procedure test;
var
TPA : TPointArray;
begin
TPA := TPAFromBox(IntToBox(0,0,2000,2000));
TPA := TPAFromBox(IntToBox(0,0,2000,2000)); // boo
end;

var
a : integer;
begin
for a := 0 to 1000 do
test();
end.

So how are you supposed to assign such things to a global value? Well create a local var and assign it to result then 0 it out. Only way I see working for globals so far other than passing by reference.


for a:=0 to 1000 do
begin
Setlength(TPA,0); //it might left 2 bytes anyway:P (array length byte ,and array start byte?)
TPA := TPAFromBox(IntToBox(0,0,2000,2000));
end;

nielsie95

2012-05-29 05:09:34

Good point. Then it is indeed a nasty bug in Pascal Script.
Arrays are reference counted, which means they should exist as long as there is still one variable pointing at them. At the very moment the variable stops pointing, the memory should be freed.

Brandon

2012-05-29 05:16:29

So the solution is to then use the SetLength way of fixing things:


procedure test;
var
TPA : TPointArray;
begin
SetLength(TPA, 0); //Initialize/Zero out
TPA := TPAFromBox(IntToBox(0, 0, 2000, 2000));
SetLength(TPA, 0); //Before Next assignment, Initialize/Zero out
TPA := TPAFromBox(IntToBox(0, 0, 2000, 2000));
SetLength(TPA, 0); //After the TPA is finished being used, this MAY or May not be needed since it should be destroyed automatically.
end;

var
a : integer;
begin
for a := 0 to 1000 do
test();
end.

Of course the more you call SetLength, the more CPU your using :c so either you make more and more variables or just Zero length the same variable, no difference in CPU between the two methods but I'm going to stick to setlength or local vars. This type of stuff will get someone's character killed -__-

Gu3st...

2012-05-29 06:32:50

Better solution: Use Lape! :D