SRL Archive - Research & Development Lounge

Mouse movement fun #2 [Dynamic UpText search]


Flight

2012-04-08 08:46:24

Hi there,

So this concept has crossed most of our minds but it seems no one has really worked towards this, but what if we could monitor UpText while the mouse is still moving?

I've seen quite a few posts of people saying it couldn't be done because of the fact Simba is not multi-threaded? Well that may be, but we can come mighty close to getting the same results.

Many of you know the advantage of having a function that searches for UpText while the mouse is still in movement; I'm sure alot of you know where I'm going with this. On the other hand, it's not much of a gain, but should it work successfully than our mouse movements are one step closer to human, no?

Right now, every script that searches for a moving on-screen object, like an NPC, will statically move to a generated coordinate and then check/wait for an UpText or array of UpText. But what happens if that object moves position while the mouse has already been told where to move and is moving? Then we've failed to find that object while a normal human would never miss.

An example of NPC-finding:

1. [NPC is at on-screen coordinate Point(250, 175)]

Normal scripts would begin to move the mouse to this coordinate, and while until the mouse has reached the coordinate, nothing else is checked nor updated.

2. [NPC changes location to coordinate Point(150, 90)]

A dynamic UpText searching function will constantly monitor the UpText for each mouse 'step' from the origin to the destination. So if the NPC moves location but is still within the path of the mouse, the function could detect the UpText, all the way until the destination is reached. Should the UpText be found before the destination is reached then we could tell the script to do something else. Cancel the mouse-movement and left/right click perhaps?


I was tweaking around with this and I believe I have it working just how I want it now. What I do is stand beside the Castle Wars bank chest and move the mouse from, roughly, 200 "Y's" above the chest down to 200 "Y's" below the chest. If the UpText string array of "['Use', 'Bank', 'hest']" is found within that mouse path, the script will right-click. I've done everything I could think of to speed up the response time while retaining a human delay.

Here's what I've come to so far:


program New;
{$DEFINE SMART}

{$i SRL/srl.simba}


function IsUpTextReMutli(DaText: TStringArray): Boolean;
var
TheText: string;
i, n: Integer;
begin
TheText := GetUpText;
n := High(DaText);
for i := 0 to n do
if ExecRegExpr(DaText[i], TheText) then
begin
Result := True;
Exit;
end;
end;

procedure FastClick(button: variant);
var
a,b,c : integer;
iButton: Integer;
begin
if (button) then
iButton := mouse_left
else
iButton := mouse_right;

GetMousePos(b, c);

HoldMouse(b, c, iButton);

for a := 0 to 2 do
Wait(20 + Random(30));

GetMousePos(b, c);
ReleaseMouse(b, c, iButton);
end;

{
-Spline by BenLand100
-Shift speed every mouse step
-Search for an uptext string array while the mouse is in movement
-Left or right click if uptext is found
}
Function DynamicWindMouse(xs, ys, xe, ye, gravity, wind, minWait, maxWait, targetArea: extended;
Text: TStringArray; LClick: Boolean): Boolean;
var
veloX,veloY,windX,windY,veloMag,dist,randomDist,lastDist,step: extended;
lastX,lastY,MSP,W,maxStep: integer;
sqrt2,sqrt3,sqrt5: extended;
begin
MSP := MouseSpeed;
sqrt2:= sqrt(2);
sqrt3:= sqrt(3);
sqrt5:= sqrt(5);

repeat

dist:= hypot(xs - xe, ys - ye);
wind:= minE(wind, dist);
maxStep := (RandomRange(7, 11)); //I've decreased the maxStep so we have more steps along our path
//This means a higher chance of finding our UpText

if dist >= targetArea then
begin
windX:= windX / sqrt3 + (random(round(wind) * 2 + 1) - wind) / sqrt5;
windY:= windY / sqrt3 + (random(round(wind) * 2 + 1) - wind) / sqrt5;
end else
begin
windX:= windX / sqrt2;
windY:= windY / sqrt2;
end;

veloX:= veloX + windX;
veloY:= veloY + windY;
veloX:= veloX + gravity * (xe - xs) / dist;
veloY:= veloY + gravity * (ye - ys) / dist;

if hypot(veloX, veloY) > maxStep then
begin
randomDist:= maxStep / 2.0 + random(round(maxStep) div 2);
veloMag:= sqrt(veloX * veloX + veloY * veloY);
veloX:= (veloX / veloMag) * randomDist;
veloY:= (veloY / veloMag) * randomDist;
end;

lastX:= Round(xs);
lastY:= Round(ys);
xs:= xs + veloX;
ys:= ys + veloY;

if (lastX <> Round(xs)) or (lastY <> Round(ys)) then
MoveMouse(Round(xs), Round(ys));

if (Length(Text) > 0) then //Only if we're searching for text
begin
if IsUpTextReMutli(Text) then
begin
FastClick(LClick);
Result := True;
Exit;
end;

case Random(50) of
1..25: W := (MSP + (Random((MSP/4))));
26..50: W := (MSP - (RandomRange((MSP/2), MSP-1)));
end;
if (W < 1) then
W := 1;

wait(W);
end else
begin
case Random(50) of
1..25: W := (MSP + (Random((MSP/4))));
26..50: W := (MSP - (RandomRange((MSP/2), MSP-1)));
end;
if (W < 1) then
W := 1;

wait(W);
end;

step:= hypot(xs - lastX, ys - lastY);
lastdist:= dist;
until(hypot(xs - xe, ys - ye) < 1)

if (Round(xe) <> Round(xs)) or (Round(ye) <> Round(ys)) then
MoveMouse(Round(xe), Round(ye));

MouseSpeed := MSP;
end;

Procedure HumanMMouse(eX, eY, ranX, ranY: Integer; Text: TStringArray; LClick: Boolean);
var
randSpeed: extended;
X,Y,X2,Y2,A,Dist,MP: integer;
begin
A := MouseSpeed;
GetMousePos(X, Y);
Dist := Distance(X, Y, eX, eY);
MP := Round(Dist/150);
if MP < 0 then
MP := 1;

randSpeed := (random(MouseSpeed) / 2.0 + MouseSpeed) / 10.0;
X2 := RandomRange(eX-(A*MP), eX+(A*MP));
Y2 := RandomRange(eY-(A*MP), eY+(A*MP));

if DynamicWindMouse(X, Y, X2, Y2, 9, 5, 10.0 / randSpeed, 15.0 / randSpeed, 10.0 * randSpeed, Text, LClick) then
Exit;

GetMousePos(X, Y);
MMouse(eX, eY, ranX, ranY);
MouseSpeed := A;
end;

begin
Smart_Server := 86;
Smart_Members := True;
Smart_Signed := True;
Smart_SuperDetail := False;

setupSRL();
ClickNorth(SRL_Angle_High);

MouseSpeed := 20; //I like 15-25 (My formula is actually backwards, high # = slower)

MMouse(300, 120, 5, 15); //Move the mouse above the chest
Wait(RandomRange(1000, 1400));
HumanMMouse(300, 300, 5, 15, ['Use', 'Bank', 'hest'], False); //Anywhere below the chest
end.

If you'd like to see it for yourself, stand beside the Castle Wars bank chest and run the script. Run it first as is, then change line 175 to this (below) and run it a second time:

HumanMMouse(300, 300, 5, 15, [], False);
The empty array means we don't search for text, and this would mimic the normal mouse-movement.

Anywho, I'm guessing most people would regard this as "not worth it", which is fine, but if anyone is interested in the idea please do share some comments and suggestions. You understand the concept here, it's not just limited to checking UpText, we can constantly search for colors at a certain coordinate while the mouse is moving as well; anything we want.

Ashaman88

2012-04-08 09:22:04

This looks pretty cool, will def have to check it out. Great job flight!

Sir Ducksworthy

2012-04-08 14:29:22

This is completely awesome flight,
Their is unlimitless potential for functions like this, the potential between a ban and being in the clear can come down to this exact content in a script and I know that one day soon this type of content is going to mean everything to people in the botting game, Color Bots statistically speaking are not going to be stopped effectively any other way then more advanced coordinate tracking and overall account mouse speed/movement path averaging.

So that's why we need to keep this kind of stuff private and away from the general public so if that day comes only people skilled enough to avoid the detection will be avoiding the flag while everyone else will be getting flagged and taking the heat.

Ironically it would be just like what's happening now with Color bot's compared to injection bot's but at the next stage.

Shatterhand

2012-04-08 14:31:52

Looks interesting!

Kyle Undefined

2012-04-08 14:46:57

Oh this is interesting!

Harry

2012-04-08 14:50:54

The problem with this is the fact that checking uptext takes quite a few ms. This will probably cause the mouse to "stutter" a bit, especially on more laggy systems. I just looked through it quick, not sure if you have a solution for that or not? I don't think that can be truly done without multithreading.

Olly

2012-04-08 15:40:45

Looks promising looks like the mouse could be slightly laggy though?

eska

2012-04-08 16:03:38

From: Flight

A dynamic UpText searching function will constantly monitor the UpText for each mouse 'step' from the origin to the destination.


If I understand correctly, and from looking at your code I think I did, you move the mouse for a "step", then check the uptext, then move again, then uptext.. until you reach destination OR find the uptext.

Doesn't that mean that at every "step" the mouse will stop moving for a few ms? If not, awesome, else, wouldn't that be very bot like?

To me it seems like there's no real way to fix this problem without multi threading :s

Abu

2012-04-08 16:09:46

From: sir eska eau

Doesn't that mean that at every "step" the mouse will stop moving for a few ms? If not, awesome, else, wouldn't that be very bot like?


I don't think it will be bot like - it's only a few milliseconds…

OT: I don't even think that Jagex monitor mouse movements this accurately, but I guess it would be useful in terms of finding moving NPC's

Markus

2012-04-08 16:44:15

From: Harry
The problem with this is the fact that checking uptext takes quite a few ms. This will probably cause the mouse to "stutter" a bit, especially on more laggy systems. I just looked through it quick, not sure if you have a solution for that or not? I don't think that can be truly done without multithreading.


That's the main reason we never did this before, uptext checking was way too slow! However, since the first versions of Divi we already got < 50 ms uptext, which is good enough for fluid mouse-movements. Wrote a real quick demo to test this back then (2007), worked fine! but it got buried somewhere on my hard drive unused forever.

[/historical-rambling]

On-topic: nice job! Maybe this can be combined with FindObject in SRL?

Flight

2012-04-08 21:11:27

From: Harry
The problem with this is the fact that checking uptext takes quite a few ms. This will probably cause the mouse to "stutter" a bit, especially on more laggy systems. I just looked through it quick, not sure if you have a solution for that or not? I don't think that can be truly done without multithreading.


Yes that's what I thought at first, and to truly experiment with the speed I left the (wait) in between each mouse step even while checking for UpText; the results were no slower than normally moving the mouse. I know it would seem to slow the mouse down checking an array of UpText after every step, but my results state otherwise.

To make increase the chance of the UpText being found along the mouse path I decreased the maxStep. So there's more movements between where we started and the destination coordinate. Obviously lower maxSteps = slower mouse speed, but this can also be combated with less (wait) time in between each step. I believe I have the settings just right to where a difference won't be noticed.

I do invite you to try this yourself, Harry. I'm hoping if you see the smooth, fast movement your opinion will change. ;)

Harry

2012-04-08 21:36:08

I don't have any opinions on it, was just inquiring if that's a problem in your testing. If it doesn't cause problems, that's neat.

Heysus

2012-04-08 22:42:11

A couple years ago I made a function similar to this. It would actually 'follow' an npc.

It would first search the entire screen for the npc. Then as the mouse was moving it would search a small area around the target mouse coordinates and constantly update the coordinates that the mouse was moving to.

It didn't seem to lag at all and the search was very fast because it only searched a small (100x100?) area.

I can't seem to find the script but I remember it working very well. It improved clicking accuracy by a lot.

Edit: I found the script. I'm trying to decipher the mess and comment it a bit then i'll post it.

Flight

2012-04-08 23:07:09

From: Heysus
A couple years ago I made a function similar to this. It would actually 'follow' an npc.

It would first search the entire screen for the npc. Then as the mouse was moving it would search a small area around the target mouse coordinates and constantly update the coordinates that the mouse was moving to.

It didn't seem to lag at all and the search was very fast because it only searched a small (100x100?) area.

I can't seem to find the script but I remember it working very well. It improved clicking accuracy by a lot.

Edit: I found the script. I'm trying to decipher the mess and comment it a bit then i'll post it.


Yep yep, I was thinking along the same lines of constantly monitoring colors in a certain area to make sure the object we're searching for remains at the same, original coordinate. Should the color not remain where we originally found it then break our current mouse movement and restart the original search. Good idea for sure, Heysus. :)

Heysus

2012-04-08 23:33:52

Ok, here's the function.

After reading it maybe I can explain it a little better.

It moves the mouse toward the npc
As the mouse moves (and the npc moves) it constantly updates the target coordinates of the mouse.
When the mouse is close to the npc, it checks for the uptext color (faster than checking for uptext)
After three consecutive loops of positive uptext color checks, it will click the npc.

It's kind of complicated and messy but it was a pretty awesome function and improved clicking accuracy a lot.

Edit: Also the mouse function already has a wait built into it. So in order to reduce lag, I timed how long my extra functions took and subtracted it from the mouse wait.


Function ClickNPCMS: boolean;
var
x, y, i, t, UpText: integer;
a, d: extended;
Cols: TIntegerArray;
TP, LastTP: TPoint;
TPA, TempTPA: TPointArray;
ATPA: T2DPointArray;

xs, ys, xe, ye, gravity, wind, minWait, maxWait, maxStep, targetArea: extended;
veloX, veloY, windX, windY, veloMag, dist, randomDist, lastDist, step: extended;
randSpeed: Extended;
lastX, lastY: integer;
sqrt2, sqrt3, sqrt5: extended;
begin
// mouse movement parts are copied from mouse.simba

// setup mouse
GetMousePos(x, y);
xs := x;
ys := y;
Gravity := 9.0;
Wind := 3.0;
randSpeed:= (random(MouseSpeed) / 2.0 + MouseSpeed) / 10.0;
minWait := 10.0/randSpeed;
maxWait := 15.0/randSpeed;
maxStep := 10.0*randSpeed;
targetArea := 10.0*randSpeed;

sqrt2:= sqrt(2);
sqrt3:= sqrt(3);
sqrt5:= sqrt(5);
//////////////////

Cols := [3687553, 2436436];

MarkTime(t);
While (TimeFromMark(t) < 5000) do //follow npc for 5 seconds before giving up
begin

GetMousePos(x, y);
if Distance(x, y, Round(xe), Round(ye)) < 65 then //if we are close to npc, check for uptext and click

//check for uptext color (faster than using uptext) and click if we are on the npc
if FindColorTolerance(x, y, 121823, MSX1, MSY1, MSX1 + 150, MSY1 + 25, 45) then
begin
if UpText = 2 then //click on the third consecutive loop of positive uptext find (this helped improve clicking accuracy)
begin
GetMousePos(x, y);
Mouse(x, y, 0, 0, true);

if DidClick(true, RandomRange(400, 500)) then
begin
Result := true;
MouseAntiBan;
exit;
end;

//reset some variables and try to click again if we missed
veloX:= 0;
veloY:= 0;
windX:= 0;
windY:= 0;
Gravity := 9.0;
Wind := 3.0;
randSpeed:= (random(MouseSpeed) / 2.0 + MouseSpeed) / 10.0;
minWait := 10.0/randSpeed;
maxWait := 15.0/randSpeed;
maxStep := 10.0*randSpeed;
targetArea := 10.0*randSpeed;

end else
Inc(UpText);
end else
UpText := 0;

//search for NPC
SetArrayLength(TPA, 0);
SetArrayLength(ATPA, 0);

for i:=0 to High(Cols) do
begin
if (LastTP.x = 0) and (LastTP.y = 0) then //search the whole screen in the first loop
FindColorsSpiralTolerance(MSCX, MSCY, TempTPA, Cols[i], MSX1, MSY1, MSX2, MSY2, 8)
else //otherwise search a 100x140 area
FindColorsSpiralTolerance(LastTP.x, LastTP.y, TempTPA, Cols[i], Max(MSX1, Round(LastTP.x) - 50), Max(MSY1, Round(LastTP.y) - 70), Min(MSX2, Round(LastTP.x) + 50), Min(MSY2, Round(LastTP.y) + 70), 10);

TPA := CombineTPA(TPA, TempTPA);
end;

if GetArrayLength(TPA) = 0 then
exit;

ATPA := TPAToATPA(TPA, 25);


if (LastTP.x = 0) and (LastTP.y = 0) then
SortATPAFrom(ATPA, Point(MSCX, MSCY))
else
SortATPAFrom(ATPA, LastTP); //sort from last npc point

for i:=0 to High(ATPA) do
begin
if GetArrayLength(ATPA[i]) < 25 then
continue;

TP := MiddleTPA(ATPA[i]);
if FindColor(x, y, 10496, Max(MSX1, TP.x - 20), Max(MSY1, TP.y - 70), Min(MSX2, TP.x + 20), Min(MSY2, TP.y - 5)) then
continue;
if FindColor(x, y, 198716, Max(MSX1, TP.x - 20), Max(MSY1, TP.y - 70), Min(MSX2, TP.x + 20), Min(MSY2, TP.y - 5)) then
continue;

break;
end;

if (LastTP.x = 0) and (LastTP.y = 0) then //first loop has no last npc coordinate so we set it to current npc position
LastTP := TP;

//not sure what this is, i think it calculates the next mouse 'step'
a := ArcTan2(TP.y - LastTP.x, TP.x - LastTP.y);
d := hypot(LastTP.x - TP.x, LastTP.y - TP.y)*0; //multiply by 0 (wtf??)
xe := TP.x + cos(a)*d;
ye := TP.y - sin(a)*d;
LastTP := TP;


// move mouse
if hypot(xs - xe, ys - ye) > 3 then
begin
dist:= hypot(xs - xe, ys - ye);
wind:= minE(wind, dist);
if dist >= targetArea then
begin
windX:= windX / sqrt3 + (random(round(wind) * 2 + 1) - wind) / sqrt5;
windY:= windY / sqrt3 + (random(round(wind) * 2 + 1) - wind) / sqrt5;
end else
begin
windX:= windX / sqrt2;
windY:= windY / sqrt2;
if (maxStep < 3) then
begin
maxStep:= random(3) + 3.0;
end else
begin
maxStep:= maxStep / sqrt5;
end;
end;
veloX:= veloX + windX;
veloY:= veloY + windY;
veloX:= veloX + gravity * (xe - xs) / dist;
veloY:= veloY + gravity * (ye - ys) / dist;
if hypot(veloX, veloY) > maxStep then
begin
randomDist:= maxStep / 2.0 + random(round(maxStep) / 2);
veloMag:= sqrt(veloX * veloX + veloY * veloY);
veloX:= (veloX / veloMag) * randomDist;
veloY:= (veloY / veloMag) * randomDist;
end;
lastX:= Round(xs);
lastY:= Round(ys);
xs:= xs + veloX;
ys:= ys + veloY;
if (lastX <> Round(xs)) or (lastY <> Round(ys)) then
MoveMouse(Round(xs), Round(ys));

step:= hypot(xs - lastX, ys - lastY);
wait(round((maxWait - minWait) * (step / maxStep) + minWait));
lastdist:= dist;
end;
end;
end;

Flight

2012-04-09 00:46:50

Ha wow, that's a lot to do between each step. :p

vashanddou

2012-04-09 01:23:28

EDIT: I see now you talked about something similair up above but i'll leave this here still.


Im not a very advanced scripter, just finishing up my first prayer script (pretty sure it runs 100% flawless I added a bit more variables to it before it ran for 6 hours sometimes, now should run untill client token every time)

anyways
Would it require multi-threading to have it check the positions where it is automatically set to go to before even doing the actions of the position.

For example:
The item is a bank chest at 200, 200
it checks for the colour of the bank chest at 200, 200 before automatically moving the mouse there, if it is not found it will auotmatically start searching for it and completly cancel out moving the mouse to that position. but if it was found at 200, 200 it will move there and go to the chest and skip the finding it somewhere else part.

Coh3n

2012-04-09 01:26:25

Looks neat. On a completely unrelated topic, why are all your functions indented from the margin?

E: This is the kind of thing I'd like to see in SRL6. :)

Flight

2012-04-09 02:43:20

From: Coh3n
Looks neat. On a completely unrelated topic, why are all your functions indented from the margin?

E: This is the kind of thing I'd like to see in SRL6. :)


Sweet. And to answer your question, it's just my own style. I've been doing that since the start and it's stuck as a habit for me. :p

Also if you'd like, I've made a practical use out of it, or atleast in a similar manner. This thread (http://villavu.com/forum/showthread.php?t=79279) shows how monitoring the screen for a DTM (or colors/bitmaps) can also be used. That should successfully scroll through your entire bank until a certain item (by DTM) is found.

Imagine

2012-04-10 21:35:09

This is really interesting, I'll try to see if I can use it in a few of my personal scripts :)

NCDS

2012-04-10 22:04:36

I like it, but I would prefer HumanMMouse take a pointer rather than a string array. Other than that, assuming everything runs smooth, I like the concept. :)

Flight

2012-04-10 23:46:07

From: NCDS
I like it, but I would prefer HumanMMouse take a pointer rather than a string array. Other than that, assuming everything runs smooth, I like the concept. :)


Thanks NCDS. :) Also, I'm not quite sure what you mean by 'take a pointer'.

Kyle Undefined

2012-04-11 00:06:11

PS can't use pointers, however, Lape can.

Taken from here, http://villavu.com/forum/showthread.php?t=68613:

var
a: Integer;
b: ^Integer;
begin
a := 2;
b := @a;
b^ := 3;
WriteLn(a);
end;

Neros

2012-04-11 00:12:47

hmm this looks rather interesting, i keep reading little bits and pieces like this, i wanna get a good knowledge of simba's potential before i actually learn to code, because if i start learning to script at a more advanced level, it "should" take me less time to get to grips with it, probably increase the quality of work too

it's a good piece of work flight, im assuming this would go down well in an autofighter :P