SRL Archive - Senior Scripts and Snippets

Item Scanning (Instant Update)


Narcle

2011-11-07 00:12:55

Wasn't sure where to put this, but I'll throw this idea out to the SSRL+ first.

This is an idea spin off from blacklists that Niels made awhile ago. I used his base and created my own that it still used today in my scripts. And I even use it to update on the spot.

I was wondering if this idea could be expanded. Perhaps auto (D)DTMs or the like above. Then these can be placed in a file or the like for their database. I believe its similar to what RiD does now.

I have a feeling this will be required soon if Jagex wants to 'fight back'.

Thoughts?

BraK

2011-11-07 00:23:42

Sounds like an interesting Idea. Got code?

YoHoJo

2011-11-07 00:24:36

So you want to make an include/database of DTMs/Bitmaps/DDTMs whatever works best for all to use? How would it auto update? Like during runtime? Or people would need to update the DTMs in the library manually IF the items were to change?
Whatever it is I like it.

Coh3n

2011-11-07 01:40:25

I'm interested.

Narcle

2011-11-07 02:03:17

The way I do it currently relies on the user to place items in correct spot(s) to be scanned. Black list is easy but DTM… no clue lol.

This is what I currently use:

Type
TTPAItem = Record
OutLine, Width, Height, Color: Integer;
end;

function TPAItem(cOutLine, cWidth, cHeight, cColor: Integer): TTPAItem;
begin
with Result do
begin
OutLine := cOutLine;
Width := cWidth;
Height := cHeight;
Color := cColor;
end;
end;

function FindTPAItem(Item: TTPAItem; B: TBox): Boolean;
var
TPA: TPointArray;
Bt: TBox;
x,y: integer;
begin
FindColorsTolerance(TPA, srl_outline_black, B.x1, B.y1, B.x2, B.y2, 0);
if Length(TPA) < 1 then
Exit;
Bt := GetTPABounds(TPA);
if (Length(TPA) = Item.OutLine) and (iAbs(Bt.x2-Bt.x1) = Item.Width) and (iAbs(Bt.y2-Bt.y1) = Item.Height) then
if Item.Color > -1 then
Result := FindColorTolerance(x, y, Item.Color, Bt.x1, Bt.y1, Bt.x2, Bt.y2, 6)
else
Result := True;
end;

Function UpdateItem(var Item: TTPAItem; InvN: integer): Boolean;
var
TPA: TPointArray;
T: TPoint;
B: TBox;
begin
if (Item.OutLine > 0) then
Exit;
Wait(300+random(200));
B := InvBox(InvN);
FindColorsTolerance(TPA, srl_outline_black, B.x1, B.y1, B.x2, B.y2, 0);
if Length(TPA) < 1 then
Exit;
B := GetTPABounds(TPA);
T := MiddleTPA(TPA);
Item := TPAItem(Length(TPA), iAbs(B.x2-B.x1), iAbs(B.y2-B.y1), GetColor(T.x, T.y));
Result := true;
Writeln('Item := TPAItem('+inttostr(Item.Outline) +', '+inttostr(Item.Width)+', '+inttostr(Item.Height)+', '+inttostr(Item.Color)+');');
Exit;
end;

function iCountArr(Item: TTPAItem): TIntegerArray;
var
i, n: integer;
begin
SetArrayLength(Result, 28);
for i := 1 to 28 do
if FindTPAItem(Item, InvBox(i)) then
begin
Result[n] := i;
Inc(n);
end;
SetArrayLength(Result, n);
end;

function iCount(Item: TTPAItem): integer;
begin
Result := Length(iCountArr(Item));
end;

Currently I have it called first time when a load is done. However if we use it as a scanner we could read uptexts then scan and save the info to a file for future use.

Sadly I can't think of any good way to go about updating without the user putting in some effort.

Edit:
Hell this would be useful for the Dev as a auto updater we can run. Just keep a account with the items to scan and it scans them then updates to a online database or w/e.

Coh3n

2011-11-07 02:11:04

Looks nice. Theoretically, if you have the width, height, outline, and colour we should be able to ID any item fairly easily. It's just IDing in the bank or with numbers above would be the issue.

I think it would be worth working on Nielsie's blacklist system since it's done. It may just need tweaking.

http://villavu.com/forum/showpost.php?p=803737&postcount=107

Nava2

2011-11-07 02:12:03

Uhm, what about just having scripts "callibrate" at the beginning.

Sending information to a remote server is the easy part.. working this into scripts is harder. :(

Narcle

2011-11-07 02:19:33

From: Coh3n
Looks nice. Theoretically, if you have the width, height, outline, and colour we should be able to ID any item fairly easily. It's just IDing in the bank or with numbers above would be the issue.

I think it would be worth working on Nielsie's blacklist system since it's done. It may just need tweaking.

http://villavu.com/forum/showpost.php?p=803737&postcount=107

Yeah the Bank items I'm not sure on… DTMs are the best.

From: Nava2
Uhm, what about just having scripts "callibrate" at the beginning.

Sending information to a remote server is the easy part.. working this into scripts is harder. :(


That's basically what I do is calibrate at the start.



I had a thought while responding. What if you do a Blacklist scan then using that make a DTM usable for the bank… Just have to get a good formula for picking the points for the DTM.

nielsie95

2011-11-07 02:21:41

The script I posted with my blacklist implementation actually created a new blacklist for every item that it didn't recognize (using its uptext to identify the item).

WT-Fakawi

2011-11-07 05:36:54

HLR already does this. Works 100%. I will write a short paradigm tonight.

Narcle

2011-11-07 05:44:52

From: WT-Fakawi
HLR already does this. Works 100%. I will write a short paradigm tonight.


Ahh, figured someone else would do this. :)

WT-Fakawi

2011-11-07 05:53:52

Mine's crappy and unsuited for release in SRL, but the principle is brilliant. Try searching for blacklist on the forum. You will find all you need. Also there was a working example by ray…which I failed to implement (me = stupid)

WT-Fakawi

2011-11-07 17:36:15

Because I wanted it soo badly, I wrote it myself. (Because I am to nooberish to comprehend any other code but my own). Just look five seconds at my HLR video and you will instantly understand why this is what we have needed for years. And what was written already by Niels, and to a certain agree before him by masquerader (amount still works?) and probably by many others before them.

The general paradigm is simple:

being able to runtime refer to objects with their 'name'!


Advantage: i.e. straight from HLR: Cook('Raw'). Cooks any kind of food it grabs. Because all food is 'Raw'. Still impressed it works with any kind of food.
Also from HLR: burn('log'). Mind you: I do not know what a 'log' is until I analyse it (and add it to the database) runtime it in my inventory!

Never, ever having to make a dtm, or a bitmap for any object anymore. Just a name, thats all. What a relief that would be!


Philosophy.

  1. The Item Database is kept locally, on every users's computer. (Can be skipped, but you loose the database when script stops.)
  2. The Item Database is updated (or when skipped 1. created) runtime.
  3. Inventory/Bank/Equip is constantly monitored, scanned and database is accordingly updated as part of FindNormalRandoms!

This would be my preferred approach. You choose if you store the database locally on your computer or in memory (or both). I would really avoid "Central Servers" with a "Dynamically Regenerated ItemBase" etc for legal purposes. Nice for internettraffic, but no thank you. Just a plain text file with a string, and a couple of numbers, thats all we need really. Nothing copyrighted.


Implementation.

I am sorry, I am not an expert in that area. I have not been able to grasp Nielsie's code. I have asked for a solution in this thread. Ray solved it in the last post. But alas, I was unable to comprehend this, so I partly reinvented the wheel myself.
I used:
  1. Object width and height. (Filters 80%)
  2. Number of outlined pixels, better known as or blacklist. (Filters 95%)
  3. For the remaining 5% I use one stupid pixel-color-integer no other object shared in my DB. (More precise: the sum of 25 pixels filtered in a strict grid from the background noise ) Super stupid, but superfast. One integer, ghehe, works 100%. Should of course be replaced by a sophisicated sum of blah, or w/e you prefer. Also Bank and Equip can be scanned if preferred (with or without mouse scanning (depending on the "needs" of the database engine: is the outline familiair? is the checksum ok?)

loo tong;ridn't deed: Make it so!


Functionality.

This is what I have come up with in HLR:

-finding-

Function CountInvObj(name:String): Integer; // Returns sum of name.
Function CountAnyNamedInvObj(name:String): Integer; // Returns sum of any instance of name.
Function FindInvObj(var slot: Integer; name: String): Boolean; // Finds name and returns slotnumber and boolean.
Function FindAnyNamedInvObj(var Slot: Integer; name:String): Boolean;// Finds any instance of name and returns slotnumber and boolean.

-clicking-

Function ClickInvObj(name: String; Left:Boolean): Boolean; // clicks name, Left or Right.
Function ClickAnyNamedInvObj(name: String; Left:Boolean): Boolean; // clicks any instance of name, Left or Right.

-misc-

Function Name2DBase(Name: String):InvItem; // returns Type InvItem from name.
Function RepairInv:Boolean; // compares and fixes the current inventory to the existing database: Error check (bones and white areas distorting UpText analysis)

You will probably come up with or already have come up with more FindInvObjEx-functions.

Can you all agree on this being a nice starting point? I would definately go for the "simple" approach, nothing too fancy maybe. All code has been written more then once before here. Argh… Just make it work!


WT.

Coh3n

2011-11-09 02:02:03

The concept is brilliant in my opinion. Making it work with every item… that's another story. I'm confident it can be done, but I think a path should be chosen and stuck to. Not having 8 people writing their own version. It probably a big project, so hosting on github may not be a bad idea.

Fakawi, I agree about keeping it simple. Often the simplest approaches work the best.

WT-Fakawi

2011-11-09 08:42:51

In Camo Developer? Looks he is busy with this too.

Nava2

2011-11-09 11:12:57

I like Nielsie's "LoadFrom/StoreString" approach. This makes storing in files/databases very easy. :)

Kyle Undefined

2011-12-18 04:18:45

Small bump. I'm actually still interested in this, what would need to be done to get this working 100%?

I really think this is the way to go.

Gu3st...

2011-12-18 07:55:29

Yes, we should absolutely focus more on this idea

The only problem I can think of (That WT-F already addressed) is the database storing problem. If we store it online then JaGeX might find grounds to sue (Or could they? Fan-made websites are storing it online too, right?), and if we store it locally, people with problems are going to post their database on the forum to help others, which in turn, is (almost) equal to storing databases online…

Maybe we could utilize some of the already existent fan pages in some way?

Sir R. M8gic1an

2011-12-18 08:13:28

From: Zyt3x
Yes, we should absolutely focus more on this idea

The only problem I can think of (That WT-F already addressed) is the database storing problem. If we store it online then JaGeX might find grounds to sue (Or could they? Fan-made websites are storing it online too, right?), and if we store it locally, people with problems are going to post their database on the forum to help others, which in turn, is (almost) equal to storing databases online…

Maybe we could utilize some of the already existent fan pages in some way?


Jagex sue? all we're storing are points and colours….

well maybe but surely it would look nothing like their own database?

Also I see no real reason for people to post their entire DBs. After all, if one had an issue we all would as our item info would be created through the same algorithm

-RM

WT-Fakawi

2011-12-18 11:08:11

The Db should be as simple as:

  • Object name
  • Object Height, Object Width
  • Object outline count
  • Object colors

In a textfile like this:

'Rune Hatchet', 28, 32, 108, {2345066,24355056, 80594903, 7060493}
'Willow log', 26, 26, 78, {26584,59349329, 90923439, 37231279}

So for every object you have once investigated, you will have one line in your textfile. The format of the textfile corresponds to the itemdatabase, so we can search through the file and read it into our script at runtime. New objects found at runtime will be added to the textfile. We add the toplevel function UpdateItemDatabase to FindRandoms.

Thats all there is to it really.

Also in many cases, the object width and height alone are enough to distinguish your objects…. so you dont need to re-scan them.

NCDS

2011-12-18 12:08:30

We should really just convert this idea into a new bitmaps.simba as the current one is never up to date with everything.

Kyle Undefined

2011-12-18 19:11:49

Why a text file? Couldn't we incorporate XML into Simba? That would make it A LOT easier to get data from the file and also to update it when it become outdated.

I like NCDS's idea, we would need to get an initial list of blacklists and all the data needed. My tool that WT linked to (I got noticed by WT :D) can do a lot of this. I could just take out the silly stuff like seeing if the item is stackable and equipable since that really wouldn't matter.

WT-Fakawi

2011-12-19 01:16:02

I noticed with HLR that it really doesnt matter if a object all-of-a-sudden changes shape, like a stack of coins f.i. The last scanned shape will be "coins"

In my world you would start with an empty db. This would grow over time, 100% suited for your own needs. So every simba-user has his own private db.

Narcle

2011-12-19 01:19:43

From: WT-Fakawi
I noticed with HLR that it really doesnt matter if a object all-of-a-sudden changes shape, like a stack of coins f.i. The last scanned shape will be "coins"

In my world you would start with an empty db. This would grow over time, 100% suited for your own needs. So every simba-user has his own private db.


This is what I was thinking more of. Then the scripters can pull from that database and put which items they need to scan or w/e.

Kyle Undefined

2011-12-19 02:29:28

Ah ok, I understand now. That makes sense. So basically when the script first starts up, it would do some setup to get the blacklists for the objects needed. Then later on, the script will just call the stored blacklist, and update it if needed.

Home

2011-12-19 15:52:27

From: Kyle Undefined
Ah ok, I understand now. That makes sense. So basically when the script first starts up, it would do some setup to get the blacklists for the objects needed. Then later on, the script will just call the stored blacklist, and update it if needed.


Tomorrow you will see ;)


~Home

Gu3st...

2011-12-19 16:26:59

From: Home
Tomorrow you will see ;)


~Home
http://villavu.com/forum/showthread.php?t=69526

Good going!

Kyle Undefined

2011-12-19 16:38:11

That's only beginning, with Home's system and a blacklist creator, we can have VERY good item detection.

Gu3st...

2011-12-19 19:12:21

From: Kyle Undefined
That's only beginning, with Home's system and a blacklist creator, we can have VERY good item detection.
True, true… If we combine all our powers, we'll be very hard to break.

WT-Fakawi

2012-01-11 08:49:55

What is the status on this project? I would love to see this implemented in SRL:staycool:

Kyle Undefined

2012-01-11 09:20:17

We have a project called "RBOSS", it's taking these methods and it's progressing nicely. I haven't had time to do much development for it though. It's kind of dropped iirc.

Gu3st...

2012-01-11 17:17:24

Put on hold, not dropped ;)

Kyle Undefined

2012-01-11 17:21:26

Meh, that's what I meant, sorry.

What's caused the hold up?

Gu3st...

2012-01-11 17:30:27

Well, I'm working on SAIL, I don't know what the others are working on

Kyle Undefined

2012-01-11 17:31:19

Ah ok, I'll get back into this :)

Narcle

2012-01-11 17:40:42

Been focusing on SRL updates / scripts.

WT-Fakawi

2012-01-11 17:46:45

Today ran into the problem that "lamp" actually equals "5 yanillan seeds". I hear it also resembles "feathers". Had to stop and adjust "lamp" so it not messed up my player any further :)

Seriously, with this new system, all we need to know is……"lamp". Never a dtm or a bitmap…ever.

Home

2012-01-13 09:21:48

I'v been busy with life and stuff mostly :/

I will catch up with this project soon as possible :)

~Home

WT-Fakawi

2012-02-01 10:14:14

Bump. Almost all bitmaps have changed. All Bitmap script, all DTM scripts…all broken. HLR is still working 100% (albeit no uptext today).

I rest my case.

Coh3n

2012-02-01 14:18:57

From: WT-Fakawi
Bump. Almost all bitmaps have changed. All Bitmap script, all DTM scripts…all broken. HLR is still working 100% (albeit no uptext today).

I rest my case.
Where did you get that from? MSI is working fine for me, and from stats it looks like my Draynor script is working fine as well.

WT-Fakawi

2012-02-26 10:35:52

From: Coh3n
Where did you get that from? MSI is working fine for me, and from stats it looks like my Draynor script is working fine as well.
I ment everyone's script that relied on bitmaps/DTM's . Wasn't referring to SRL, sry for being unclear.

Anyway: I solidified my blacklist system for the Mossgiant fighter. I decided to countercheck my objects to a predefined list of objects.



KeepList := ['Uncut diamond','Grimy guam','oins','sword','spear','shield', // any occurence of this list is picked up.
'helm','javelin', 'staff', 'dart','bolts','teel arrow', 'ragonstone',
'warf','napdragon','Dwarf weed seed', 'anarr','oadflax', 'antadym','orstol',
'rune','charm', 'Spinach roll', 'Long bone', 'key', 'bones']; // BONES IS LAST IN LIST so we pickup anything before bones :))))

MeltList := ['Rune javelin', 'Black full helm','Black sq shield','Steel helm','Steel kiteshield',
'Magic staff','Mithril spear', 'Mithril sword','Mithril spear']; // this we melt.

UseLessList:= ['Air rune','Iron arrow','Grimy','guam', 'Strawberry seed'];

RefList := ['Fire rune', 'Nature rune', 'Law rune', 'Air rune', 'Earth rune', // runes
'Cosmic rune ', 'Chaos rune', 'Blood rune', 'Death rune',
'Gold charm' , 'Green charm', 'Crimson charm', 'Blue charm', // charms
'Black full helm','Black sq shield','Steel helm','Steel kiteshield', // armour
'Magic staff','Mithril spear', 'Mithril sword','Mithril spear', 'Rune javelin',
'Ranarr seed', 'Toadflax seed', 'Snapdragon seed', 'Dwarf weed seed', // seeds
'Lantadyme seed', 'Loop half of a key',
'Uncut dragonstone', 'Uncut diamond', // gems
'Big bones', 'Long bone','Grimy guam', 'Clean guam', // bones herbs
'Feather', 'Bait', 'Salmon', 'Trout', 'Raw salmon', 'Raw trout', // fish
'Pike', 'Herring', 'Raw pike', 'Raw herring', 'Spinach roll',
'Steel arrow', 'Dragon spear', 'Rune spear'];

First list is what I pickup through GetChooseOptions, second list is what I High Alch, third is what I remove from my inv (useless), and last is my Reference List. This new system has been running over 100 hours now. I stresstested with unknnown objects, and depending on my action they where either dropped or welcomed to the list.


tl; dr: Script new style: no more bitmaps/dtms for objects, Objects are known at script startup!

Coh3n

2012-02-26 15:44:27

I know what you meant. MSI relies heavily on DTMs for all item finding, and it never broke. :confused: Although, an alternative, more stable system would be nice.

BraK

2012-02-27 10:21:20

I don't think I released down on SSRL what was going on with me. I'm moving back to the states and my wife is currently pregnant this is what has me tied up at the moment. I'll probably start working on RBOSS again this summer.

Kyle Undefined

2012-02-27 10:55:08

Hope the move goes successful and the pregnancy smooth! RBOSS is next on my todo list :)

Home

2012-02-27 11:33:12

From: BraK
I don't think I released down on SSRL what was going on with me. I'm moving back to the states and my wife is currently pregnant this is what has me tied up at the moment. I'll probably start working on RBOSS again this summer.


Early Gz :) As for me, I'm busy with real life :/


~Home

BraK

2012-02-27 11:38:30

BTW I'm sure none of the people already working on RBOSS will mind if some others want to work on it as well. It's on GITHub at https://github.com/GitBraK/SRL-Development Feel free to look at the code and contribute as you see fit. All of the Team members have commit access. If you would like commit access then post on the RBOSS Thread and I'll add your GitHub Account.

WT-Fakawi

2012-09-09 05:34:05

I might have some time to burn in the next couple of months. I would really love to see a system like this implemented. Why? I just ran HLR (After a year or so). Most inventory objects are new, yet all recognized and properly maintained by HLR, completely without bitmaps or dtms!!!!

The local itemdatabase that is build runtime from HLR suffices for proper everything!

Gu3st...

2012-09-09 06:56:37

Yes, this is most definitely the future :) I think we should group together and plan SRL-6 sometime soon. Let's brainstorm a lot of ideas and make something completely new