SRL Archive - Development

Ok so other than allowing BegoLib to access SMART's buffer ( which can be implented easily )

Ive finally got down into the nitty gritty stuff..

BegoLib will use several object recognition alogorithms, based on what's required to be detected.

Currently most medium->large mainscreen objects can be detected via applying the following filters.

Blur image (reduce noise)
Detect Edges
(change thresholds depending on how much content is on screen)
Apply object point detection (for the final cords)

Heres the first test, NOTE its fully customisable, and tweaking settings and threshholds can dramatically increase detection.

The pictures below just show an example of how BegoLib will 'see' the RS screen, and this is only basic algorithms.


I will be creating further filters to seperate objects based on size, number of colors in the object, location of object, shape of object.

Also working on haar training, which can be trained to detect SPECIFIC objects.

Wow.

All the best! Can't wait for the finished thing :)

Looks promising, Keep it up! +rep :)

Damn I've been checking every week to see if you've made a thread for Begolib, glad to see it's finally here, if you need any assistance give me a pm!

Wow, this is some awesome work! :)

You stated it works with:
Blur image (reduce noise)
Detect Edges

So isn't that the same as canny edge? Or do you use different methods? I saw that benland once created that for scar. I also made one for simba but didn't had the same results. Wasn't fast enough. Will this be opensource if it is not canny edge?

Edge detection will be the most simplistic object recognition in BegoLib, Ill have many custom filters which we can apply to the detection routines..

Imagine applying the above to the current color finding functions, That alone would increase speed and accuracy.

Im still debating on how to work/interact with the found objects… Of course there will be a Simba wrapper, which most probarly will be added to SRL.

But what im not sure about is returning a structure (every point which is a object) from the library to Simba, Ive not worked with pascal and pointers, but depsite the fact how smart works it should be all fine.

When we apply filters, we usually get a single channel mask (grayscale), thus way easyier to work with for detecting which points are actually objects, That being said theres a limitation involved there where from the library we would need to keep two buffers, one for the actual image/screen and one for the mask so we can later detect colors in objects, Althou we will be using SMART's buffer for the screen, so effciency shouldnt be a problem.

Since BegoLib was originally going to be aimed at simba only, Ive decided to rename the project CMacro which will work on loaded images/screen data or in our case SMART's buffer.

Looks interesting!

I posted something like this just a few days ago…

if you need help with corner counting or any other type of things let me know.

these are corners that have been detected…


[ATTACH]15547[/ATTACH]

From: i3egoHan

Since BegoLib was originally going to be aimed at simba only, Ive decided to rename the project CMacro which will work on loaded images/screen data or in our case SMART's buffer.

With that being said I'm assuming this is done in some form of C? Have you ran any tests to give us an idea of the speed?

From: SKy Scripter
I posted something like this just a few days ago…

Was my very first thought, lol.

Haar training looks so cool:

[YOUTUBE]DLSQebBrKsc[/YOUTUBE]

I understand what you have done in the example. But it would be amazing if you are able to implement those other object recognition functions!

From: masterBB
You stated it works with:
Blur image (reduce noise)
Detect Edges

So isn't that the same as canny edge?

Ben's cannycore is only edge detection , doesn't have blur. Some difference are visible:

@i3egoHan

I can't wait to use your filters.:)

Imppreessiiveee stuff man.

From: masterBB
Haar training looks so cool:

[YOUTUBE]DLSQebBrKsc[/YOUTUBE]

I understand what you have done in the example. But it would be amazing if you are able to implement those other object recognition functions!

Well haar training wouldnt be so benficial for us it requires a fair bit of work to train a decent cascade, And this would only be able to match simular objects… For example… a bank booth one would be very usefull and give us very accurate instant banking. But for NPC's other objects each object would need training.

From: NCDS
With that being said I'm assuming this is done in some form of C? Have you ran any tests to give us an idea of the speed?


Was my very first thought, lol.

C++ (oop), Havent benchmarked anything yet, But given CMacro will have direct access to SMART's buffer its just a matter of running the algorithms on the buffer (and there pretty fast)

From: SKy Scripter
I posted something like this just a few days ago…

if you need help with corner counting or any other type of things let me know.

these are corners that have been detected…


[ATTACH]15547[/ATTACH]

nice, ill keep it in mind.

I've released a couple of function using SRL to detect edge of mixed color without extra plugin/library, but its speed varies how accurate you wish to find the object. (Check in the "goodies" section, if a mod sees this.. Could you please move that thread to this forum?)

I definitely hope this is fast, and would it be possible to do this without smart?

From: Main
I've released a couple of function using SRL to detect edge of mixed color without extra plugin/library, but its speed varies how accurate you wish to find the object. (Check in the "goodies" section, if a mod sees this.. Could you please move that thread to this forum?)

I definitely hope this is fast, and would it be possible to do this without smart?

Its possible to use without smart yes (that's how its working right now), The library can capture from screen, (DibSection)… Yes thats why it needs to be in a library else speed is a killer.

Simba should be able to manipulate a pointer to a structure of 2 int's easyily enough right?

Looks like some cool computer vision algorithms are at work here, good luck with this :)

What do you use to do your edge detection? Im more familiar with LoG or DoG approach (laplacian/difference of gaussian) i was wondering how you would run this so quickly with so little lag?

I wonder how well this would work (speed-wise) if we were to do the following.

Apply selective filters/object recognition, Return a pointer to a new buffer to simba, set the pointer as target, Do all our whacky image/pixel finding, then return the old buffer.

This would have the advantage of using all the simba functions (so all scripts will work fine) and people can apply 'filters' which will be contained in this library to filter out more then 70% of unwanted/unmatching objects.

Very nice indeed.
I really like what your doing and hope it all goes to plan.
Keep up the good work :)

It's looking very promising! :)

what i have found is there is a faster way to getting objects on the game…
essentially its by filtering all the mass pixels out and leaving only the objects in there.


you can grey scale the picture and make a histogram[255]
by doing that you can count the amount of area with each avg pixel.
your result would look something like this.

            int[] histogram = new int[256];
            // Gray Scale Histogram
            int max = 0;
            int min = 100000;
            int av = 0;
            int ct = 0;
            for (int y=eb; y<h; y++) 
                 for (int x=sa; x<w; x++)
                    // if (inb(x, y))
                     {
                         Color co = ColorTranslator.FromWin32(c[x, y]);
                         int avg = (co.R + co.G + co.B) / 3;
                            
    

                      //   c[x, y] =   (avg << 16) | (avg << 8) | avg;
                         histogram[avg]++;
                         if (max < histogram[avg]) max = histogram[avg];
                     if (histogram[avg] > 0)
                         if (min > histogram[avg]) min = histogram[avg];
                       //  c[x, y] = ColorTranslator.ToWin32(new Color().;
                    av = av + histogram[avg];
                     ct++;
                     }

            // finding mass obj.
            av = av / ct;
            int[] gram = histogram;
            Array.Sort(gram);
            richTextBox1.AppendText("Count Max:" + max.ToString() + " Min: " + min.ToString() + 
                " Av:" +av.ToString()+"\n");

            pts = new Point[w*h];
           // int cnt = 0;

            // Ananlzing threshold;

            int thres = 27;
            // Analyzing Points
          //  richTextBox1.AppendText("thres: " + thres.ToString());
            for (int y = eb; y < h; y++)
                for (int x = sa; x < w; x++)
                 //  if (c[x, y] > 0)
                    {
                        Color co = ColorTranslator.FromWin32(c[x, y]);
                        int avg = (co.R + co.G + co.B) / 3;
                        
                        if (histogram[avg] > thres) {
                            pts[cnt] = new Point(x, y);
                            c[x, y] = 0;
                         //   richTextBox1.AppendText("Hue: " + getHue(co.R, co.G, co.B).ToString() + "\n");
                            for (int i = 0; i < 8; i++)
                            {
                                int a = x + cx[i];
                                int b = y + cy[i];
                                if ((a > 0) && (b > 0) && (a < w) && (b < h))
                                {
                                    int newc = c[a, b];
                                    if ((newc != clpurple) && (newc != 0))
                                    {
                                        Color co2 = ColorTranslator.FromWin32(newc);
                                        int av2 = (co2.R + co2.G + co2.B) / 3;
                                        if (histogram[av2] < thres)
                                        {
                                            c[x, y] = clpurple;
                                            break;
                                        }
                                    }

                                }
                            }
                           
                            cnt++;
                            
                           // c[x, y] = 0;
                        }
                    }


            pts = pts.Take(cnt).ToArray();


            foreach (Point np in pts)
            {
                ptlist.Add(np);
            }

Nice work, Ive actually added alot since i last posted, this months a busy month for me though, but will post updates asap.

Edit: btw, yeah i was doing kinda the same think, i marked filtered pixels in red though… just to show everyone a debug view :D

Strange I never uploaded my edge detection test script.

The problem with this approach is that you can recognize an object, but you can't identify the object. A rock is a rock, you don't know whether it's gold, tin, copper (etc).

From: mastaraymond
Strange I never uploaded my edge detection test script.

The problem with this approach is that you can recognize an object, but you can't identify the object. A rock is a rock, you don't know whether it's gold, tin, copper (etc).

But then would this be very good for monster killing as a example? Or chopping trees with a unique shape :)

From: mastaraymond
Strange I never uploaded my edge detection test script.

The problem with this approach is that you can recognize an object, but you can't identify the object. A rock is a rock, you don't know whether it's gold, tin, copper (etc).

Find object-> search for a certain color with a certain tolerance to identify

wow! very nice. If you combine all color search algorithms you can really make a nice function to detect all kinds of objects! Nice to see that some1 has started to make such a function.