Rapise ImageFinder

For example, Google Maps may find several food courts:

Food courts

And we can enumerate them on the map by finding the pictogram:

Food icon

To do it we need a template image that we want to find in our application.


Attached ImageFinder.zip  file should be extracted into the folder c:\Users\Public\Documents\Rapise\Scripts (so that ImageFinder sub-folder should appear).


ImageFinder should be included into the test by adding the following line into the Functions file:

eval( File.Include('ImageFinder/SeSImageFinder.js') )

Then it may be used as follows:

ImageFinder.DoFindInObject( 'ObjectId', Global.GetFullPath('Template.png') )

Where 'ObjectId' is for a learned object containing image to search and Template.png is an image to search within the canvas.

ImageFinder.DoFindInObject returns number of found matches of the image.

    var foundCount = ImageFinder.DoFindInObject('ObjectId', Global.GetFullPath('Template.png') )

    Tester.Assert('At least 5 items found', foundCount>5);

ImageFinder.DoClickOnMatch used to click on found image.

    // Click in the center of 0'th match
    // Right-click on match #2 with offset (5,5) px.
    ImageFinder.DoClickOnMatch(5, 5, 2, "R")

Template selection

It is important to choose templates using unique and non-overlapping pattern. For example, if we choose piece like that as a template:

Icon with background

The number of found items will be not high because it contains parts of the road and fragment of the river. So it is unique to a specific location.

Image finder does not support transparency, so cleaning the surrounding of the round icon does not make sense. Also overlapping icons would still be a problem in this case:

Overlapping icons

So recommended, resilient solution for this case would be use of fork&knife part as a template:

Good template


Sample test is available as an attachment to this topic. Note, that you also need to download and  unpack OpenCV binaries used by the test, as described earlier in this topic.