Rapise - Using Optical Character Recognition (OCR)

March 19th, 2015 by Adam Sandman

You have tried nearly everything. You tried learning, that didn't work. You tried simulating the object, but it is moving around. All hope is lost. Here he comes to save the day, that means UnderDog is on the way!!!In this case, underdog, or the hero, may be OCR. Rapise has many built in capabilities to facilitate the learning and identification of objects used in a test, and OCR is one of them. But what is OCR anyway?

What is OCR?

OCR stands for Optical Character Recognition. I won't go into specific details, but provide a high level overview. OCR takes the object, area, or window and converts it to an image. OCR then scans this image for patterns it recognizes as part of a known character set. Sounds sketchy? Not as much as you might think.

OCR has not always had a great reputation, and think about it, we had underpowered computers that could barely keep up with the tasks at hand, much less power operations such as image processing and character recognition. With modern systems and algorithms, this task has become much better. It is still not perfect but in a pinch it can be incredibly useful.

A Quick Tip about OCR

If you intend to use OCR as a part of your testing, you MUST turn off font smoothing as this is a display layer smoothing that will throw off the OCR capabilities in ANY OCR reliant process weather it be from Rapise or other commercial grade OCR tool.

With OCR you can identify a known string in an image or area. You can identify the points that form the string and interact with them. You can capture values and a host of other activities.

Using OCR in Rapise for Testing

To get to the OCR commands, expand the Global item in the Object tree, within Global expand OCR and you can drag and drop the following commands into your test and add parameters:

DoClickTextInObject(objId, text, clickType) - allows Rapise to initiate a click based on the position of a text string expected within a defined object. Define the ID of the object that is expected to contain the string, define the string, and define the click type.
DoClickTextInWindow(objId, text, clickType) - looks is a window for an object, finding that object the built in OCR engine searches the window for a text string and clicks on it with the defined clicktype.
DoExtractFromImage(img)- extracts the text found via OCR from an image (path to image defined in img)
DoExtractFromObject(objId) - extracts the OCR recognized text from a previously defined object
DoExtractFromScreen(left, top, right, bottom) - extracts OCR recognized text from a defined area (x1,y1.x2,y2)
DoExtractFromWindow(objId) - for a window that contains a defined object, Rapise will OCR the entire window to return the text
DoSelectOcrEngine(ocrName) - for power players. Ability to define the OCR engine available to Rapise