AiRobot makes interactions with the application completely automatic. It understands the screen contents, moves the mouse and presses the keyboard keys.

AiRobot may interact with the Web Browser,  with window or with Desktop.

Installing AiRobot

Right-click Modules/Pages and choose Import Public Page Object > AiRobot.

After installation find AiRobot inside Modules/Pages.

Configuring AiRobot

You need to use AI Dashboard and configure access to Amazon Bedrock. The model is Anthropic Claude Sonnet V2. You may find more detailed information about the configuration here:

https://www.youtube.com/watch?v=9Cx5rNPet5A

Environment Restrictions

We recommend you to only use it in a restricted environment.

If you want the AiRobot to interact with full desktop, make sure you only run it in special test environment (i.e. VM). I.e. don't use it on your own laptop where  you have your personal information and sensitive data. The AiRobot is creative and sometimes it is not predictable. It may read something from the screen and accept it as instructions. For example, if the test is running for the whole desktop and you have a Teams Message popping up saying something like "Reboot now", it may reboot the machine (or shut it down). 

Window Size

AiRobot may struggle to operate applications on screens with resolutions higher than XGA (1024×768) or WXGA (1280×800) due to issues with image scaling. So we recommend to resize the browser or application to fit into that size explicitly for achieving the best results, i.e.:

 

Be More Specific

The AiRobot may choose different paths to do the same action. For example, if we say "Edit the book 'Matilda'", there are different options on how to proceed.

You may find more information about it here:

https://www.youtube.com/watch?v=Ict6iCC5OI8

AiRobot vs AI

Rapise has an AI command in RVL which look very similar to AiRobot.DoWebBrowser / AiRobot.DoDesktop / AiRobot.DoWindow. The key difference here is runtime token usage. While AI generates a code once and then uses it for all test executions without accessing the AI and spending any tokens, the AiRobot is interactive - it is spending tokens during the execution, all the time. 

 

Examples

Editing a book (interacting with web browser):

 

Interacting with the desktop calculator:

 

More Information

You may find more information about AI in Rapise from this YouTube playlist:

https://www.youtube.com/watch?v=ztMQsWBDfAg&list=PL1GncVUgF5nsm6OxusGnzY8iBE-nEp36C