Handling such applications may require a deeper understanding of the application and its components to dynamically traverse the object tree and find a required element by its properties. This article provides and demonstrates a set of utility functions that can be used to automate these applications.

Consider a sample application with a ListBox control as shown below:

Suppose we have Learned just a single item (listBoxItem4). And now our goal is to click any other items in the same list box.

First, let’s look at the locator part of the learned item:

It looks as shown below:

tabControl1/tabItem1/listBox1/listBoxItem4/listBoxItem4

This path matches the tree structure that we may see from the UI Automation Spy:

So if we need to click a specific item while having object pointing to listBoxItem4 then we need to do the following:

  1. Climb 2 levels up to the whole [List].
  2. Search through all the items matching specific text (i.e. listBoxItem5 or so).

Modern Method

function Test()
{
    Global.DoLaunch("AUTWPF\\AUTWPF.exe", null, true, "AUTWPF");


    // We have object listBoxItem4, let's click it
    SeS('listBoxItem4').DoClick();

    // Example 1: We learned item #4. But now we want to click item #2 by overriding locator
    // Please, note the object 'listBoxItem4' has 'Ignore Object Name' property set to 'true'
    SeS('listBoxItem4', { "location": "tabControl1/tabItem1/listBox1/listBoxItem2/listBoxItem2" }).DoClick();

    // Now we want to click items by text or name.    
    // So we want to start from 'listBoxItem4'
    // Then climb 2 levels up to whole listbox
    // and then find for specific item by its name or text.
    // So here we introduce several utility functions:    
    // UIAParentObj, UIAGetChildCount, UIAGetChildAt, UIAFindByText
    // You may find them in the WPFCalc.user.js file and use in your own projects.
    // Here is how it works:
    // Climb 2 levels (2 because we can see from the locator:    
    // ....tabItem1/listBox1/listBoxItem4/listBoxItem4
    // that 2 last chunks correspond to an item and we need to climb up to    
    // the ...tabItem1/listBox1 ):
    var par = SeS('listBoxItem4').GetParent().GetParent();
    // Now par is a listBox1. Search for item named listBoxItem5 recursively:
    var i5 = par.DoFindByText("listBoxItem5");
    // And click it:
    i5.DoClick();
}

See UIAObject.DoFindByText for more details.

Legacy Method

The attached Rapise sample contains a number of functions and a working sample demonstrating how it can be achieved.

In addition, here is the list of utility functions (you may find them in the attached sample):

//#region UIA utility functions

/**
 * UIAInstance - used internally by other functions
 */
function UIAInstance(obj)
{
    var instance = obj;
    try
    {
        if (obj && obj.instance)
        {
            instance = obj.instance;
        }
    } catch (e) { }
    return instance;
}

/**
 * Find parent object (or n-th parent)
 * @param obj root object
 * @param lvl {=1} number of levels to climb
 * @return found object or null
 */
function UIAParentObj(obj, lvl)
{
    if (typeof (obj) == "string")
    {
        obj = SeS(obj);
    }

    lvl = lvl || 1;
    var instance = UIAInstance(obj);
    var par = SeSGetUIAutomationParent(instance);
    if (lvl && lvl > 1)
    {
        return UIAParentObj(par, lvl - 1);
    }
    return SeSTryMatch(par, SeSUIAObjectRule);
}

/**
 * Find number of children inside this object
 * @param obj root object
 * @return number of children, 0 - no children
 */
function UIAGetChildCount(obj)
{
    var instance = UIAInstance(obj);
    return SeSGetUIAutomationChildrenCount(instance);
}

/**
 * Find number of children inside this object
 * @param obj root object
 * @param ind ind's child
 * @return found child object
 */
function UIAGetChildAt(obj, ind)
{
    var instance = UIAInstance(obj);
    var chld = SeSGetUIAutomationChildAt(instance, ind);
    return SeSTryMatch(chld, SeSUIAObjectRule);
}

/**
 * Recursively search for a child having specified name or text
 * @param obj start object
 * @param findTxt string to match object text or name
 * @param [fndClsName] optional class name to match (sometimes we need Text object, not ListBoxItem, so className should help to filter)
 * @return found object
 */
function UIAFindByText(obj, findTxt, findClsName)
{
    var instance = UIAInstance(obj);
    var txt = SeSGetUIAutomationText(instance);
    var cls = SeSGetUIAutomationClass(instance);
    var name = SeSGetUIAutomationName(instance);
    if (SeSCheckString(findTxt, name) || SeSCheckString(findTxt, txt))
    {
        if (findClsName)
        {
            if (SeSCheckString(findClsName, cls))
            {
                return SeSTryMatch(instance, SeSUIAObjectRule);
            }
        } else
        {
            return SeSTryMatch(instance, SeSUIAObjectRule);
        }
    }
    // No match, check children

    var cnt = UIAGetChildCount(instance);
    for (var i = 0; i < cnt; i++)
    {
        var chld = SeSGetUIAutomationChildAt(instance, i);
        var res = UIAFindByText(chld, findTxt, findClsName);
        if (res) return res;
    }
    return null;
}
            //#endregion UIA utility functions

The test working with these functions is following:

function Test()
{
    Global.DoLaunch("AUTWPF\\AUTWPF.exe", null, true, "AUTWPF");


    // We have object listBoxItem4, let's click it
    SeS('listBoxItem4').DoClick();

    // Example 1: We learned item #4. But now we want to click item #2 by overriding locator
    // Please, note the object 'listBoxItem4' has 'Ignore Object Name' property set to 'true'
    SeS('listBoxItem4', { "location": "tabControl1/tabItem1/listBox1/listBoxItem2/listBoxItem2" }).DoClick();

    // Now we want to click items by text or name.    
    // So we want to start from 'listBoxItem4'
    // Then climb 2 levels up to whole listbox
    // and then find for specific item by its name or text.
    // So here we introduce several utility functions:    
    // UIAParentObj, UIAGetChildCount, UIAGetChildAt, UIAFindByText
    // You may find them in the WPFCalc.user.js file and use in your own projects.
    // Here is how it works:
    // Climb 2 levels (2 because we can see from the locator:    
    // ....tabItem1/listBox1/listBoxItem4/listBoxItem4
    // that 2 last chunks correspond to an item and we need to climb up to    
    // the ...tabItem1/listBox1 ):
    var par = UIAParentObj('listBoxItem4', 2);
    // Now par is a listBox1. Search for item named listBoxItem5 recursively:
    var i5 = UIAFindByText(par, "listBoxItem5");
    // And click it:
    i5.DoClick();
}