Aug 142011
 

Testing the user interface has to be the most boring part of testing software. Not only is it not fun, it can be horribly repetitive as well with you having to check the same functionality over and over again. The time spent on it could be put to use to far better purposes. Automated Testing tools like NUnit and MS Test were developed in order to counter this. Along came processes like Test Driven development that emphasized the need to write test cases and write testable software. Though all this helped a lot, its benefits were confined mostly to the developer himself, it wasn’t of any use to the testers who again put in redundant effort in testing the UI.

It was then that the idea of automating UI testing starting getting thrown about and many such tools and frameworks appeared in the market. These frameworks essentially provide an API to programmatically access the user interface of Windows applications and gives the developer to simulate the end user’s way of accessing our software. One such framework which I found extremely useful was the UI Automation library from Microsoft. It works pretty well with any UI technology including relatively recent ones such as WPF.

At the core of the UI Automation library is the type AutomationElement – All UI elements in the application get wrapped into this AutomationElement and provide a common way of interacting with the UI elements. For e.g. The application window will be an AutomationElement just like a small button inside the window. Both support different set of Actions but share the commmon types, this makes writing code for UI automation extremely simple.

I wrote a simple calculator application and a console application which tests the calculator giving random values and verifying the output. The XAML Code for the WPF application is:-

    <grid x:Name="LayoutRoot">
        </grid><grid .RowDefinitions>
            <rowdefinition Height="1*" />
            <rowdefinition Height="1*" />
            <rowdefinition Height="1*" />
            <rowdefinition Height="4*" />
        </grid>
        <grid .ColumnDefinitions>
            <columndefinition Width="2*" />
            <columndefinition Width="4*" />
        </grid>
        <label Grid.Row="0" Grid.Column="0" Content="First Number" />
        <textbox Grid.Row="0" Grid.Column="1" AutomationProperties.AutomationId="txtFirstNumber" Text="{Binding FirstNumber,Mode=TwoWay,UpdateSourceTrigger=PropertyChanged}" />
        <label Grid.Row="1" Grid.Column="0" Content="Second Number" />
        <textbox Grid.Row="1" Grid.Column="1" AutomationProperties.AutomationId="txtSecondNumber" Text="{Binding SecondNumber,Mode=TwoWay,UpdateSourceTrigger=PropertyChanged}" />
        <label Grid.Row="2" Grid.Column="0" Content="Result" />
        <textbox Grid.Row="2" Grid.Column="1" AutomationProperties.AutomationId="txtResult" Text="{Binding Result,Mode=TwoWay,UpdateSourceTrigger=PropertyChanged}" />
        
        <grid Grid.Row="3" Grid.Column="0" Grid.ColumnSpan="2">
            </grid><grid .ColumnDefinitions>
                <columndefinition Width="1*" />
                <columndefinition Width="1*" />
                <columndefinition Width="1*" />
                <columndefinition Width="1*" />
            </grid>
            <button Grid.Column="0" AutomationProperties.AutomationId="btnAdd" Content="Add" Command="{Binding Add}" />
            <button Grid.Column="1" AutomationProperties.AutomationId="btnSubtract" Content="Subtract" Command="{Binding Subtract}" />
            <button Grid.Column="2" AutomationProperties.AutomationId="btnMultiply" Content="Multiply" Command="{Binding Multiply}" />
            <button Grid.Column="3" AutomationProperties.AutomationId="btnDivide" Content="Divide" Command="{Binding Divide}" />
   

This creates a simple calculator app with four functions Add, Subtract, Multiply and Divide. Here is a screenshot of the app

The viewmodel is pretty simple as well

    public class MainViewModel : ViewModelBase
    {
       
        private double _firstNumber;
        private double _secondNumber;
        private double _result;
        private RelayCommand _add;
        private RelayCommand _multiply;
        private RelayCommand _subtract;
        private RelayCommand _divide;

        public double FirstNumber
        {
            get { return _firstNumber; }
            set
            {
                _firstNumber = value;
                RaisePropertyChanged("FirstNumber");
            }
        }

        public double SecondNumber
        {
            get { return _secondNumber; }
            set
            {
                _secondNumber = value;
                RaisePropertyChanged("SecondNumber");
            }
        }

        public double Result
        {
            get { return _result; }
            set
            {
                _result = value;
                RaisePropertyChanged("Result");
            }
        }

        public ICommand Add
        {
            get { _add = _add ?? new RelayCommand(() => Result = FirstNumber + SecondNumber); return _add; }
        }

        public ICommand Subtract
        {
            get { _subtract = _subtract ?? new RelayCommand(() => Result = FirstNumber - SecondNumber); return _subtract; }
        }

        public ICommand Multiply
        {
            get { _multiply = _multiply ?? new RelayCommand(() => Result = FirstNumber * SecondNumber); return _multiply; }
        }

        public ICommand Divide
        {
            get { _divide = _divide ?? new RelayCommand(() => Result = FirstNumber / SecondNumber); return _divide; }
        }
  }

The project also contains a simple Console application which launches along with the project and automates the testing of the application by entering values in the textboxes and pressing the four buttons and verifying the result values.

namespace CalculatorTest
{
    class TestCalc
    {
        AutomationLibrary library = new AutomationLibrary();

        AutomationElement desktopElement;
        AutomationElement mainWindow;

        public TestCalc()
        {
            desktopElement = GetDesktop();
        mainWindow = library.GetElement(desktopElement, AutomationElement.NameProperty, "Simple Calculator");
        }
        internal void StartTest()
        {
            for (int i = 0; i < 25; i++)
            {
                Random r = new Random();
                double firstNumber = r.Next(1300);
                Thread.Sleep(30);
                double secondNumber = r.Next(1000);
                if (VerifyResult(firstNumber, secondNumber))
                    Console.WriteLine(" All Tests Have Passed for {0} & {1}",firstNumber,secondNumber);
                else
                    Console.WriteLine("Tests have failed for {0} & {1}", firstNumber, secondNumber);
            }

        }

        public bool VerifyResult(double firstNumber,double secondNumber)
        {
            bool allTestPass = true; 
            library.SetValueOnTextBox(mainWindow, AutomationElement.AutomationIdProperty, "txtFirstNumber", TreeScope.Descendants, firstNumber.ToString());
            library.SetValueOnTextBox(mainWindow, AutomationElement.AutomationIdProperty, "txtSecondNumber", TreeScope.Descendants, secondNumber.ToString());
            double addResult = PressOpButtonandGetValue("btnAdd");
            double subResult = PressOpButtonandGetValue("btnSubtract");
            double mulResult = PressOpButtonandGetValue("btnMultiply");
            double divResult = PressOpButtonandGetValue("btnDivide");

            if (addResult != (firstNumber + secondNumber))
                allTestPass = false;
            if (subResult != (firstNumber - secondNumber))
                allTestPass = false;
            if (mulResult != (firstNumber * secondNumber))
                allTestPass = false;
            if (Math.Round(divResult,5) != Math.Round((firstNumber / secondNumber),5))
                allTestPass = false;
            return allTestPass;
        }

        public double PressOpButtonandGetValue(string buttonID)
        {
            library.PressButtonOnWindow(mainWindow, AutomationElement.AutomationIdProperty, buttonID, TreeScope.Descendants);
            Thread.Sleep(200);
            string result = library.GetTextFromTextElement(mainWindow, AutomationElement.AutomationIdProperty, "txtResult", TreeScope.Descendants);
            double actualResult;
            if (double.TryParse(result, out actualResult))
                return actualResult;
            throw new ArgumentException("Wrong Value detected");
        }

        private AutomationElement GetDesktop()
        {
            Console.WriteLine("Getting Desktop");
            var desktopElement = library.GetRootElement();
            if (desktopElement == null)
            {
                Console.WriteLine("Unable to get the desktop Element, Exiting the application");
                throw new ElementNotAvailableException("Unable to get the desktop Element");
            }
            return desktopElement;
        }
    }
}

Directly interacting with the Microsoft UI Automation API can lead to extremely repetitive code since most of the functions are used again and again. So I wrote a small library to simply this. Here is the code. It is still largely uncommented and not refactored yet, will improve it in future blog posts.

class AutomationLibrary
    {
        public AutomationElement GetRootElement()
        {
            return AutomationElement.RootElement;
        }

        public AutomationElement GetElement(AutomationElement rootElement, AutomationProperty property, object value)
        {
            return GetElement(rootElement, property, value, TreeScope.Children);
        }

        public AutomationElement GetElement(AutomationElement rootElement, AutomationProperty property, object value, TreeScope searchScope)
        {

            AutomationElement aeMainWindow = null;

            int numWaits = 0;
            do
            {
                aeMainWindow = rootElement.FindFirst(searchScope, new PropertyCondition(property, value));
                ++numWaits;
                Thread.Sleep(200);
            } while (aeMainWindow == null && numWaits < 50);
            return aeMainWindow;
        }

        public AutomationElement GetElementWithoutWait(AutomationElement rootElement, AutomationProperty property, object value)
        {
            return GetElementWithoutWait(rootElement, property, value, TreeScope.Children);
        }

        public AutomationElement GetElementWithoutWait(AutomationElement rootElement, AutomationProperty property, object value, TreeScope searchScope)
        {
            AutomationElement aeMainWindow = rootElement.FindFirst(searchScope, new PropertyCondition(property, value));
            return aeMainWindow;
        }


        public bool PressButtonOnWindow(AutomationElement element, AutomationProperty property, object value)
        {
            return PressButtonOnWindow(element, property, value, TreeScope.Children);
        }

        public bool PressButtonOnWindow(AutomationElement element, AutomationProperty property, object value, TreeScope treeScope)
        {
            try
            {
                //var window = GetElementWithoutWait(element, property, value,treeScope);
                //if (window == null)
                //    window = GetElement(element,property,value,treeScope);
                //if(window==null)
                //    return false;
                //else
                //{
                var buttonInvoke = GetInvokePattern(element, property, value, TreeScope.Descendants);
                buttonInvoke.Invoke();
                return true;
                // }
            }
            catch
            {
                return false;
            }
        }

        public InvokePattern GetInvokePattern(AutomationElement window, AutomationProperty property, object value, TreeScope treeScope)
        {
            AutomationElement aeButtonElement;
            int numWaits = 0;
            do
            {
                aeButtonElement = window.FindFirst(treeScope, this.GetPropertyCondition(property, value));
                ++numWaits;
                Thread.Sleep(500);
            } while (aeButtonElement == null && numWaits < 75);
            object objPattern;
            InvokePattern invokePatternObj;
            if (aeButtonElement.TryGetCurrentPattern(InvokePattern.Pattern, out objPattern))
            {
                invokePatternObj = objPattern as InvokePattern;
                return invokePatternObj;
            }
            return null;
        }

        public ValuePattern GetValuePatternWithoutWait(AutomationElement window, AutomationProperty property, object value, TreeScope searchScope)
        {
            AutomationElement aeTextBoxElement = window.FindFirst(searchScope, GetPropertyCondition(property, value));
            if (aeTextBoxElement == null)
                throw new ElementNotAvailableException("TextBoxElement Element not Available");
            object objPattern;
            ValuePattern valuePatternObj;
            if (aeTextBoxElement.TryGetCurrentPattern(ValuePattern.Pattern, out objPattern))
            {
                valuePatternObj = objPattern as ValuePattern;
                return valuePatternObj;
            }
            else
                throw new ElementNotEnabledException("The Value Pattern was not retrieved from the element");
        }

        public InvokePattern GetInvokePatternWithoutWait(AutomationElement window, AutomationProperty property, object value, TreeScope searchScope)
        {
            AutomationElement aeButtonElement = window.FindFirst(searchScope, GetPropertyCondition(property, value));
            if (aeButtonElement == null)
                throw new ElementNotAvailableException("Button Element not Available. Try the GetInvokePattern method which has a wait");
            object objPattern;
            InvokePattern invokePatternObj;
            if (aeButtonElement.TryGetCurrentPattern(InvokePattern.Pattern, out objPattern))
            {
                invokePatternObj = objPattern as InvokePattern;
                return invokePatternObj;
            }
            else
                throw new ElementNotEnabledException("The Invoke Pattern was not retrieved from the element");
        }


        public ExpandCollapsePattern GetExpandCollapsePattern(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            AutomationElement aeExpanderElement;
            int numWaits = 0;
            do
            {
                aeExpanderElement = GetFirstChildNode(element, property, value, searchScope);
                ++numWaits;
                Thread.Sleep(300);
            } while (aeExpanderElement == null && numWaits < 75);
            object objPattern;
            ExpandCollapsePattern togPattern;
            if (true == aeExpanderElement.TryGetCurrentPattern(ExpandCollapsePattern.Pattern, out objPattern))
            {
                togPattern = objPattern as ExpandCollapsePattern;
                return togPattern;
            }
            else
                return null;

        }

        public void SetValueOnTextBox(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope, string valueToBeSet)
        {
            var valuePattern = GetValuePatternWithoutWait(element, property, value, searchScope);
            if (valuePattern != null)
                valuePattern.SetValue(valueToBeSet);
        }

        public ExpandCollapsePattern GetExpandCollapsePatternWithoutWait(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            try
            {
                AutomationElement aeExpanderElement = GetFirstChildNode(element, property, value, searchScope);
                if (aeExpanderElement == null)
                    throw new ElementNotAvailableException("Expander Element not available. Try the GetExpandCollapsePattern which has a wait");
                object objPattern;
                ExpandCollapsePattern togPattern;
                if (true == element.TryGetCurrentPattern(ExpandCollapsePattern.Pattern, out objPattern))
                {
                    togPattern = objPattern as ExpandCollapsePattern;
                    return togPattern;
                }
                else
                    return null;
            }
            catch
            {
                return null;
            }
        }

        public SelectionItemPattern GetSelectionItemPattern(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            AutomationElement aeSelectionPattern;
            int numWaits = 0;
            do
            {
                aeSelectionPattern = GetFirstChildNode(element, property, value, searchScope);
                ++numWaits;
                Thread.Sleep(300);
            } while (aeSelectionPattern == null && numWaits < 75);
            object objPattern;
            SelectionItemPattern selectionItemPattern;
            if (true == element.TryGetCurrentPattern(SelectionItemPattern.Pattern, out objPattern))
            {
                selectionItemPattern = objPattern as SelectionItemPattern;
                return selectionItemPattern;
            }
            else
                return null;
        }

        public SelectionItemPattern GetSelectionItemWithoutWait(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            try
            {
                AutomationElement aeExpanderElement = GetFirstChildNode(element, property, value, searchScope);
                if (aeExpanderElement == null)
                    throw new ElementNotAvailableException("Expander Element not available. Try the GetSelectionItemPattern which has a wait");
                object objPattern;
                SelectionItemPattern togPattern;
                if (true == element.TryGetCurrentPattern(SelectionItemPattern.Pattern, out objPattern))
                {
                    togPattern = objPattern as SelectionItemPattern;
                    return togPattern;
                }
                else
                    return null;
            }
            catch
            {
                return null;
            }
        }

        public TextPattern GetTextPatternWithoutWait(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            try
            {
                AutomationElement aeTextElement = GetFirstChildNode(element, property, value, searchScope);
                if (aeTextElement == null)
                    throw new ElementNotAvailableException("Text Element not available");
                object objPattern;
                TextPattern txtPattern;
                if (true == aeTextElement.TryGetCurrentPattern(TextPattern.Pattern, out objPattern))
                {
                    txtPattern = objPattern as TextPattern;
                    return txtPattern;
                }
                else
                    return null;
            }
            catch
            {
                return null;
            }
        }

        public string GetTextFromTextElement(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            var textPattern = GetTextPatternWithoutWait(element, property, value, searchScope);
            if (textPattern != null)
                return textPattern.DocumentRange.GetText(-1);
            return null;

        }



        public PropertyCondition GetPropertyCondition(AutomationProperty property, object value)
        {
            return new PropertyCondition(property, value);
        }

        public AutomationElementCollection GetAllChildNodes(AutomationElement element, AutomationProperty automationProperty, object value, TreeScope treeScope)
        {
            var allChildNodes = element.FindAll(treeScope, GetPropertyCondition(automationProperty, value));
            if (allChildNodes == null)
                throw new ElementNotAvailableException("Not able to find the child nodes of the element");
            return allChildNodes;
        }

        public AutomationElement GetFirstChildNode(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            var firstchildNode = element.FindFirst(searchScope, GetPropertyCondition(property, value));
            if (firstchildNode == null)
                throw new ElementNotAvailableException("Not able to find the first child node of the element");
            return firstchildNode;
        }

        public bool FirstChildTextNodeContains(AutomationElement element, string toCheck)
        {
            var firstTextNode = this.GetFirstChildNode(element, AutomationElement.ControlTypeProperty, ControlType.Text, TreeScope.Children);
            if (firstTextNode.Current.Name == toCheck)
                return true;
            else
                return false;
        }

        public void SelectComboBoxItem(AutomationElement element, int indexToBeSelected, AutomationProperty property, object value, TreeScope searchScope)
        {
            var allChildren = this.ExpandComboBoxViewAndReturnChildren(element, property, value, searchScope);
            if (allChildren.Count < indexToBeSelected)
                throw new Exception("The combobox has fewer items than those that need to be selected");
            var itemToBeSelected = allChildren[indexToBeSelected];
            var selectPattern = this.GetSelectionItemPattern(itemToBeSelected, AutomationElement.ControlTypeProperty, ControlType.ListItem, TreeScope.Element);
            if (selectPattern != null)
                selectPattern.Select();
            var togglePattern = this.GetExpandCollapsePattern(element, property, value, TreeScope.Descendants);
            if (togglePattern != null)
                togglePattern.Collapse();
        }

        public AutomationElementCollection ExpandComboBoxViewAndReturnChildren(AutomationElement element, AutomationProperty property, object value, TreeScope searchScope)
        {
            try
            {
                var comboBox = GetFirstChildNode(element, AutomationElement.AutomationIdProperty, value, searchScope);
                var expandPattern = GetExpandCollapsePattern(comboBox, AutomationElement.ControlTypeProperty, ControlType.ComboBox, TreeScope.Element);
                if (expandPattern == null)
                    throw new ElementNotAvailableException("Couldnt Find Expand Pattern in combobox");
                expandPattern.Expand();
                return this.GetAllChildNodes(comboBox, AutomationElement.ControlTypeProperty, ControlType.ListItem, TreeScope.Children);
            }
            catch (Exception e1)
            {
                throw e1;
            }
        }

        public void WorkaroundPopulateControlTree(int x1, int y1)
        {
            uint x = (uint)x1;
            uint y = (uint)y1;
            NativeMethods.SetCursorPos(x1, y1);
            Thread.Sleep(200);
            NativeMethods.mouse_event(NativeMethods.MOUSEEVENTF_LEFTDOWN | NativeMethods.MOUSEEVENTF_LEFTUP, x, y, 0, 0);
        }


        public bool WaitTillElementSelected(AutomationElement aeMainWindow, AutomationProperty automationProperty, object value, TreeScope treeScope)
        {
            var btnStartAnalysis = GetElement(aeMainWindow, automationProperty, value, treeScope);
            int numWait = 0;
            do
            {
                if (btnStartAnalysis.Current.IsEnabled == true)
                    return true;
                Thread.Sleep(2000);
            } while (btnStartAnalysis.Current.IsEnabled == false && numWait < 90);

            return false;
        }
    }

    class NativeMethods
    {

        [DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
        public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);

        public const int MOUSEEVENTF_LEFTDOWN = 0x02;
        public const int MOUSEEVENTF_LEFTUP = 0x04;
        public const int MOUSEEVENTF_RIGHTDOWN = 0x08;
        public const int MOUSEEVENTF_RIGHTUP = 0x10;

        /// Return Type: BOOL->int  
        ///X: int  
        ///Y: int  
        [System.Runtime.InteropServices.DllImportAttribute("user32.dll", EntryPoint = "SetCursorPos")]
        [return: System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.Bool)]
        public static extern bool SetCursorPos(int X, int Y);

    }
}

Running the application launches the console application which enters random inputs and tests the result for all mathematical operations.

The code can be downloaded here.