Exploring test application: Accessibility

Accessibility looks more promising:

Active Accessibility (AA) is technology developed by Microsoft to make the Windows platform more accessible to people with disabilities. It consists of a set of COM interfaces and methods which allow an application to expose information to third parties, making it easier to implement tools such as magnifiers and screen readers. It proves very useful also for testing utilities. Microsoft has implemented the accessibility interfaces for the standard Windows controls.

AccExplorer (download) shows you the test application structure:



You see nice hierarchical window structure. Every object has several properties:

• Name - e.g. caption of button
• Value - e.g. value of list box
• Role Text - type of the component
• Location - position on the screen
  
• State - e.g. disabled, normal, etc.
• Def Action - default action for the component
  

The properties provides more information than Win32 API. But when you look closer, you see it is not enough:

• We still cannot distinguish the similar controls. Both buttons have role push buttons and can be distinguished only by displayed name. Which is problematic when names change or you need to test localized applications.
• Some components do not have name at all. In our example it is the text box - its name in AccExplorer is NAMELESS.
• The third party components usually lack support for the accessibility in the same quality as Microsoft. For example, DevExpress grid only tells you "I am a grid". It does not tell you how many rows it contains, what cells are there, etc...
  

The good news is developers can change this behavior. They can define AccessibleName property for a component so we can later easily find exactly what we need. If I have set AccessibleName for the text box to myTextBox, we would have seen myTextBox instead of NAMELESS in AccExplorer now. For grid, they can define the whole structure of accessible objects to represent the grid (see Exposing Data Tables through Microsoft Active Accessibility). Unfortunately, when your application does not have sufficient support for accessibility, it is quite lot of work...

Microsoft tries to address testing issues with his new framework in .NET 3.5: Microsoft UI Automation. I have not tried it as our applications do not run on .NET 3.5.

Also DevExpress promised to enhance their components with better support for UI testing tools - see the 2008 roadmap.

But we don't want to wait, right? We want to test our .NET application right now! I will show how in the next article.

Exploring test application: Win32 API

Let's look on what we can discover about our simple test application when it is running. Before IronPython, we had two options:

• Win32 API
• Accessibility

Win32 API provides several functions that return information about an application. First, we have to find the appliction handle. We utilize functions EnumWindows and GetWindowText from Python for Windows extensions project:

def enum(hwnd, extra):
   global app_handle
   if winxpgui.GetWindowText(hwnd).find('GUIAT - Proof') >= 0:
       app_handle = hwnd

app_handle = None
winxpgui.EnumWindows(enum, '')
print 'Test application handle:', app_handle

When we know the application handle, we can enumerate its children:

def enum_child(hwnd, extra):
   print '%s %s:    %s' % (hwnd, winxpgui.GetClassName(hwnd),
           winxpgui.GetWindowText(hwnd))

winxpgui.EnumChildWindows(app_handle, enum_child, '')

When you run the whole script (source), you get:

Test application handle: 2949744
2621886 WindowsForms10.BUTTON.app.0.378734a:    Add Item
7668314 WindowsForms10.STATIC.app.0.378734a:    New listbox item
4194952 WindowsForms10.EDIT.app.0.378734a:
2753142 WindowsForms10.LISTBOX.app.0.378734a:
3342924 WindowsForms10.BUTTON.app.0.378734a:    Quit

Here is the catch. How shall we identify Add Item button from Quit button? Both have the same class name WindowsForms10.BUTTON.app.0.378734a. Sure, the text differs, but this isn't always true. Imagine two text boxes - both having the same class name WindowsForms10.EDIT.app.0.378734a. What then? Of course, you can check positions. But this is leads you back to record/play tools and that is exactly what I want to avoid.

So this is the terminus for Win32 API. Next time - Accessibility.

GUI Application to Test

Let's start with something easy. We have a Windows GUI application that looks like this:


The application contains a text box, two buttons, and a list box. It simply adds a text from text box as the last item to the list box (exe, source).

The testing steps might be as follows:

1. Run the application.
2. Check the 'New listbox item' fields is empty.
3. Check the list box contains three rows with texts: 'This is', 'GUIAT', 'demo.'.
4. Enter a text (e.g. 'new line') into text box, press 'Add Item' button. Verify your text ('new line') appears as the fourth line/item in the list box.
5. Press 'Quit' button. Verify the application terminates.

Any record/play testing tool can do the test up to point 4. There it cannot verify whether the text was added to the correct place in the list box. These tools also cannot handle moving 'Add Item' button closer to the text box. They would then click to empty place. Sure there are more sophisticated tools that can handle the above problems. But I do not know about any for free.

Next time we try some dead ends from my beginnings :-)

The beginning

Testing GUI application can be tedious work. Usually, you have to manually click through the interfaces. The first round is fun, the second is OK, and from the third round on it is boring.

When I joined Radiant, I started to think about automation. I searched on Internet but I did not find anything satisfying. All tools were basically record/play applications. That is not sufficient for testing. There are many concerns, for example:

• How do you verify the test result?
• How easy can you change the script (add testing of a new field, new form, etc.)
  
• Scalability - how easy can you join basic scripts and create bigger (regression) script?
• How the test application works with your amended components?
• etc.
  

I know the testing tool cannot do everything. So I set up several goals I'd like to achieve:

• Allow to control GUI application.
  That means to start it, navigates through it, close it. To know its status.
• Allow to read values from the screen.
  You have to know what is on the screen to be able to test it.
• Allow to set values.
  Also, you have to be able to set/write values into the application to be able to test it.
• Easy to create.
  Testers are not experienced programmers. The tool must be easy to use. Anybody, who knows what a program is, should be able to write a test script.
  
• Easy to maintain.
  When the tested application changes, amending a test script should not be necessary or be a piece of cake.
  
• Scalability.
  Joining test scripts together, creating regression scripts.

In next few weeks, you'll find out on this blog how I succeed in this task.