Exploring test application: IronPython (2)

We demonstrated how to run a .NET application from IronPython in the previous part. However, we couldn't control it while it was running. The solution for this problem is to run the tested application in a separate thread. Thus we will be able to enter commands in IronPython console while the tested application is running. Here is a snippet from the source:

import clr
clr.AddReference('System')
clr.AddReference("System.Windows.Forms")
from System import *
from System.Reflection import *
from System.Threading import *
from System.Windows.Forms import Application
from time import sleep

def RunMeCallBack(var):
    global App
    asm = Assembly.LoadFrom('GUIAT_PoC.exe')
    asm_type = asm.GetType('GUIAT_PoC.frmGUIAT')
    App = Activator.CreateInstance(asm_type)
    Application.Run(App)

App = None
ThreadPool.QueueUserWorkItem(WaitCallback(RunMeCallBack))
while not App:
    sleep(0.2)

The RunMeCallBack function starts the tested application the same way as we showed in the previous part. We create an independent thread and run this function in it so it does not block the console. The thread finishes its work when the tested application terminates (the main form of the application is closed) or when the console is closed.

The important part is line

App = Activator.CreateInstance(asm_type)

Here we remember the instance of the main form in the variable App. We have access to the whole application thanks to the App variable! The while cycle at the end ensures waiting until the App variable is not None. Which only happens when our tested application is up and running.

The App variable is our Holy Grail. Let's explore what is inside:

>>> App
<GUIAT_PoC.frmGUIAT object at 0x000000000000002B
[GUIAT_PoC.frmGUIAT, Text: GUIAT - Proof of Concept]>
>>> App.Text
'GUIAT - Proof of Concept'

Basically, we have access to all public properties and methods. Try dir(App) and you'll see. With a trick, we can even access private properties and methods (using the power of reflection).

To find what components are on the main form, iterate through the Controls collection:

>>> for c in App.Controls:
...   print c.Name, c.GetType()
...
btnAddItem System.Windows.Forms.Button
lblNewItem System.Windows.Forms.Label
txtNewItem System.Windows.Forms.TextBox
lbxItems System.Windows.Forms.ListBox
btnQuit System.Windows.Forms.Button

To find out what text is in the text box, try the following:

>>> App.Controls[2].Text
''

Now, write something directly into the text box in the tested application and call the statement again:

>>> App.Controls[2].Text
'something'

Cool, isn't it? ;-)

Next time, I show you how to simulate user interaction programatically - how to send a text or click to the tested application.

Exploring test application: IronPython (1)

How can IronPython help with exploring the test application? The answer is .NET Reflection:

Reflection is the mechanism of discovering information solely at run time.

We can discover both class and instance information. That's important. Knowing class information, we know what the object is capable of. We know whether it is a button having click method or a text edit having value property. And knowing instance information, we can find out what text the text edit displays.

That's nice but to be able to utilize the reflection, we need access to the tested application's objects from our testing framework. That's easy. The tested application is a .NET application so we can run it directly from IronPython.

When you look to Program.cs (source), you see how the tested application is started. If you don't have the sources of tested application available, you can use Lutz Roeder's .NET Reflector to find the information.

static void Main()
{
   Application.EnableVisualStyles();
   Application.SetCompatibleTextRenderingDefault(false);
   Application.Run(new frmGUIAT());
}

Let's mimic the start in IronPython. We are not interested in the first two rows. The third row is the core. The Application class is part of System.Windows.Forms namespace. So we just need to create an instance of frmGUIAT class. To create it, we use method CreateInstance of Activator class from System namespace. The CreateInstance method needs to know the type of the future instance. We find the type utilizing the reflection.

The whole code looks like this:

import clr
clr.AddReference('System')
clr.AddReference("System.Windows.Forms")
from System import *
from System.Reflection import *
from System.Windows.Forms import Application

asm = Assembly.LoadFrom('GUIAT_PoC.exe')
asm_type = asm.GetType('GUIAT_PoC.frmGUIAT')
App = Activator.CreateInstance(asm_type)
Application.Run(App)

When you run the above code from IronPython console, the tested application is displayed. Note the IronPython must be started form the directiory where GUIAT_PoC.exe is located. The disadvantage is we cannot do anything in IronPython console while the tested application is running. The solution is to run it in separate thread. I will show it in the next article.

Exploring test application: Accessibility

Accessibility looks more promising:

Active Accessibility (AA) is technology developed by Microsoft to make the Windows platform more accessible to people with disabilities. It consists of a set of COM interfaces and methods which allow an application to expose information to third parties, making it easier to implement tools such as magnifiers and screen readers. It proves very useful also for testing utilities. Microsoft has implemented the accessibility interfaces for the standard Windows controls.

AccExplorer (download) shows you the test application structure:



You see nice hierarchical window structure. Every object has several properties:

• Name - e.g. caption of button
• Value - e.g. value of list box
• Role Text - type of the component
• Location - position on the screen
  
• State - e.g. disabled, normal, etc.
• Def Action - default action for the component
  

The properties provides more information than Win32 API. But when you look closer, you see it is not enough:

• We still cannot distinguish the similar controls. Both buttons have role push buttons and can be distinguished only by displayed name. Which is problematic when names change or you need to test localized applications.
• Some components do not have name at all. In our example it is the text box - its name in AccExplorer is NAMELESS.
• The third party components usually lack support for the accessibility in the same quality as Microsoft. For example, DevExpress grid only tells you "I am a grid". It does not tell you how many rows it contains, what cells are there, etc...
  

The good news is developers can change this behavior. They can define AccessibleName property for a component so we can later easily find exactly what we need. If I have set AccessibleName for the text box to myTextBox, we would have seen myTextBox instead of NAMELESS in AccExplorer now. For grid, they can define the whole structure of accessible objects to represent the grid (see Exposing Data Tables through Microsoft Active Accessibility). Unfortunately, when your application does not have sufficient support for accessibility, it is quite lot of work...

Microsoft tries to address testing issues with his new framework in .NET 3.5: Microsoft UI Automation. I have not tried it as our applications do not run on .NET 3.5.

Also DevExpress promised to enhance their components with better support for UI testing tools - see the 2008 roadmap.

But we don't want to wait, right? We want to test our .NET application right now! I will show how in the next article.

Exploring test application: Win32 API

Let's look on what we can discover about our simple test application when it is running. Before IronPython, we had two options:

• Win32 API
• Accessibility

Win32 API provides several functions that return information about an application. First, we have to find the appliction handle. We utilize functions EnumWindows and GetWindowText from Python for Windows extensions project:

def enum(hwnd, extra):
   global app_handle
   if winxpgui.GetWindowText(hwnd).find('GUIAT - Proof') >= 0:
       app_handle = hwnd

app_handle = None
winxpgui.EnumWindows(enum, '')
print 'Test application handle:', app_handle

When we know the application handle, we can enumerate its children:

def enum_child(hwnd, extra):
   print '%s %s:    %s' % (hwnd, winxpgui.GetClassName(hwnd),
           winxpgui.GetWindowText(hwnd))

winxpgui.EnumChildWindows(app_handle, enum_child, '')

When you run the whole script (source), you get:

Test application handle: 2949744
2621886 WindowsForms10.BUTTON.app.0.378734a:    Add Item
7668314 WindowsForms10.STATIC.app.0.378734a:    New listbox item
4194952 WindowsForms10.EDIT.app.0.378734a:
2753142 WindowsForms10.LISTBOX.app.0.378734a:
3342924 WindowsForms10.BUTTON.app.0.378734a:    Quit

Here is the catch. How shall we identify Add Item button from Quit button? Both have the same class name WindowsForms10.BUTTON.app.0.378734a. Sure, the text differs, but this isn't always true. Imagine two text boxes - both having the same class name WindowsForms10.EDIT.app.0.378734a. What then? Of course, you can check positions. But this is leads you back to record/play tools and that is exactly what I want to avoid.

So this is the terminus for Win32 API. Next time - Accessibility.