Tuesday, July 8, 2008

Simulate user's input

We have learned in the previous part how to explore the tested application and read its values. To test it, we also need set values to its fields.

The first idea may be to utilize the .NET objects. When we can read values from the tested application fields, we can also set them. Use the code snippet from the previous part to run the tested application and follow it by command

App.Controls[2].Text = 'GUIATtext'

The tested application is started and GUIATtext appears in the New listbox item text box.

Well, it does not appear as a proper testing to me. I'd rather simulate user's interaction with mouse and keyboard. Unfortunately, sending key strokes and mouse events is not job for .NET framework. We have to dive deep into Windows and use Win32 API to perform these tasks.

I have created a simple DLL to make Win32 API functions available for IronPython [1]. Moreover, I have added functions to simulate user's input. Here is the list of functions in the Win32API namespace with a short description:

        return handle of the active window
        set the active window to the window with handle         return boolean
ShowWindow(handle, state)
        set window with handle to state (minimalized, maximalized, ...)         return boolean
GetWindowText(handle, title, capacity)
        return title of window with handle, title variable type is String of capacity size
MouseClick(X, Y)
        simulate click with left mouse button on position X, Y
        simulate pressing virtual_key
        simulate typing a string

I will not go into details here. Anyone interested can explore the sources of the Win32API.dll.

Using these functions is straightforward in IronPython. Just download the Win32API.dll and look at the example below how to perform a mouse click on the specific position:

import clr
from Win32API import Win32API
Win32API.MouseClick(10, 10)

The code snippet moves mouse to the coordinates 10, 10 (that is to the top-left corner of the screen) and perform a click with the left mouse button. More useful examples will follow in next parts.

Finally, couple of warnings. The simulated mouse click is caught by the top most window on the position. And the simulated key strokes are sent to the active window. That means we have to pay attention to position and active state of the tested application.

In the next part, we start building the GUIAT testing framework.

[1] I do not know how to access Win32 API directly from IronPython.


Mark said...


Had a look at pywinauto? (pywinauto.openqa.org).

I am the author of that :-) It's handling of .NET could be improved for sure.

I think finally the best way to automate the windows UI is to use the Accessibility API (one example is pyAA) but I haven't looked into that in much detail yet.

As far as I know .NET has an automation framework built upon the accessibility features. But I don't have a lot of details on that.

ftorres said...

Very interesting what you do with IronPython for Windows Testing.

Did you ever heard of InCisif.net.
It is a functional web testing tool that support C#, VB.NET and IronPython.

You can find our IronPython demo at

You can download the trial, the IronPython demo is part of it.

Lukas Cenovsky said...

To mark:

Pywinauto looks good - I will definitely look closer on it ;-)

I have explored Accessibility (I've tried pyAA). But with poor results. I describe it a little bit in Exploring test application: Accessibility. The biggest proble is lack of support from third party components.

To ftorress:
I do not focus web testing so I haven't heard of your tool. I am not fan of recording tools, but you approach when you check the id of componetns is good. I am thinking about it too as a future enhancement of my framework.

bitmonster said...


And to give another option that might be interesting on the topic:

I'm the administrator of the project. Its intention is not primarily GUI unit testing but I think many features can be used for it. And missing features can be added through its plugin capabilities.

It is not written in IronPython (mostly CPython/wxPython), but I often thought about porting it over to IronPython instead of using wxPython. But it would be much work of course.

As you quickly see, you set up macros in EventGhost mainly through its GUI, but every action can also be used from Python scripts (and these also can be inserted/edited through the EventGhost GUI)

Especially interesting might be the "Window/Find Window" action for you. It does filter GUI handles by different properties (window name, window class, process name, etc.) and then optionally simply uses a fixed index into this filtered list. So if a frame has 10 TextEdit boxes with the same class/name, you can say you want to target the second one for example, as the order of the handles in a typical Windows application seldom changes.

jagallout said...

I am working in IronPython to create a simple windows "macro". I am attempting to import the win32api that you reference for its ability to simulate keypresses, but get an error:

"System.IO.IOException: Could not add reference to assembly Win32API"
The line with clr.AddReference is where the script is breaking.

import clr
from Win32API import Win32API

Any ideas?