GUI Automated Testing

WCF Service in pure IronPython with config file

noreply@blogger.com (Lukáš Čenovský) — Sun, 22 Nov 2009 19:28:00 +0000

I was wrong when I wrote in the last post that the IronPython service cannot be saved into an assembly. It can. Which opens a way to use .config file to configure the service.

This is a simple config file for the service:

ConfigService.exe.config




    
      
        
          
            
          
        
                    binding="basicHttpBinding"
            contract="TestServiceInterface.ImyService"/>

The interface is the same as in the previous version. The only difference in the service to the previous version is in the ServiceHost initialization - we omit the service configuration parameters because they are in the .config file. I also changed the clr namespace:

ConfigService.py

import clr
import clrtype
clr.AddReference('System.ServiceModel')
from TestServiceInterface import ImyService
from System import Console, Uri
from System.ServiceModel import (ServiceHost,
        BasicHttpBinding, ServiceBehaviorAttribute,
        InstanceContextMode)

class myService(ImyService):
    __metaclass__ = clrtype.ClrClass
    _clrnamespace = "ConfigService"
    _clrclassattribs = [ServiceBehaviorAttribute]

    def GetData(self, value):
        return "IronPython config service: You entered: %s" % value

sh = ServiceHost(myService)
sh.Open()
Console.WriteLine("Press  to terminate\n")
Console.ReadLine()
sh.Close()

If you want to run this script, you must save the ConfigService.exe.config as ipy.exe.config to the folder with the IronPython interpreter ipy.exe.

To save the service as an assembly, run the following command:

C:\IronPython-2.6\ipy.exe C:\IronPython-2.6\Tools\Scripts\pyc.py  /out:ConfigService /target:exe /main:ConfigService.py clrtype.py TestServiceInterface.py

The ConfigService.dll and ConfigService.exe are created. Add the ConfigService.exe.config to the same folder and when you run ConfigService.exe, the service starts. Note you also need all IronPython .dlls in the same folder.

You can adjust the .config file to expose a MEX endpoint (ConfigService.mex.exe.config) but I don't see a big point in it because svcutil.exe generates C# or VB code. Anyway - here are the generated files: myService.cs, myService.config

You can run the old TestClient.py and it will successfully retrieve value from the service. But the old TestClient.py does not use .config file. If we want to use .config file for the client, we have to rewrite the WCF client. First, here is the sample client .config file:

ConfigClient.exe.config



  
    
                    binding="basicHttpBinding"
            contract="TestServiceInterface.ImyService"/>

You can see it is very similar to the generated one. We do not specify details of the binding but we specify full name of the contract interface.

If you check the generated client proxy class by svcutil.exe, you see it is based on System.ServiceModel.ClientBase and the interface ImyService. There are some empty constructors and all methods from ImyService interface return result of the same method name call on Channel property. That's why I have created WcfClient helper function. The client source then looks like the following:

ConfigService.py

import clr
clr.AddReference('System.ServiceModel')
import System.ServiceModel
from TestServiceInterface import ImyService

def WcfClient(interface):

    class WcfClientBase(System.ServiceModel.ClientBase[interface]):

        def __getattr__(self, name):
            # if name is method from interface, return the Channel method
            if name in (k[0] for k in interface.emitted_methods.keys()):
                return getattr(self.Channel, name)

    return WcfClientBase()

wcfcli = WcfClient(ImyService)
print "WCF config client returned:\n%s" % wcfcli.GetData(11)

The WcfClient helper function returns an instance of class based on System.ServiceModel.ClientBase. The __getattr__ checks if the requested attribute name is interface method and if so, it returns the Channel's method with the same name. Which is the same behavior as the generated client proxy class in couple of lines of code.

To save the client as an assembly, run the following command:

C:\IronPython-2.6\ipy.exe C:\IronPython-2.6\Tools\Scripts\pyc.py   /out:ConfigClient /target:exe /main:ConfigClient.py clrtype.py TestServiceInterface.py

The ConfigClient.dll and ConfigClient.exe are created. Add the ConfigClient.exe.config to the same folder and when you run ConfigClient.exe, the client calls the service.

Having this I think there is only a small step to use the IronPython WCF services in IIS. Unfortunately, I do not know how to do it...

WCF Service in pure IronPython

noreply@blogger.com (Lukáš Čenovský) — Tue, 17 Nov 2009 12:07:00 +0000

I wrote about implementing WCF service in IronPython a couple of weeks ago. Meanwhile I pushed Shri a little bit with the clrtype.py and he has implemented ClrInterface metaclass there so we can create the whole WCF service in IronPython now.

The IronPython interface implementation is straightforward:

TestServiceInterface.py

import clr
import clrtype
clr.AddReference('System.ServiceModel')
from System.ServiceModel import (
        ServiceContractAttribute,
        OperationContractAttribute)
OperationContract = clrtype.attribute(
        OperationContractAttribute)

class ImyService(object):
    __metaclass__ = clrtype.ClrInterface
    _clrnamespace = "TestServiceInterface"
    _clrclassattribs = [ServiceContractAttribute]

    @OperationContract()
    @clrtype.accepts(int)
    @clrtype.returns(str)
    def GetData(self, value):
        raise RuntimeError("this should not get called")

Also switching from C# interface to IronPython interface is easy:

TestService.py

import clr
import clrtype
clr.AddReference('System.ServiceModel')
from TestServiceInterface import ImyService
from System import Console, Uri
from System.ServiceModel import (ServiceHost,
        BasicHttpBinding, ServiceBehaviorAttribute,
        InstanceContextMode)

class myService(ImyService):
    __metaclass__ = clrtype.ClrClass
    _clrnamespace = "myWcfService"
    _clrclassattribs = [ServiceBehaviorAttribute]

    def GetData(self, value):
        return "IronPython: You entered: %s" % value

sh = ServiceHost(myService, Uri(
        "http://localhost:9000/myWcfService"))
sh.AddServiceEndpoint(clr.GetClrType(ImyService),
        BasicHttpBinding(), "")
sh.Open()
Console.WriteLine("Press  to terminate\n")
Console.ReadLine()
sh.Close()

Notice that we call ServiceHost with myService which is the type and not the instance of our service. Because of this, the ServiceBehavior attribute does not need to have InstanceContextMode.Single parameter.

Finally, here is the test client:

TestClient.py

import clr
clr.AddReference('System.ServiceModel')
import System.ServiceModel
from TestServiceInterface import ImyService

mycf = System.ServiceModel.ChannelFactory[ImyService](
        System.ServiceModel.BasicHttpBinding(),
        System.ServiceModel.EndpointAddress(
            "http://localhost:9000/myWcfService"))
wcfcli = mycf.CreateChannel()
print "WCF service returned:\n%s" % wcfcli.GetData(11)

The disadvantage of having just a single service instance is gone, ~~the harder configuration remains~~. One new disadvantage can be ~~it is not possible (yet) to compile the interface and save it to disk~~ nor use it from other .NET languages.

Edit 22. 11. 2009: See WCF Service in pure IronPython with config file

INotifyPropertyChanged and databinding in Silverlight

noreply@blogger.com (Lukáš Čenovský) — Mon, 16 Nov 2009 10:14:00 +0000

In the previous article, I wrote about IronPython and databinding in WPF applications. The last note was it does not work in Silverlight. Thanks to Shri Borde (IronPython/IronRuby dev lead) who updated clrtype module, the note is not true any more.

Let's create a small Silverlight app in IronPython from scratch. I use IronPython 2.6 RC2. Follow http://lists.ironpython.com/pipermail/users-ironpython.com/2009-October/011543.html to avoid bugs in IronPython 2.6 RC2.

Create a new project:

C:\IronPython-2.6\Silverlight\script\sl.bat python BindTest

Change the app.xaml to

    xmlns="http://schemas.microsoft.com/client/2007"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
            Orientation="Horizontal">
        
        
                    Text="{Binding size, Mode=TwoWay}" />

The difference comparing to WPF version is we have to specify binding mode because the default mode for TextBox in Silverlight is OneWay. And we cannot use UpdateSourceTrigger=PropertyChanged because Silverlight does not have such UpdateSourceTrigger.

Silverlight binding is limited comparing to WPF. That's why we have to create CLR properties to Silverlight be able to see them. DevHawk has a nice serie about clr types on his blog.

Creating CLR property with clrtype.py is easy. Shri described it on IronPython mailing list. Because I use my enhanced @notify_property decorator, I can write:

class ViewModel(NotifyPropertyChangedBase):
    __metaclass__ = clrtype.ClrClass
    _clrnamespace = "BindTest"
    
    def __init__(self):
        super(ViewModel, self).__init__()
        # must be string to two-way binding work
        # correctly
        self.size = '10'

    @notify_property
    @clrtype.returns(str)
    def size(self):
        return self._size

    @size.setter
    @clrtype.accepts(str)
    def size(self, value):
        self._size = value
        print 'Size changed to %r' % self.size

The NotifyPropertyChangedBase class is the same as for WPF version. The enhanced @notify_property decorator calls automatically clrtype.accepts() for getter and clrtype.returns() for setter so we do not need to call them manually for every property:

class notify_property(property):

    def __init__(self, getter):
        def newgetter(slf):
            #return None when the property does not
            # exist yet
            try:
                return getter(slf)
            except AttributeError:
                return None
        getter = clrtype.accepts()(getter)
        clrtype.propagate_attributes(getter, newgetter)
        super(notify_property, self).__init__(newgetter)

    def setter(self, setter):
        def newsetter(slf, newvalue):
            # do not change value if the new value is
            # the same, trigger PropertyChanged event
            # when value changes
            oldvalue = self.fget(slf)
            if oldvalue != newvalue:
                setter(slf, newvalue)
                slf.OnPropertyChanged(setter.__name__)
        setter = clrtype.returns()(setter)
        clrtype.propagate_attributes(setter, newsetter)
        return property(
            fget=self.fget,
            fset=newsetter,
            fdel=self.fdel,
            doc=self.__doc__)

Then App looks similarly to the WPF counterpart:

class App:
    def __init__(self):
        self._vm = ViewModel()
        self.root = Application.Current.LoadRootVisual(
                UserControl(), "app.xaml")
        self.DataPanel.DataContext = self._vm
        self.Button.Click += self.OnClick

    def OnClick(self, sender, event):
        # must be string to two-way binding work
        # correctly
        self._vm.size = '10'

    def __getattr__(self, name):
        # provides easy access to XAML elements
        # (e.g. self.Button)
        return self.root.FindName(name)

a = App()

Run Chiron with the BindTest app

C:\IronPython-2.6\Silverlight\script\sl.bat python BindTest

and check the application in the browser on http://localhost:2060/index.html.

Whatever you write into the text box appears in the label in front of the text box when the text box loses the focus. When you click the button, the value is reseted. You can also change the value from the console:

a._vm.size= '3'

Download app.xaml and app.py. You also need clrtype.py and pyevent.py (from C:\IronPython-2.6\Tutorial\pyevent.py) in the BindTest folder.

INotifyPropertyChanged and databinding in IronPython WPF

noreply@blogger.com (Lukáš Čenovský) — Wed, 11 Nov 2009 11:22:00 +0000

INotifyPropertyChanged is important interface for building WPF or Silverlight applications using M-V-VM concept (MSDN article).

In simple language, you have a Model which provides access to your data (e.g in database, files, web, etc.). Then you have a ViewModel that access data in the Model via Model's interface and provides data to a View which is XAML file with UI layout. Linkage between ViewModel and View is done by binding that utilizes PropertyChanged event to properly update all UI elements.

I have found two examples how to implement INotifyPropertyChanged interface in IronPython. The first one uses __setattr__ hook. Personally, I don't like it - it is not clear and easily readable code. The second one is better because it uses properties. But you have to write self.OnPropertyChanged("my_property_name") for every property. Not ideal.

That's why I sit down and write a notify_property:

class notify_property(property):

    def __init__(self, getter):
        def newgetter(slf):
            try:
                return getter(slf)
            except AttributeError:
                return None
        super(notify_property, self).__init__(newgetter)

    def setter(self, setter):
        def newsetter(slf, newvalue):
            oldvalue = self.fget(slf)
            if oldvalue != newvalue:
                setter(slf, newvalue)
                slf.OnPropertyChanged(setter.__name__)
        return property(
            fget=self.fget,
            fset=newsetter,
            fdel=self.fdel,
            doc=self.__doc__)

With this subclass I aimed several goals:

usage simple as @property decorator (actualy no other usage is possible as I implemented __init__ with just one parameter that must be the getter)
when property is on yet defined, it should return None
automaticaly handle PropertyChanged event when and only when property has changed

We also need to implement INotifyPropertyChanged interface in IronPython so we can call OnPropertyChanged method. See Overiding events in IronPython\Doc\dotnet-integration.rst to understand what means add_ and remove_ methods.

class NotifyPropertyChangedBase(INotifyPropertyChanged):
    PropertyChanged = None

    def __init__(self):
        self.PropertyChanged, self._propertyChangedCaller = pyevent.make_event()

    def add_PropertyChanged(self, value):
        self.PropertyChanged += value

    def remove_PropertyChanged(self, value):
        self.PropertyChanged -= value

    def OnPropertyChanged(self, propertyName):
        if self.PropertyChanged is not None:
            self._propertyChangedCaller(self, PropertyChangedEventArgs(propertyName))

Now we can implement a simple class with properties with change notification:

class DataObject(NotifyPropertyChangedBase):
    
    def __init__(self, size):
        super(DataObject, self).__init__()
        self.size = size

    @notify_property
    def size(self):
        return self._size

    @size.setter
    def size(self, value):
        self._size = value

You can see it is very easy - just like any other property in Python.

Finaly, let's put all together. When you run the code below, it shows a window with label, textbox and button. The label is updated as you type into the textbox and a message is written into the console as well. By default, the textbox is updated when it looses focus, so I have to change UpdateSourceTrigger to PropertyChanged. When you click the button, the value is reset. Note if you use type int instead of string the two-way bindign would not work.

notpropwpf.py

import clr
import System
clr.AddReference('PresentationFramework')
clr.AddReference('PresentationCore')

from System.Windows.Markup import XamlReader
from System.Windows import Application, Window
from System.ComponentModel import INotifyPropertyChanged, PropertyChangedEventArgs
import pyevent

XAML_str = """    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" Width="250" Height="62">
    
        
        
        
        
    
"""

class notify_property(property):

    def __init__(self, getter):
        def newgetter(slf):
            #return None when the property does not exist yet
            try:
                return getter(slf)
            except AttributeError:
                return None
        super(notify_property, self).__init__(newgetter)

    def setter(self, setter):
        def newsetter(slf, newvalue):
            # do not change value if the new value is the same
            # trigger PropertyChanged event when value changes
            oldvalue = self.fget(slf)
            if oldvalue != newvalue:
                setter(slf, newvalue)
                slf.OnPropertyChanged(setter.__name__)
        return property(
            fget=self.fget,
            fset=newsetter,
            fdel=self.fdel,
            doc=self.__doc__)

class NotifyPropertyChangedBase(INotifyPropertyChanged):
    PropertyChanged = None

    def __init__(self):
        self.PropertyChanged, self._propertyChangedCaller = pyevent.make_event()

    def add_PropertyChanged(self, value):
        self.PropertyChanged += value

    def remove_PropertyChanged(self, value):
        self.PropertyChanged -= value

    def OnPropertyChanged(self, propertyName):
        if self.PropertyChanged is not None:
            self._propertyChangedCaller(self, PropertyChangedEventArgs(propertyName))

class ViewModel(NotifyPropertyChangedBase):
    
    def __init__(self):
        super(ViewModel, self).__init__()
        # must be string to two-way binding work correctly
        self.size = '10'

    @notify_property
    def size(self):
        return self._size

    @size.setter
    def size(self, value):
        self._size = value
        print 'Size changed to %r' % self.size

class TestWPF(object):

    def __init__(self):
        self._vm = ViewModel()
        self.root = XamlReader.Parse(XAML_str)
        self.DataPanel.DataContext = self._vm
        self.Button.Click += self.OnClick
        
    def OnClick(self, sender, event):
        # must be string to two-way binding work correctly
        self._vm.size = '10'

    def __getattr__(self, name):
        # provides easy access to XAML elements (e.g. self.Button)
        return self.root.FindName(name)

tw = TestWPF()
app = Application()
app.Run(tw.root)

You need pyevent.py from IronPython\Tutorial\ folder to run to example.

Unfortunately, this does not work in Silverlight, probably because the property is not .NET field.

WCF Service in IronPython

noreply@blogger.com (Lukáš Čenovský) — Fri, 30 Oct 2009 17:08:00 +0000

Edit 17. 11. 2009: See the article about WCF service in pure IronPython.

Until IronPython 2.6, it was not possible to create WCF service host in pure IronPython. The closest way was to create stub in C# and subclass it in IronPython or create the whole service in C# and run it from IronPython. It is now much simpler with IronPython 2.6 although you still have to write a little C# code.

Simple WCF service implemented in C# looks like this:

TestServiceInterface.cs

using System;
using System.ServiceModel;

namespace TestServiceInterface
{
    [ServiceContract]
    public interface ImyService
    {
        [OperationContract]
        string GetData(int value);
    }
}

TestService.cs

using System;
using System.ServiceModel;
using TestServiceInterface;

namespace myWcfService
{
    public class myService : ImyService
    {
        public string GetData(int value)
        {
            return string.Format("You entered: {0}", value);
        }
    }

    public class mySvc
    {
        public static void Main()
        {
            ServiceHost sh = new ServiceHost(
     typeof(myService),
     new Uri("http://localhost:9000/myWcfService"));
   sh.AddServiceEndpoint(
     typeof(ImyService),
     new BasicHttpBinding(),
        "");
            sh.Open();
            Console.WriteLine("Press  to terminate\n");
            Console.ReadLine();
            sh.Close();
        }
    }
}

You build it:

csc /target:library TestServiceInterface.cs
csc /r:TestServiceInterface.dll TestService.cs

The reason I put TestServiceInterface into separate file is that you cannot create interfaces in IronPython. So this is the only part written in C# when implementig WCF service in IronPython.

The implementation then looks like this:

TestService.py

import clr
import clrtype
clr.AddReference('System.ServiceModel')
clr.AddReference('TestServiceInterface')
from TestServiceInterface import ImyService
from System import Console, Uri
from System.ServiceModel import (ServiceHost, BasicHttpBinding,
        ServiceBehaviorAttribute, InstanceContextMode)
ServiceBehavior = clrtype.attribute(ServiceBehaviorAttribute)

class myService(ImyService):
    __metaclass__ = clrtype.ClrMetaclass
    _clrnamespace = "myWcfService"
    _clrclassattribs = [
            ServiceBehavior(InstanceContextMode=InstanceContextMode.Single)]

    def GetData(self, value):
        return "IronPython: You entered: %s" % value

sh = ServiceHost(
        myService(),
        Uri("http://localhost:9000/myWcfService")
    )
sh.AddServiceEndpoint(
        clr.GetClrType(ImyService),
        BasicHttpBinding(),
        "")
sh.Open()
Console.WriteLine("Press  to terminate\n")
Console.ReadLine()
sh.Close()

The myService class must have InstanceContextMode.Single ServiceBehavior attribute because we are passing service instance to the ServiceHost constructor. This is done via new __clrtype__ metaclass. See the error if we don't use the attribute. I was not able to pass type into the ServiceHost constructor.

To test the service, you can use C# or IronPython client implementation:

TestClient.cs

using System;
using System.ServiceModel;
using TestServiceInterface;

namespace myWcfClient
{
    public class cli
    {
        public static void Main()
        {
   ChannelFactory mycf = new ChannelFactory(
     new BasicHttpBinding(),
        new EndpointAddress("http://localhost:9000/myWcfService"));
   ImyService wcfcli = mycf.CreateChannel();
   Console.WriteLine("Calling GetData(33) returns:\n{0}", wcfcli.GetData(33));
        }
    }
}

TestClient.py

import clr
clr.AddReference('System.ServiceModel')
import System.ServiceModel
clr.AddReference('TestServiceInterface')
from TestServiceInterface import ImyService

mycf = System.ServiceModel.ChannelFactory[ImyService](
        System.ServiceModel.BasicHttpBinding(),
        System.ServiceModel.EndpointAddress("http://localhost:9000/myWcfService"))
wcfcli = mycf.CreateChannel()
print "WCF service returned:\n%s" % wcfcli.GetData(11)

Disadvantages:

You can have only single instance of the service because you are passing the service instance instead of service type.
You cannot easily use .config file to configure your service.

Note: If you see closing tags (e.g. enter or ImyService) at the end of listings, it looks like SyntaxHighlighter bug in Google Chrome...

How to clone VirtualBox virtual machine

noreply@blogger.com (Lukáš Čenovský) — Thu, 24 Sep 2009 21:53:00 +0000

VirtualBox is great piece of software for virtualization. I like it more than VMWare or VirtualPC. The downside of it is its lack of easy cloning or copying capabilities. There are none. And because everything in VirtualBox has UUID (guid), pure file copy does not work well.

All advices I have seen were about: create the new virtual machine manually, clone HDD .vdi file with VBoxManage clonevdi command and attach it to the newly created virtual machine. Manually!?

That's why I've created a simple Python script that cares about exporting and importing virtual machines (Thanks for the VirtualBox 3 SDK!). When exporting, you can select the virtual machine and snapshot you want to export:

The exported machines are stored in exp folder at the script file. The time stamp is added to the machine name for unique identification. When importing, you are offered with the saved virtual machines and you simply choose one. Of course you can copy the exported machine to different computer (having the same VirtualBox versions is recommended).

The script is very simple and targeted to my needs. That means it exports only primary HDD and network adapter 0. It does not export CD/DVD-ROM, Floppy nor Serial ports.

Try it and let me know whether it's helpful: download script (it requires ConfigObj 4).

Update 1. 10. 2009: I struggled running the script on my new Win 7 64-bit system. It complained class is not registered for VirtualBox.Session. The reason was mixing 32-bit and 64-bit programs - see the explanation. If you use 64-bit OS, you have to have installed 64-bit version of Python and pywin32!

Update 11. 11. 2009: I have updated the script to better serve my needs. For example, when restoring virtual machine, you can enter a new name for it. Be carefull when restoring one saved machine to two new ones as you have to chagne MAC address manually.

Unfortunately, the old config file is not compactible with the new version. But I think you can guess from the Python exception what is wrong with the config file and fix it. The old version is still available.

Pycon 2009: Functional Testing of Desktop Applications

noreply@blogger.com (Lukáš Čenovský) — Sat, 16 May 2009 16:26:00 +0000

Here is a link to Michael Foord talk at Pycon 2009 in Chicago about functional testing of desktop applications:

Functional Testing of Desktop Applications
Many advocates of unit testing believe that testing desktop (GUI) applications is so difficult as to not be worth it. Not only is it possible, but 'functional testing' (testing an application from the point of view of the user) is an important part of an agile development process.

I watched it yesterday and it is very good!

Testing an unknown application

noreply@blogger.com (Lukáš Čenovský) — Fri, 15 May 2009 15:25:00 +0000

From time to time you may need to test an unknown application. I have encounter it when I needed to test our installation program. The task was simple - install the application for automated smoke tests. In such cases I gladly return to Accessibility.

Let's pretend our testing application is not written in .NET and the only way how to explore it is through the accessibility (AccExplorer):

All we need to do is to access Accessibility objects from IronPython. There is a project called Managed Windows API that nicely wraps accessibility for .NET. It also wraps other Win32 API calls (mouse clicks etc.) but I stay with Win32API.dll because of my laziness :-).

Utilizing the Managed Windows API you can control the accessibility objects. Here is the snippet:

import clr
# Win32API provide access to Win32 API functions
clr.AddReference('Win32API')
from Win32API import Win32API
# ManagedWinapi provide access to Accessibility objects
clr.AddReference('ManagedWinapi')
import ManagedWinapi.Accessibility as ma
import ManagedWinapi.Windows as mw

def GetGUIATWindow():
   """ Return GUIAT_PoC window accessibility object. """
   def callback(aWindow):
       return aWindow.Title == 'GUIAT - Proof of Concept'
   guiat_window = mw.SystemWindow.FilterToplevelWindows(callback)
   # assume guiat_window is list with just one object
   return ma.SystemAccessibleObject.FromWindow(guiat_window[0],
       ma.AccessibleObjectID.OBJID_WINDOW)

guiat_acc = GetGUIATWindow()
# position of the 'New listbox item' text box
pos = guiat_acc.Children[3].Children[1].Location
# focus the text box
Win32API.MouseClick(pos.X + pos.Width/2, pos.Y + pos.Height/2)
Win32API.SendString('Accessibility test')
# position of the 'Add Item' button
pos = guiat_acc.Children[3].Children[2].Location
# click the button
Win32API.MouseClick(pos.X + pos.Width/2, pos.Y + pos.Height/2)

To run it, download managedwinapi-0.3.zip and extract the ManagedWinapi.dll to the same folder as the source code.

When you run the code, it enters Accessibility test text into New listbox item text box and clicks Add Item button. The controls are always on the same position within the Children enumeration so we can use direct referencing: e.g. guiat_acc.Children[3].Children[2] is the button accessibility object.

Enhancing this example is up to you - I have created several functions that take care about entering the text into text boxes, selecting buttons or a checking check boxes. That's all I need to create a script that installs our application.

Building the framework (3)

noreply@blogger.com (Lukáš Čenovský) — Thu, 30 Oct 2008 07:50:00 +0000

After a while, it's time to continue building the GUI Automated Testing framework. Today, I focus on a list box. You can download the source as usual.

I assume standard windows list box component where only unique items are stored and only one item can be selected. These two conditions are set only because of simplicity of the example. If you don't like them, enhance the example by yourself as your homework :-)

The list box component is again a subclass of BaseComponent. It is quite simple - it has one method and two properties:

Select(self, aItem)
The method selects aItem in the list box.

def Select(self, aItem):
  """ select an item from list box
  @aItem - string with item to select
  """
  if aItem not in self.items:
    raise Exception("Item '%s' not in list box" % aItem)
  self.guiat.Activate()
  self._CheckVisibility()
  pos = self.location
  # click on the first item to focus the list box
  Win32API.MouseClick(pos[0] + (pos[2]/2), pos[1] + 3)
  # send Home to be sure we are on the first item
  # (we could be scrolled down a little)
  Win32API.SendKey(Win32API.VK_HOME)
  # simulate pressing down arrow until we find the item
  # we should find it because it is among self.items
  while self.value != aItem:
    Win32API.SendKey(Win32API.VK_DOWN)

First, it checks whether aItem is in the list box and raises an exception if not. Then it clicks onto the first visible list box item, simulates pressing Home key to focus the first item and repeatedly press Down key until the value equals aItem.
This approach has the advantage we don't need to care about scroll box. The basic windows list box does not provide nice properties or methods to return its state. We would have to dive into Win32 API to find it. For example, the DevExpress ListBoxControl has method GetViewInfo that returns information about internal list box state.

This brings us to the important note:

We do not test the list box component. We test the application.

One way of selecting item in a list box is enough. Of course, we must be aware of its limitations and side effects. Selecting an item with our GUIAT component fires one OnClick and several OnChange events.

items
The property contains read only list of all items in the list box.

value
The property contains the selected item in the list box. User can assign a string to it to select the string in the list box.

Incorporating the new GUIAT ListBox class into the framework is easy - just add it into the RecognizableComponents dictionary of the Form class.

This is the last post about controlling components. You know the idea so developing a new GUIAT class controlling your component should be easy.

Next time, I show how to control application that is already started.

GUI Testing at Resolver Systems

noreply@blogger.com (Lukáš Čenovský) — Thu, 25 Sep 2008 22:55:00 +0000

I'm currently quite busy at work and with my other activities (like photos). That's why instead of writing another chapter of the GUI Automated Testing framework, I point you to the great article of Michael Foord:

Functional Testing of GUI Applications

Michael describes his experiences with GUI automated testing in Resolver Systems.

They use quite similar approach to run the tested application. It is run in another thread but the thread invoking is different. I am not a specialist in threading but I will try to find out more about which way is better or whether there are any catches[1].

They also do not use win32api for simulating user's input. They usually call directly the .NET method responsible for generating the requested event (e.g. PerformClick). That can be dangerous because your test can pass even when user is unable to click the button - the button could be hidden behind another component.

[1] My current problem can be caused by threading. When I run our application from automated testing tool, two buttons overlap. When I run it normally, it is OK. To be it stranger, it occurs only on my PC. I have no idea why...

Update 10/30/2008: I have not found any problem with my way of starting the tested application. I asked my co-workers, searched for articles about threading and found both ways should be similar. [1] is still mystery to me, though.

Update 8/20/2009: Mystery [1] solved - it was bug in our application :-)

Building the framework (2)

noreply@blogger.com (Lukáš Čenovský) — Sun, 17 Aug 2008 22:30:00 +0000

Last time we have started building the GUI Automated Testing framework. Today we are going to enhance it. Download the source to be able to follow the text.

The first version implemented the BaseComponent class to represent windows components. We could not do anything but find out location of the component. It would be nice to be able write text into a text box or click a button by simple method, wouldn't it?

To do that, we need to know the type of each component. The type name of .NET components is stored in component.GetType().Name. Knowing the type, we create subclass of BaseComponent for each type. In the subclass, we provide nice methods and properties.

First, we enhance BaseComponent class. We add _CheckVisibility(self) method that checks whether the component is visible. If not, it raises an exception. We need it for checking if button is visible so we can click it or text box is visible to write it. And we add guiat property to be able to activate the tested application from a component by calling self.guiat.Activate().

Let's look on a button. The Button class is very simple:

from time import sleep
from Win32API import Win32API
from BaseComponent import BaseComponent

class Button(BaseComponent):
   """interface to the Button component"""

   def Click(self):
       """ perform left mouse click on the center of the button, no parameters"""
       self.guiat.Activate()
       self._CheckVisibility()
       pos = self.location
       Win32API.MouseClick(pos[0] + (pos[2]/2), pos[1] + (pos[3]/2))
       sleep(0.1)

The Click(self) method activates the tested application, checks whether the component is visible, finds out the position of the button, clicks into the middle of the button position, and finally waits a little bit. I do not like the waiting but it is here for safety. Windows may repainting some areas or do some other cool thinks, so it is better to give them some time for it.

The TextBox class provides three methods and one property:

FocusEditor(self)
The method essentially does the same as Button's Click method. It activates the tested application, checks whether the component is visible, and clicks into the middle of the TextBox position to focus the editor.
Clear(self)
The method focuses the text box, moves cursor to the start position simulating pressing Home key, and simulates pressing Delete key until the text in text box is empty. If the text is not empty then, it raises exception.
Write(self, aString)
The method clears the text box and uses Win32API method SendString to simulate typing aString. Then checks whether the text box contains the aString value and raises exception if not.
value
The property contains the actual value of the text box. When user assigns a string to it, the string is written to the text box.

Now look on the creating instances of GUIAT components in the Form class. We have simply added dictionary with known component types (RecognizableComponents). When we go through all components on the form in the _AnalyzeStructure method, we first check whether we know the component type or not. If so, we create instance of the appropriate component type class (e.g. Button or TextBox). If not, we create instance of the BaseComponent class.

Let's try a small example:

>>> import GUIAT
>>> g = GUIAT.GUIAT()
>>> g.Run()
Starting GUIAT...
"frmGUIAT" (frmGUIAT)
  "btnAddItem" (Button)
  "lblNewItem" (Label)
  "txtNewItem" (TextBox)
  "lbxItems" (ListBox) - UNKNOWN
  "btnQuit" (Button)
Starting GUIAT done.
>>> txt = g.mainForm._GetComponentByName('txtNewItem')
>>> btn = g.mainForm._GetComponentByName('btnAddItem')
>>> txt.value = 'Hello world!'
>>> btn.Click()

The result of the above small script is a new line in the list box. You see that the button, label, and text box are know component types. The label is treated as BaseComponent (see RecognizableComponents in Form.py). The list box is unknown component.

Today, we have shown how to create button and text box GUIAT classes that control respective .NET components. We can already script our small tested application!

Next time, we add list box class and logging.

Building the framework (1)

noreply@blogger.com (Lukáš Čenovský) — Tue, 29 Jul 2008 21:28:00 +0000

Today we start building the automated test framework. We have prepared tools in previous parts. See them to learn how to run tested application in a separate thread or how to simulate user's input.

The foundation of our test framework are IronPython classes that control Windows components. Simply said, everything in Windows is a component. So we create IronPython layer that allows controlling each Windows component easily. Then we create classes that will control forms with many components. And finally, we build test scripts and test suites for the whole application.

We start with just three simple classes. The first class, GUIAT, is the core class taking care of running, activating (focusing), and inspecting the tested application. The second class, BaseComponent, is ancestor of all component classes. The last one, Form, descendant of BaseComponent, is ancestor of all forms.

The BaseComponent class provides access to common properties of all components. There is just one in the first version:

location - size and position of the component on the screen (not within the parent form)

The BaseComponent class has also several private variables and methods:

_name - name of the component, not all components have this field filled
_control - reference to the .NET component instance
_guiat - reference to the main GUIAT object (see below)
_guiatComponents - dictionary with all child GUIAT components
_GetComponentByName - method that searches for component with given name

Note: The .NET component means instance of the Windows component in the tested application. The GUIAT component means instance of the IronPython class that controls the .NET component.

The Form class extends the BaseComponent class. Its purpose is to find and store all components on itself. It has one private method that does it:

_AnalyzeStructure - method that searches and stores components on a form. It is recursively called for each member of the Controls collection. It prints out the name, type, and depth level of the found components. Then it creates BaseComponent instance and stores it into the _guiatComponents dictionary of the parent component.

The GUIAT class as the core class of the framework contains in the first version only one property and two methods:

mainForm - GUIAT representation of the main form of the tested application (Form instance)
Run - method that runs the tested application in separate thread and creates the Form instance of the main form
Activate - method that activates (focuses) the tested application

Let's try a small example:

>>> import GUIAT
>>> g = GUIAT.GUIAT()
>>> g.Run()
Starting GUIAT...
"frmGUIAT" (frmGUIAT)
  "btnAddItem" (Button)
  "lblNewItem" (Label)
  "txtNewItem" (TextBox)
  "lbxItems" (ListBox)
  "btnQuit" (Button)
Starting GUIAT done.
>>> print g.mainForm._GetComponentByName('txtNewItem').location
(212, 177, 96, 16)
>>> g.Activate()
True

The above code runs the tested application, lists name and type of all components, then prints location of the text box, and finally activates the tested application.

To sum up the process again:

Create instance of the GUIAT class - let's call it g
g.Run() runs the tested application
g.Run() creates instance of IronPython Form class for the main form of the tested application
The Form instance searches during its initialization all components on the main form, prints information about them, and creates BaseComponent instance for each of them
To find location of "txtNewItem" component, execute:
g.mainForm._GetComponentByName("txtNewItem").location
To activate tested application, execute:
g.Activate()

Today I presented the core of my GUI automated testing framework (source). I'm going to extend it and make it more user friendly in next parts.

Simulate user's input

noreply@blogger.com (Lukáš Čenovský) — Tue, 08 Jul 2008 19:12:00 +0000

We have learned in the previous part how to explore the tested application and read its values. To test it, we also need set values to its fields.

The first idea may be to utilize the .NET objects. When we can read values from the tested application fields, we can also set them. Use the code snippet from the previous part to run the tested application and follow it by command

App.Controls[2].Text = 'GUIATtext'

The tested application is started and GUIATtext appears in the New listbox item text box.

Well, it does not appear as a proper testing to me. I'd rather simulate user's interaction with mouse and keyboard. Unfortunately, sending key strokes and mouse events is not job for .NET framework. We have to dive deep into Windows and use Win32 API to perform these tasks.

I have created a simple DLL to make Win32 API functions available for IronPython [1]. Moreover, I have added functions to simulate user's input. Here is the list of functions in the Win32API namespace with a short description:

GetForegroundWindow()

return handle of the active window

SetForegroundWindow(handle)

set the active window to the window with handle
return boolean

ShowWindow(handle, state)

set window with handle to state (minimalized, maximalized, ...)
return boolean

GetWindowText(handle, title, capacity)

return title of window with handle, title variable type is String of capacity size

MouseClick(X, Y)

simulate click with left mouse button on position X, Y

SendKey(virtual_key)

simulate pressing virtual_key

SendString(string)

simulate typing a string

I will not go into details here. Anyone interested can explore the sources of the Win32API.dll.

Using these functions is straightforward in IronPython. Just download the Win32API.dll and look at the example below how to perform a mouse click on the specific position:

import clr
clr.AddReference('Win32API')
from Win32API import Win32API
Win32API.MouseClick(10, 10)

The code snippet moves mouse to the coordinates 10, 10 (that is to the top-left corner of the screen) and perform a click with the left mouse button. More useful examples will follow in next parts.

Finally, couple of warnings. The simulated mouse click is caught by the top most window on the position. And the simulated key strokes are sent to the active window. That means we have to pay attention to position and active state of the tested application.

In the next part, we start building the GUIAT testing framework.

[1] I do not know how to access Win32 API directly from IronPython.

Exploring test application: IronPython (2)

noreply@blogger.com (Lukáš Čenovský) — Tue, 24 Jun 2008 19:12:00 +0000

We demonstrated how to run a .NET application from IronPython in the previous part. However, we couldn't control it while it was running. The solution for this problem is to run the tested application in a separate thread. Thus we will be able to enter commands in IronPython console while the tested application is running. Here is a snippet from the source:

import clr
clr.AddReference('System')
clr.AddReference("System.Windows.Forms")
from System import *
from System.Reflection import *
from System.Threading import *
from System.Windows.Forms import Application
from time import sleep

def RunMeCallBack(var):
    global App
    asm = Assembly.LoadFrom('GUIAT_PoC.exe')
    asm_type = asm.GetType('GUIAT_PoC.frmGUIAT')
    App = Activator.CreateInstance(asm_type)
    Application.Run(App)

App = None
ThreadPool.QueueUserWorkItem(WaitCallback(RunMeCallBack))
while not App:
    sleep(0.2)

The RunMeCallBack function starts the tested application the same way as we showed in the previous part. We create an independent thread and run this function in it so it does not block the console. The thread finishes its work when the tested application terminates (the main form of the application is closed) or when the console is closed.

The important part is line

App = Activator.CreateInstance(asm_type)

Here we remember the instance of the main form in the variable App. We have access to the whole application thanks to the App variable! The while cycle at the end ensures waiting until the App variable is not None. Which only happens when our tested application is up and running.

The App variable is our Holy Grail. Let's explore what is inside:

>>> App
<GUIAT_PoC.frmGUIAT object at 0x000000000000002B
[GUIAT_PoC.frmGUIAT, Text: GUIAT - Proof of Concept]>
>>> App.Text
'GUIAT - Proof of Concept'

Basically, we have access to all public properties and methods. Try dir(App) and you'll see. With a trick, we can even access private properties and methods (using the power of reflection).

To find what components are on the main form, iterate through the Controls collection:

>>> for c in App.Controls:
...   print c.Name, c.GetType()
...
btnAddItem System.Windows.Forms.Button
lblNewItem System.Windows.Forms.Label
txtNewItem System.Windows.Forms.TextBox
lbxItems System.Windows.Forms.ListBox
btnQuit System.Windows.Forms.Button

To find out what text is in the text box, try the following:

>>> App.Controls[2].Text
''

Now, write something directly into the text box in the tested application and call the statement again:

>>> App.Controls[2].Text
'something'

Cool, isn't it? ;-)

Next time, I show you how to simulate user interaction programatically - how to send a text or click to the tested application.

Exploring test application: IronPython (1)

noreply@blogger.com (Lukáš Čenovský) — Sun, 22 Jun 2008 11:33:00 +0000

How can IronPython help with exploring the test application? The answer is .NET Reflection:

Reflection is the mechanism of discovering information solely at run time.

We can discover both class and instance information. That's important. Knowing class information, we know what the object is capable of. We know whether it is a button having click method or a text edit having value property. And knowing instance information, we can find out what text the text edit displays.

That's nice but to be able to utilize the reflection, we need access to the tested application's objects from our testing framework. That's easy. The tested application is a .NET application so we can run it directly from IronPython.

When you look to Program.cs (source), you see how the tested application is started. If you don't have the sources of tested application available, you can use Lutz Roeder's .NET Reflector to find the information.

static void Main()
{
   Application.EnableVisualStyles();
   Application.SetCompatibleTextRenderingDefault(false);
   Application.Run(new frmGUIAT());
}

Let's mimic the start in IronPython. We are not interested in the first two rows. The third row is the core. The Application class is part of System.Windows.Forms namespace. So we just need to create an instance of frmGUIAT class. To create it, we use method CreateInstance of Activator class from System namespace. The CreateInstance method needs to know the type of the future instance. We find the type utilizing the reflection.

The whole code looks like this:

import clr
clr.AddReference('System')
clr.AddReference("System.Windows.Forms")
from System import *
from System.Reflection import *
from System.Windows.Forms import Application

asm = Assembly.LoadFrom('GUIAT_PoC.exe')
asm_type = asm.GetType('GUIAT_PoC.frmGUIAT')
App = Activator.CreateInstance(asm_type)
Application.Run(App)

When you run the above code from IronPython console, the tested application is displayed. Note the IronPython must be started form the directiory where GUIAT_PoC.exe is located. The disadvantage is we cannot do anything in IronPython console while the tested application is running. The solution is to run it in separate thread. I will show it in the next article.

Exploring test application: Accessibility

noreply@blogger.com (Lukáš Čenovský) — Thu, 12 Jun 2008 21:50:00 +0000

Accessibility looks more promising:

Active Accessibility (AA) is technology developed by Microsoft to make the Windows platform more accessible to people with disabilities. It consists of a set of COM interfaces and methods which allow an application to expose information to third parties, making it easier to implement tools such as magnifiers and screen readers. It proves very useful also for testing utilities. Microsoft has implemented the accessibility interfaces for the standard Windows controls.

AccExplorer (download) shows you the test application structure:

You see nice hierarchical window structure. Every object has several properties:

Name - e.g. caption of button
Value - e.g. value of list box
Role Text - type of the component
Location - position on the screen
State - e.g. disabled, normal, etc.
Def Action - default action for the component

The properties provides more information than Win32 API. But when you look closer, you see it is not enough:

We still cannot distinguish the similar controls. Both buttons have role push buttons and can be distinguished only by displayed name. Which is problematic when names change or you need to test localized applications.
Some components do not have name at all. In our example it is the text box - its name in AccExplorer is NAMELESS.
The third party components usually lack support for the accessibility in the same quality as Microsoft. For example, DevExpress grid only tells you "I am a grid". It does not tell you how many rows it contains, what cells are there, etc...

The good news is developers can change this behavior. They can define AccessibleName property for a component so we can later easily find exactly what we need. If I have set AccessibleName for the text box to myTextBox, we would have seen myTextBox instead of NAMELESS in AccExplorer now. For grid, they can define the whole structure of accessible objects to represent the grid (see Exposing Data Tables through Microsoft Active Accessibility). Unfortunately, when your application does not have sufficient support for accessibility, it is quite lot of work...

Microsoft tries to address testing issues with his new framework in .NET 3.5: Microsoft UI Automation. I have not tried it as our applications do not run on .NET 3.5.

Also DevExpress promised to enhance their components with better support for UI testing tools - see the 2008 roadmap.

But we don't want to wait, right? We want to test our .NET application right now! I will show how in the next article.

Exploring test application: Win32 API

noreply@blogger.com (Lukáš Čenovský) — Tue, 03 Jun 2008 22:30:00 +0000

Let's look on what we can discover about our simple test application when it is running. Before IronPython, we had two options:

Win32 API
Accessibility

Win32 API provides several functions that return information about an application. First, we have to find the appliction handle. We utilize functions EnumWindows and GetWindowText from Python for Windows extensions project:

def enum(hwnd, extra):
   global app_handle
   if winxpgui.GetWindowText(hwnd).find('GUIAT - Proof') >= 0:
       app_handle = hwnd

app_handle = None
winxpgui.EnumWindows(enum, '')
print 'Test application handle:', app_handle

When we know the application handle, we can enumerate its children:

def enum_child(hwnd, extra):
   print '%s %s:    %s' % (hwnd, winxpgui.GetClassName(hwnd),
           winxpgui.GetWindowText(hwnd))

winxpgui.EnumChildWindows(app_handle, enum_child, '')

When you run the whole script (source), you get:

Test application handle: 2949744
2621886 WindowsForms10.BUTTON.app.0.378734a:    Add Item
7668314 WindowsForms10.STATIC.app.0.378734a:    New listbox item
4194952 WindowsForms10.EDIT.app.0.378734a:
2753142 WindowsForms10.LISTBOX.app.0.378734a:
3342924 WindowsForms10.BUTTON.app.0.378734a:    Quit

Here is the catch. How shall we identify Add Item button from Quit button? Both have the same class name WindowsForms10.BUTTON.app.0.378734a. Sure, the text differs, but this isn't always true. Imagine two text boxes - both having the same class name WindowsForms10.EDIT.app.0.378734a. What then? Of course, you can check positions. But this is leads you back to record/play tools and that is exactly what I want to avoid.

So this is the terminus for Win32 API. Next time - Accessibility.

GUI Application to Test

noreply@blogger.com (Lukáš Čenovský) — Wed, 28 May 2008 18:40:00 +0000

Let's start with something easy. We have a Windows GUI application that looks like this:

The application contains a text box, two buttons, and a list box. It simply adds a text from text box as the last item to the list box (exe, source).

The testing steps might be as follows:

Run the application.
Check the 'New listbox item' fields is empty.
Check the list box contains three rows with texts: 'This is', 'GUIAT', 'demo.'.
Enter a text (e.g. 'new line') into text box, press 'Add Item' button. Verify your text ('new line') appears as the fourth line/item in the list box.
Press 'Quit' button. Verify the application terminates.

Any record/play testing tool can do the test up to point 4. There it cannot verify whether the text was added to the correct place in the list box. These tools also cannot handle moving 'Add Item' button closer to the text box. They would then click to empty place. Sure there are more sophisticated tools that can handle the above problems. But I do not know about any for free.

Next time we try some dead ends from my beginnings :-)

The beginning

noreply@blogger.com (Lukáš Čenovský) — Thu, 22 May 2008 15:33:00 +0000

Testing GUI application can be tedious work. Usually, you have to manually click through the interfaces. The first round is fun, the second is OK, and from the third round on it is boring.

When I joined Radiant, I started to think about automation. I searched on Internet but I did not find anything satisfying. All tools were basically record/play applications. That is not sufficient for testing. There are many concerns, for example:

How do you verify the test result?
How easy can you change the script (add testing of a new field, new form, etc.)
Scalability - how easy can you join basic scripts and create bigger (regression) script?
How the test application works with your amended components?
etc.

I know the testing tool cannot do everything. So I set up several goals I'd like to achieve:

Allow to control GUI application.
That means to start it, navigates through it, close it. To know its status.
Allow to read values from the screen.
You have to know what is on the screen to be able to test it.
Allow to set values.
Also, you have to be able to set/write values into the application to be able to test it.
Easy to create.
Testers are not experienced programmers. The tool must be easy to use. Anybody, who knows what a program is, should be able to write a test script.
Easy to maintain.
When the tested application changes, amending a test script should not be necessary or be a piece of cake.
Scalability.
Joining test scripts together, creating regression scripts.

In next few weeks, you'll find out on this blog how I succeed in this task.