Automated testing Windows applications using Katalon and WinAppDriver

  

[WayBack] Top 45 Best Automation Testing Tools Ultimate List • Test Automation Made Easy: Tools, Tips & Training Awesomeness A few notes, partially related to Enable your device for development – UWP app developer | Microsoft Docs.
One research project I had a while ago, was to see how it was possible to use a generic testing tool like Katalon Studio (that can test many platforms), for testing Windows applications.
Katalon uses a Selenium interface that normally is used for web applications through the [WayBack] WebDriver protocol (formerly [WayBack] JsonWireProtocol) that continues to be updated: [WayBack] draft upcoming WebDriver protocol, which is more generic than just web applications:

To test mobile apps, you can use [WayBack] Appium: Mobile App Automation Made Awesome.
To test Windows applications, you can use the – also open source – WinAppDriver, which is installed as part of Appium.

Adding the WebDriver protocol was basically the biggest change between Selenium 1.0 and Selenium 2.0: [WayBack] Selenium WebDriver
WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of web browsers.
Mor information on WebDriver at [WayBack] GitHub – w3c/webdriver: Remote control interface that enables introspection and control of user agents.
WinAppDriver is at [WayBack] GitHub – Microsoft/WinAppDriver: Windows Application Driver
Finding your way is not easy, as for instance “SetCapabilities” “WinAppDriver” – Google Search returns hardly anything, it af all useful.
By now I know that should have been “SetCapability” “WinAppDriver” – Google Search, which for a word-blind person like me, looks identical, but is not.
A few notes if you want to go the same way:

DesiredCapabilities and SetCapability:

Not sure yet, but on some levels, DesiredCapabilities seems to be deprecated (more on that below the fold)
[WayBack] WinAppDriver/Session.cs at master · Microsoft/WinAppDriver · GitHub and[WayBack] WinAppDriver/TestBase.cs at master · Microsoft/WinAppDriver · GitHub have a great set of test cases showing all the Windows supported capabilities

Many more tests at [WayBack] WinAppDriver/Tests/WebDriverAPI at master · Microsoft/WinAppDriver · GitHub.

[WayBack] Windows – Appium has more tips on capabilities
[WayBack] WinAppDriver/README.md at master · Microsoft/WinAppDriver · GitHub has the WinAppDriver Capabilities:

Capabilities
Descriptions
Example

app
Application identifier or executable full path
Microsoft.MicrosoftEdge_8wekyb3d8bbwe!MicrosoftEdge

appArguments
Application launch arguments
https://github.com/Microsoft/WinAppDriver

appTopLevelWindow
Existing application top level window to attach to
0xB822E2

appWorkingDir
Application working directory (Classic apps only)
C:\Temp

platformName
Target platform name
Windows

platformVersion
Target platform version
1.0

[WayBack] Desired Capabilities – Appium has a long list of capability names, of which these are the most important:

Capability
Description
Values

platformName
Which mobile OS platform to use
iOS, Android, or FirefoxOS

platformVersion
Mobile OS version
e.g., 7.1, 4.4

deviceName
The kind of mobile device or emulator to use
iPhone Simulator, iPad Simulator, iPhone Retina 4-inch, Android Emulator, Galaxy S4, etc…. On iOS, this should be one of the valid devices returned by instruments with instruments -s devices. On Android this capability is currently ignored, though it remains required.

app
The absolute local path or remote http URL to a .ipa file (IOS), .app folder (IOS Simulator), .apk file (Android) or .apks file (Android App Bundle), or a .zip file containing one of these (for .app, the .app folder must be the root of the zip file). Appium will attempt to install this app binary on the appropriate device first. Note that this capability is not required for Android if you specify appPackage and appActivity capabilities (see below). Incompatible with browserName. See here about .apks file.
/abs/path/to/my.apk or http://myapp.com/app.ipa

newCommandTimeout
How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session
e.g. 60

.NET WinForms is supported by WinAppDriver, as this is solved: [WayBack] WinAppDriver does not work with WinForms applications · Issue #108 · Microsoft/WinAppDriver · GitHub: WPF and WinForms applications are supported by Windows Application Driver. Finding the element in those apps may differ but is consistently based on the following attributes:

Locator Strategy
Client API
Matched Attribute

accessibility id
FindElementByAccessibilityId
AutomationId

class name
FindElementByClassName
ClassName

id
FindElementById
RuntimeId (decimal)

name
FindElementByName
Name

tag name
FindElementByTagName
LocalizedControlType (upper camel case)

Using the inspect tool or page source, you can identify and use any suitable attribute to locate your element based on the rules above.
.NET too has a reference implementation in [Archive.is] Accessible.cs and this demo:

[WayBack] Making Custom Controls Accessible: Getting Started
[WayBack] Implementing MSAA for a Simple Control

Delphi is a totally different story, as it does not expose the automation interface that WebAppDriver expects: it needs IAccessible implementations.
[WayBack] Poll: are you using WinAppDriver to automate Win32, WinForms, or WPF applications? · Issue #369 · Microsoft/WinAppDriver · GitHub
[WayBack] WinAppDriver:Windows Application Driver
IAccessible:

is all because of [WayBack] Microsoft accessibility : Microsoft believes accessibility and inclusion are essential to delivering on our mission to empower everyone, everywhere.
Documentation:

[WayBack] IAccessible | Microsoft Docs
[WayBack] IAccessible Interface (Accessibility) | Microsoft Docs
[WayBack] Using the IAccessible Interface – Windows applications | Microsoft Docs
… (way more below the fold)

From Delphi:

[WayBack] Using the MSAA IAccessible interface within a Delphi Application | Steve Trefethen

WayBack: Debugger Tip: Manual stack crawling during native code debugging
WayBack: Chris’ CodeGear Debugger Blog » Debugger Tip: Manual stack crawling during native code debugging

[WayBack] accessibility – Creating Accessible UI components in Delphi – Stack Overflow
The VCL itself does not natively implement any support for MSAA. Windows provides default implementations for standard UI controls, which many standard VCL components wrap. If you need more MSAA support than Windows provides, you will have to implement the IAccessible interface yourself, and then have your control respond to the WM_GETOBJECT message so it can return a pointer to an instance of your implementation.
Update: For example, one way to add MSAA to an existing TEdit (if you do not want to derive your own component) might look something like this:
First answer uses
ObjectFromLresult
Second answer uses
AccessibleObjectFromWindow

via [WayBack] Delphi Components and Screen Readers – Stack Overflow
Be aware:
[WayBack] Memory Leak Using Windows Accessibility Tools (Microsoft Narrator, Windows Speech Recognition) With .NET Applications

[WayBack] Delphi IAccessible Get_accState influences Get_accName? – Stack Overflow
[WayBack] winforms – IAccessible Implementation – Accessible Name only in Control Window? – Stack Overflow

uses CreateStdAccessibleObject

[WayBack] delphi – IAccessible: Get Active url with Access violation – Stack Overflow
[WayBack] How to use AccessibleObjectFromWindow WinAPI function in Delphi? – Stack Overflow

[WayBack] AccessibleObjectFromEvent function | Microsoft Docs: Retrieves the address of the IAccessible interface for the object that generated the event that is currently being processed by the client’s event hook function.

[WayBack] CreateStdAccessibleObject function | Microsoft Docs: Creates an accessible object with the methods and properties of the specified type of system-provided user interface element.
[WayBack] CreateStdAccessibleProxyA function | Microsoft Docs: Creates an accessible object that has the properties and methods of the specified class of system-provided user interface element.
[WayBack] CreateStdAccessibleProxyW function | Microsoft Docs: Creates an accessible object that has the properties and methods of the specified class of system-provided user interface element.
[WayBack] How WM_GETOBJECT Works – Windows applications | Microsoft Docs: Microsoft Active Accessibility sends the WM_GETOBJECT message to the appropriate server application when a client calls one of the AccessibleObjectFromX functions.
[WayBack] Retrieving an IAccessible Object – Windows applications | Microsoft Docs:Microsoft Active Accessibility provides functions such as AccessibleObjectFromWindow and AccessibleObjectFromPoint that allow clients to retrieve accessible objects.
When a client calls AccessibleObjectFromWindow or any of the other AccessibleObjectFromfunctions that retrieve an interface to an object, Microsoft Active Accessibility sends the WM_GETOBJECT window message to the applicable window procedure within the appropriate application. To provide information to clients, servers must respond to the WM_GETOBJECT message.

[WayBack] AccessibleObjectFromEvent function | Microsoft Docs:Retrieves the address of the IAccessible interface for the object that generated the event that is currently being processed by the client’s event hook function.
[WayBack] AccessibleObjectFromWindow function | Microsoft Docs:Retrieves the address of the specified interface for the object associated with the specified window.
[WayBack] AccessibleObjectFromPoint function | Microsoft Docs:Retrieves the address of the IAccessible interface pointer for the object displayed at a specified point on the screen.

[WayBack] LresultFromObject function | Microsoft Docs: Returns a reference, similar to a handle, to the specified object. Servers return this reference when handling WM_GETOBJECT.

Return code
Description

E_INVALIDARG
One or more arguments are not valid.

E_NOINTERFACE
The object specified in the pAcc parameter does not support the interface specified in the riid parameter.

E_OUTOFMEMORY
Insufficient memory to store the object reference.

E_UNEXPECTED
An unexpected error occurred.

 [WayBack] ObjectFromLresult function | Microsoft Docs: Retrieves a requested interface pointer for an accessible object based on a previously generated object reference.

From .NET WinForms:

Control properties

[WayBack] Control.AccessibilityObject Property (System.Windows.Forms) | Microsoft Docs
[WayBack] Control.AccessibleDefaultActionDescription Property (System.Windows.Forms) | Microsoft Docs
[WayBack] Control.AccessibleDescription Property (System.Windows.Forms) | Microsoft Docs
[WayBack] Control.AccessibleName Property (System.Windows.Forms) | Microsoft Docs
[WayBack] Control.AccessibleRole Property (System.Windows.Forms) | Microsoft Docs
[WayBack] Control.Name Property (System.Windows.Forms) | Microsoft Docs

MSAA counterparts are below (see Content of Descriptive Properties)
[WayBack] Common approaches for enhancing the programmatic accessibility of your Win32, WinForms and WPF apps: Part 3 – WinForms – Microsoft Windows UI Automation Blog
[WayBack] Few tips on implementing a Coded UI Test Plugin Extension – Microsoft DevOps Blog

WinAppDriver

[WayBack] Best patter for waiting for a condition. · Issue #187 · Microsoft/WinAppDriver · GitHub
[WayBack] WinAppDriver Timing Out after Delphi 11 application launched · Issue #131 · Microsoft/WinAppDriver · GitHub
[WayBack;Translated] Windows 10 : Microsoft publie en open source WinAppDriver UI Recorder, pour simplifier l’automatisation des tests d’interface utilisateur
[WayBack] T687329 – Support WinForms UI testing with Appium / WinAppDriver | DevExpress Support Center
[WayBack] Windows Application Driver for PC integrates with Appium – Windows Developer BlogWindows Developer Blog
[WayBack] WinAppDriver Archives – Windows Admins

Of course this is not the only way to test applications; for alternatives (including commercial ones):

Comparison of GUI testing tools – Wikipedia
[WayBack] Top 45 Best Automation Testing Tools Ultimate List • Test Automation Made Easy: Tools, Tips & Training Awesomeness

Especially read the entries on Appium, Selenium, WinAppDriver, Gauge, Katalon, WebDriver.IO, Test.AI, AppDiff, AutoIt,

[WayBack] Ranorex – SOFTWARE TESTING TOOLS commercial, but good; see for instance these links:

[WayBack] Roald van Doorn på Twitter: “Integrating Ranorex with TeamCity using PsExec”

[WayBack] Integrating Ranorex with TeamCity using PsExec – Roald’s blog

[WayBack] T406360 – Ranorex UI test of dxRibbon | DevExpress Support Center

Solution: create helper classes for the ribbon and all it’s child components that implements the IAccessible interface. You will need to specify everything needed for this interface, but it will work with Ranorex and it’s Spy tool.

[WayBack] Continuous delivery with Legacy VCL Applications – Entwickler KonferenzA case study of how we applied CD principles to an older VCL application. We will take a look at the challenges we faced and the solutions we chose, the frameworks we use, release procedures and feedback loops. We will demonstrate how we safely build and deploy the windows software for Albelli and Vistaprint, and the benefits this brings to our team and our organization.Outline:

Automating your builds using TeamCity
Unit testing using DUnitX
Automated UI tests using Ranorex and Specflow
Deploy to different environments with ProGet and Octopus Deploy
Increased speed of value to customer (reduced stock)
Increased feedback to developers

[WayBack] Ranorex Joins the Idera Family – unlike the Delphi teams, Idera has not yet reformed Ranorex, so for now that is a good sign
[WayBack] Bogdan Polak on Twitter: “Great session @danieleteti about Automated Tests run by pyTest at #ITDevCon #itdevcon9 testing #delphi apps. Small comment about Ranorex and VCL. It’s possible to implement this: https://t.co/d0zbYa2ePi”

[WayBack] We are searching for some Automation Testing Framework, different from Test Complate… Any ideas? We are trying to select a tool for automating an ap… – Avatarx – Google+
[WayBack] I’m using Tokyo here at work to develop our software. The rest of our team is still using XE5. Our build is still in XE5. A QA team member came to m… – Phillip Woon – Google+

–jeroen

[WayBack] Technical Overview – Windows applications | Microsoft Docs: Microsoft Active Accessibility improves the way accessibility aids (specialized programs that help people with disabilities use computers more effectively) work with applications running on Microsoft Windows.
[WayBack] WinEvents and Active Accessibility – Windows applications | Microsoft Docs

[WayBack] Bringing automated testing to Microsoft Edge through WebDriver – Microsoft Edge BlogMicrosoft Edge Blog

[WayBack] WinAppDriver – Test any app with Appium’s Selenium-like tests on Windows – Scott Hanselman
[WayBack] Unit Testing Silverlight with Selenium – Scott Hanselman
[WayBack] Selenium Downloads
[WayBack] Distributed Automated Browser Testing with Selenium and BrowserStack – Scott Hanselman

DesiredCapabilities levels of support, deprecation and AddAdditionalCapability

[WayBack] c# – With DesiredCapabilities deprecated, how will I use SetCapability in selenium webdriver C #? – Stack Overflow
[WayBack] What to use instead of Desired capabililities for a generic framework · Issue #6278 · SeleniumHQ/selenium · GitHub
[WayBack] Selenium W3C Capabilities Support – Beta – The Sauce Labs Cookbook – Sauce Labs Documentation Wiki
[WayBack] C# implementation of RemoteWebDriver fails to install Chrome extension · Issue #2874 · SeleniumHQ/selenium · GitHub answered at

[WayBack] selenium – C# implementation of RemoteWebDriver fails to install Chrome extension – Stack Overflow : use ChromeOptions instead of DesiredCapabilities.
Simiar to [WayBack] selenium – Pass driver ChromeOptions and DesiredCapabilities? – Stack Overflow
Related to [WayBack] C# implementation of RemoteWebDriver fails to install browser extensions · Issue #1578 · SeleniumHQ/selenium · GitHub

[WayBack] Selenium Click is not working with IE11 in Windows 10 · Issue #4292 · SeleniumHQ/selenium · GitHub where adding RequireWindowFocus does not always solve the problem.
[WayBack] Chrome DesiredCapabilities C# · Issue #4970 · SeleniumHQ/selenium · GitHub which explains to always perform options.AddAdditionalCapability(CapabilityType.Platform, “LINUX”, true); (with the correct platform name)
[Archive.is] WinAppDriver/StickyNotesSession.cs at master · Microsoft/WinAppDriver · GitHub: shows to use appCapabilities.SetCapability(“deviceName”, “WindowsPC”);
[WayBack] Frequently Asked Questions · Microsoft/WinAppDriver Wiki · GitHub
[WayBack] Windows – appium
[WayBack] Testing Windows Applications using Appium and JUnit in Java – Xray Latest Documentation – Xpand Add-ons Confluence
[WayBack] Windows app testing – appium
[Archive.is] Katalon testing for Windows apps – Katalon Studio / Feature / Improvement Suggestions – Katalon Community

A big tutorial:
[WayBack] Desired Capabilities in Selenium WebDriver:

Desired capability can also be used to configure the driver instance of Selenium WebDriver.
We can configure driver instance like FirefoxDriver, ChromeDriver, InternetExplorerDriver by using desired capabilities.

Desired Capabilities are more useful in cases like:

In mobile application automation, where the browser properties and the device properties can be set.
In Selenium grid when we want to run the test cases on a different browser with different operating systems and versions.

Generaly Microsoft Active Accessibility (MSAA, provided through implementing IAccessible) maps to Microsoft UI Automation (MS UIA or MSUIA), however:

[WayBack] Reason behind using MSAA and UIA
Q
What’s the reason behind using UIA and MSAA for the Coded UI Test framework? Especially why are WinForms controls handled by MSAA instead of UIA?
MSAA doesn’t have the SearchProperty AutomationID as UIA does, so finding controls is based on the less unique names (which are more likely to change), types and classes.
A
The Win32\Windows Forms control natively supported MSAA and not UIA whereas WPF supports UIA natively.  This is the reason for using MSAA for Win32\Windows Forms and UIA for WPF.  There is MSAA to UIA & vice-versa conversion available via built in proxy\bridge but that is an extra layer of dependency which could eat into performance etc.
When you use UIA for Windows Forms, it just maps ControlName property of the window as AutomationId.  The Coded UI Test Builder also uses this ControlName property if it is present.
Thanks.
blogs.msdn.com/gautamg – Gautam Goenka (MSFT)

[WayBack] c# – MS ui automation elements and web applications – Stack Overflow

 
The basic tools to check for MSAA or MSUIA are Inspect.exe and acccheckui.exe, both from the Windows SDK, but they are not the only ones:

[WayBack] Inspect – Windows applications | Microsoft Docs: Inspect (Inspect.exe) is a Windows-based tool that enables you select any UI element and view the element’s accessibility data.
[WayBack] wpf – Why does Inspect.exe hang frequently and inconsistently display AutomationId? – Stack Overflow (on mapping MS UIA AutomationId from AccessibleName or Name).
[WayBack] Visual UI Automation Verify – Windows applications | Microsoft Docs: Visual UI Automation Verify (Visual UIA Verify) is a Windows \ 32;GUI driver for the UIA Test Library that is designed for manual testing of UI automation.
[WayBack] UI Automation Verify (UIA Verify) Test Automation Framework – CodePlex Archive

[WayBack] UIAutomationVerify.zip

[WayBack] c# – AccessibleObjectFromPoint is returning client object instead of check box – Stack Overflow: UIA like MSAA operate out-of-process, so they’re not 100% in sync with the automated app. Sometimes, you’ll have to refresh the (elements) tree they have in cache.

 
IAccessible

Microsoft Active Accessibility – Wikipedia 
[WayBack] Microsoft Active Accessibility – Windows applications | Microsoft Docs

[WayBack] Legal Information – Windows applications | Microsoft Docs
[WayBack] Getting Started – Windows applications | Microsoft Docs
[WayBack] Technical Overview – Windows applications | Microsoft Docs:

Microsoft Active Accessibility consists of the following components:

The COM interface IAccessible, which exposes information about UI elements. IAccessible also has properties and methods for obtaining information about and manipulating that UI element.
WinEvents, an event system that allows servers to notify clients when an accessible object changes.
Oleacc.dll, a support or run-time DLL.

The Microsoft Active Accessibility DLL, Oleacc.dll, consists of the following components:

Functions that allow clients to request an IAccessible interface pointer (for example, AccessibleObjectFromWindow).
Functions that allow servers to return an IAccessible interface pointer to a client (for example, LresultFromObject).
Functions for getting localized text for the role and state codes (for example, GetRoleTextand GetStateText).
Some helper functions (AccessibleChildren).
Code that provides the default implementation of IAccessible for standard USER and COMCTL controls. Because these implement IAccessible on behalf of the system controls, they are known as proxies.

[WayBack] How Active Accessibility Works – Windows applications | Microsoft Docs
[WayBack] Active Accessibility Basics – Windows applications | Microsoft Docs
[WayBack] Server Guidelines – Windows applications | Microsoft Docs

To implement IAccessible, server developers must follow this basic process.

Create accessible objects by implementing the IAccessible properties and methods for your custom user interface elements and for your application’s client. Be sure to provide a dual interface that supports both IAccessible and IDispatch so that clients written in Microsoft Visual Basic and various scripting languages can get information about the objects.
Call NotifyWinEvent to notify clients of changes to your custom user interface elements.
Handle WM_GETOBJECT to provide access to your accessible objects.

[WayBack] How Servers Implement Child IDs – Windows applications | Microsoft Docs:If you implement IEnumVARIANT, you must:

Enumerate all children, both simple elements and accessible objects. Provide child IDs for all simple elements and provide the IDispatch to each accessible object.
For accessible objects, set the vt member of the VARIANT to VT_DISPATCH. The pdispVal member must contain a pointer to the IDispatch interface. Note that the VARIANT is allocated and freed by the client.
For simple elements, the child ID is any 32-bit positive integer. Note that zero and negative integers are reserved by Microsoft Active Accessibility. Set the VARIANTstructure vt member to VT_I4 and the lVal member to the child ID.

If you do not support IEnumVARIANT, you must assign child IDs and number the children in each object sequentially starting with one.
It is recommended that clients use the Microsoft Active Accessibility function AccessibleChildren rather than call the server IEnumVARIANT interface directly.

[WayBack] Client Guidelines – Windows applications | Microsoft Docs
[WayBack] COM and Unicode Guidelines – Windows applications | Microsoft Docs: Because Microsoft Active Accessibility is based on Component Object Model (COM), developers need a moderate level of understanding about COM objects and interfaces and must know how to perform basic tasks (for example, how to initialize the COM library).

[WayBack] C/C++ Developer’s Guide – Windows applications | Microsoft Docs

[WayBack] Active Accessibility User Interface Services – Windows applications | Microsoft Docs

[WayBack] User Interface APIs – Windows applications | Microsoft Docs

[WayBack] Using the IAccessible Interface – Windows applications | Microsoft Docs: The IAccessible interface is a collection of methods that expose the most common attributes and behaviors of a wide range of UI elements. A UI element is a control, such as a menu or push button, that is part of the user interface. An accessible object is a UI element that has a meaningful IAccessible interface.Some of the IAccessible properties are not applicable to every kind of user interface element. Also, the implementation of IAccessible varies from application to application.

[WayBack] Content of Descriptive Properties – Windows applications | Microsoft Docs:

Name Property
Role Property
State Property
Value Property
Description Property
Help Property
HelpTopic Property
KeyboardShortcut Property
DefaultAction Property

When designing accessible objects, server developers should also refer to the following topics:

Choosing Which Properties to Support
Choosing the Content for Descriptive Properties

[WayBack] Selection and Focus Properties and Methods – Windows applications | Microsoft Docs
[WayBack] Object Navigation Properties and Methods – Windows applications | Microsoft Docs

[WayBack] Dynamic Annotation API – Windows applications | Microsoft Docs: The Dynamic Annotation API is an extension to Microsoft Active Accessibility that allows developers to customize existing IAccessible support without having to use error-prone subclassing or wrapping techniques.

[WayBack] Calling Active Accessibility APIs – Windows applications | Microsoft Docs
[WayBack] Developer’s Guide for Active Accessibility Clients – Windows applications | Microsoft Docs

[WayBack] Getting an Accessible Object Interface Pointer – Windows applications | Microsoft Docs
[WayBack] Checking IAccessible Return Values – Windows applications | Microsoft Docs: (note to self: also relevant for accessibility servers)Client developers should not rely on the Component Object Model (COM) macros SUCCEEDEDand FAILED to test IAccessible return values, because values other than S_OK are considered a success. For example, a method can return S_FALSE, which is considered a success by the SUCCEEDED macro, but still receive a NULL pointer in an output parameter.Client developers must guard against the possibility that some servers return error codes other than the documented values. To be safe, you must ensure that all the output parameters contain valid information and meet the following criteria:

All pointers are non-NULL.
The vt member of any VARIANT structure is not equal to VT_EMPTY.

[WayBack] Receiving Errors for IAccessible Interface Pointers – Windows applications | Microsoft Docs
[WayBack] Active Accessibility and Windows Vista Screen Scaling – Windows applications | Microsoft Docs: Windows Vista enables users to change the dots-per-inch (dpi) setting so that most UI elements on the screen appear larger.

[WayBack] Developer’s Guide for Active Accessibility Servers – Windows applications | Microsoft Docs

[WayBack] Active Accessibility Text Services – Windows applications | Microsoft Docs

[WayBack] C/C++ Reference – Windows applications | Microsoft Docs:This section provides C/C++ language reference information about the Microsoft Active Accessibility \ 32;API.In this sectionActive Accessibility User Interface ServicesActive Accessibility Text Services.

[WayBack] Active Accessibility User Interface Services – Windows applications | Microsoft Docs

[WayBack] Interfaces – Windows applications | Microsoft Docs

[WayBack] IAccessible | Microsoft Docs

[WayBack] Client Functions – Windows applications | Microsoft Docs
[WayBack] Server Functions – Windows applications | Microsoft Docs
[WayBack] Structures and Data Types – Windows applications | Microsoft Docs
[WayBack] WM_GETOBJECT Window Message – Windows applications | Microsoft Docs: The system sends this message to Microsoft Active Accessibility servers to obtain information about an object.
[WayBack] Constants – Windows applications | Microsoft Docs
[WayBack] Return Values – Windows applications | Microsoft Docs

The IAccessible methods return one of the following values, defined in winerror.h, or another standard Component Object Model (COM) error code: …
The following are return values that IAccessible methods might return. These return values are not as common as the previous ones, but you should be aware of them: …

[WayBack] Active Accessibility Text Services – Windows applications | Microsoft Docs: Active Accessibility Text Services is deprecated. Please see Microsoft Windows Text Services Framework for more information on advanced text input and natural language technologies.

[WayBack] Samples – Windows applications | Microsoft Docs
[WayBack] Appendixes – Windows applications | Microsoft Docs

[WayBack] Appendix A Supported User Interface Elements Reference – Windows applications | Microsoft Docs: This appendix contains information about the system-provided UI elements exposed by Microsoft Active Accessibility in Windows 95, Windows 98, Microsoft Windows NT, Windows 2000, Windows XP, and Windows 2000 Server.
[WayBack] Appendix B Standard Dialog Manager Support – Windows applications | Microsoft Docs: SDM is an internal Microsoft code library that provides Microsoft applications with a degree of independence from the differences between the Macintosh and Microsoft Windows operating systems. SDM is primarily used for dialog boxes in Microsoft Excel and Microsoft Word.
[WayBack] Appendix C IAccessible DISPIDs – Windows applications | Microsoft Docs: A DISPID allows the implementation of the IDispatch to look up the various methods and properties of a dual interface.
[WayBack] Appendix D Notes for Visual Basic Developers – Windows applications | Microsoft Docs: This documentation uses the C/C++ names for the IAccessible properties; however, the Visual Basic names are similar. For example, the IAccessible::get_accName property would be called accName in a Visual Basic application.
[WayBack] Appendix E Text Attributes for Active Accessibility Text Services Dictionary – Windows applications | Microsoft Docs
[WayBack] Appendix F Object Identifier Values for OBJID_QUERYCLASSNAMEIDX – Windows applications | Microsoft Docs: When OLEACC sends a WM_GETOBJECT message with the lParam parameter set to OBJID_QUERYCLASSNAMEIDX, many standard USER or common controls (COMCTL) return one of the following values.
Only USER and Windows common controls (COMCTL) will return one of the values from the table. If a window returns 0 in response to this message, the window may be one of the following:

A custom control
A control other than one of the controls in the previous table
An old version of a system control that does not recognize the WM_GETOBJECT message

If a window returns 0, clients may need to use RealGetWindowClass or GetClassName. You can use these functions to determine the type of control based on class name.

USER or common control
Return value

Listbox
65536+0

Button
65536+2

Static
65536+3

Edit
65536+4

Combobox
65536+5

Scrollbar
65536+10

Status
65536+11

Toolbar
65536+12

Progress
65536+13

Animate
65536+14

Tab
65536+15

Hotkey
65536+16

Header
65536+17

Trackbar
65536+18

Listview
65536+19

Updown
65536+22

ToolTips
65536+24

Treeview
65536+25

RichEdit
65536+28

[WayBack] Appendix A Supported User Interface Elements Reference – Windows applications | Microsoft Docs

Windows class name
UI element type
Windows Vista updates

ListBox
List boxes
None

Button
Push buttons, radio buttons, check buttons, group boxes
Split buttons can have zero or more children.

Static
Labels
None

Edit
Text boxes
None

ComboBox
Combo boxes, drop-down lists
None

ScrollBar
Scroll bars
EVENT_OBJECT_CONTENTSCROLLED is a new event for control that have scrolling functionality but do not include a standard scroll bar as part of the control.

#32768
USER menus
None

#32770
USER dialog boxes
None

#32771
Alt-tab window
Available only in classic mode.

msctls_statusbar32
Status bars
None

msctls_progress32
Progress bars
New color options for progress bars are not exposed by Microsoft Active Accessibility or Microsoft UI Automation properties.

msctls_hotkey32
Hot key controls
None

msctls_trackbar32
Trackbars, sliders
None

msctls_updown32
Up-down or spin controls
None

SysAnimate32
Animation control
None

SysTabControl32
Tab control
None

SysHeader32
List view headers
None

SysListView32
List view controls
None

SysTreeView32
Tree view controls
None

SysDateTimePick32 (versions 5 and 6)
Date and/or time picker
Version 6 of this control in Windows Vista has a native IAccessible implementation.

SysIPAddress32
IP address controls
None

tooltips_class32
ToolTips
None

ToolbarWindow32
Toolbars
None

RICHEDIT, RichEdit20A, RichEdit20W
Text fields
None

SysMonthCal32 (versions 5 and 6)
Month calendar
Version 6 of this control in Windows Vista has a native IAccessible implementation.

[WayBack] Shortcuts for Exposing Custom User Interface Elements – Windows applications | Microsoft Docs

[WayBack] Labeling Owner-Drawn Controls – Windows applications | Microsoft Docs
[WayBack] Exposing Controls Based on System Controls – Windows applications | Microsoft Docs

[WayBack] Appendix G Active Accessibility Bridge to UI Automation – Windows applications | Microsoft Docs: Microsoft Active Accessibility-based clients, such as a screenreader on Windows XP, can programmatically interact with UI Automation-based providers of UI Automation, such as a Windows Presentation Foundation (WPF) application. It is part of the UI Automation Native Core API (UIAutomationCore.dll).
[WayBack] Object Identifiers – Windows applications | Microsoft Docs: This topic describes the Microsoft Active Accessibility object identifiers, 32-bit values that identify categories of accessible objects within a window.The following constants are defined in winuser.h:

Constant
Description

OBJID_ALERT
An alert that is associated with a window or an application. System provided message boxes are the only UI elements that send events with this object identifier. Server applications cannot use the AccessibleObjectFromX functions with this object identifier. This is a known issue with Microsoft Active Accessibility.

OBJID_CARET
The text insertion bar (caret) in the window.

OBJID_CLIENT
The window’s client area. In most cases, the operating system controls the frame elements and the client object contains all elements that are controlled by the application. Servers only process the WM_GETOBJECT messages in which the lParam is OBJID_CLIENT, OBJID_WINDOW, or a custom object identifier.

OBJID_CURSOR
The mouse pointer. There is only one mouse pointer in the system, and it is not a child of any window.

OBJID_HSCROLL
The window’s horizontal scroll bar.

OBJID_NATIVEOM
In response to this object identifier, third-party applications can expose their own object model. Third-party applications can return any COM interface in response to this object identifier.

OBJID_MENU
The window’s menu bar.

OBJID_QUERYCLASSNAMEIDX
An object identifier that Oleacc.dll uses internally. For more information, see Appendix F: Object Identifier Values for OBJID_QUERYCLASSNAMEIDX.

OBJID_SIZEGRIP
The window’s size grip: an optional frame component located at the lower-right corner of the window frame.

OBJID_SOUND
A sound object. Sound objects do not have screen locations or children, but they do have name and state attributes. They are children of the application that is playing the sound.

OBJID_SYSMENU
The window’s system menu.

OBJID_TITLEBAR
The window’s title bar.

OBJID_VSCROLL
The window’s vertical scroll bar.

OBJID_WINDOW
The window itself rather than a child object.

 

[WayBack] Common Infrastructure – Windows applications | Microsoft Docs: Windows offers two API specifications for user interface accessibility and software test automation Microsoft Active Accessibility, and Microsoft UI Automation.

[WayBack] WinEvents – Windows applications | Microsoft Docs

[WayBack] What Are WinEvents – Windows applications | Microsoft Docs: Server applications and the operating system use WinEvents to notify clients when a change occurs in the system or in the user interface.
[WayBack] Registering a Hook Function – Windows applications | Microsoft Docs: Client applications receive WinEvents in a WinEventProc callback function. The actions performed by the callback function are defined by the application, but the syntax must be as specified in the prototype.
[WayBack] System-Level and Object-Level Events – Windows applications | Microsoft Docs
[WayBack] In-Context and Out-of-Context Hook Functions – Windows applications | Microsoft Docs: When registering a hook function with SetWinEventHook, clients specify whether the hook function is in-context or out-of-context. These terms describe the memory location of the hook function relative to the server’s address space.
[WayBack] Guarding Against Reentrancy in Hook Functions – Windows applications | Microsoft Docs
[WayBack] Allocation of WinEvent IDs – Windows applications | Microsoft Docs

[WayBack] WM_GETOBJECT message – Windows applications | Microsoft Docs: Sent by both Microsoft Active Accessibility and Microsoft UI Automation to obtain information about an accessible object contained in a server application.
[WayBack] The WM_GETOBJECT Message – Windows applications | Microsoft Docs: This section describes the role of the WM_GETOBJECT message in Microsoft Active Accessibility and UI Automation, and explains how to process the message in a server or provider application.

[WayBack] Handling the WM_GETOBJECT Message – Windows applications | Microsoft Docs

If the receiving application is an Microsoft Active Accessibility server and the WM_GETOBJECT message includes an object identifier of OBJID_CLIENT, the server should return the value obtained by passing the IAccessible interface of the object to the LresultFromObject function.
If the receiving application is aUI Automation provider and the object identifier is UiaRootObjectId, the provider should return the IRawElementProviderSimple interface of the object. The provider obtains the interface by calling the UiaReturnRawElementProvider function.
If the receiving application implements neither Microsoft Active Accessibility nor UI Automation, it should pass the WM_GETOBJECT message to the DefWindowProcfunction. Passing the message enables the accessibility framework to determine if a proxy is available for the specified object.
If the object identifier is neither OBJID_CLIENT nor UiaRootObjectId, the receiving application should pass the WM_GETOBJECT message to the DefWindowProc function. Passing the message enables the accessibility framework to use the default providers for standard UI elements.

[WayBack] When to Respond to the WM_GETOBJECT Message – Windows applications | Microsoft Docs: If an application supports Microsoft Active Accessibility or UI Automation for a UI element, the application must not respond to the WM_GETOBJECT message before the object that represents the UI element is fully initialized, or after the application has begun to close.
[WayBack] Exposing a Native Object Model Interface – Windows applications | Microsoft Docs : If a UI element supports a native object model other than Microsoft Active Accessibility or UI Automation, it can expose the custom interface by responding to WM_GETOBJECT messages that include the OBJID_NATIVEOM object identifier.
[WayBack] The OBJID_QUERYCLASSNAMEIDX Object Identifier – Windows applications | Microsoft Docs: Microsoft Active Accessibility uses the OBJID_QUERYCLASSNAMEIDX object identifier with the WM_GETOBJECT message to determine the class to which a standard Windows control or common control belongs.

[WayBack] Appendix F Object Identifier Values for OBJID_QUERYCLASSNAMEIDX – Windows applications | Microsoft Docs

[WayBack] Dynamic Annotation – Windows applications | Microsoft Docs

[WayBack] Dynamic Annotation API – Windows applications | Microsoft Docs

[WayBack] The IAccessibleEx Interface – Windows applications | Microsoft Docs
[WayBack] Windowless ActiveX Control Accessibility – Windows applications | Microsoft Docs

 Tooling

[WayBack] Inspect – Windows applications | Microsoft Docs: is a Windows-based tool that enables you select any UI element and view the element’s accessibility data.
[WayBack] Accessible Event Watcher – Windows applications | Microsoft Docs: (AccEvent) allows developers and testers to validate that an application’s UI elements raise proper Microsoft UI Automation and Microsoft Active Accessibility events when UI changes occur.
[WayBack] AccScope – Windows applications | Microsoft Docs: enables developers and testers to evaluate the accessibility of their app during the app’s development and design, rather than in the late testing phases of an app’s development cycle.
[WayBack] UI Accessibility Checker – Windows applications | Microsoft Docs: a tool for testing an application’s UI Automation or Microsoft Active Accessibility (MSAA) implementation.
[WayBack] UI Automation Verify – Windows applications | Microsoft Docs: UI Automation Verify (or UIA Verify) is a testing framework for manual and automated testing of a control’s or application’s implementation of Microsoft UI Automation.
[WayBack] The AccChecker Graphical User Interface – Windows applications | Microsoft Docs: This topic describes the elements that make up the AccChecker GUI.

Failures:

[WayBack] WinAppDriver and WinForms MenuStrip control · Issue #562 · Microsoft/WinAppDriver · GitHub
[WayBack] WinForms MenuStrip’s ToolStripMenuItem children are not accessible · Issue #185 · Microsoft/WinAppDriver · GitHub
wm_getobject 0xffffffe7 – Google Search

[WayBack] Menu Accessibility: Focus events not triggering and names are not announced. · Issue #10507 · electron/electron · GitHub

Comments are closed.