[Windows] UI Automation Interface example

Share your advanced PureBasic knowledge/code with the community.
breeze4me
Enthusiast
Enthusiast
Posts: 511
Joined: Thu Mar 09, 2006 9:24 am
Location: S. Kor

[Windows] UI Automation Interface example

Post by breeze4me »

Related post:
MB/s scope grab MS Win

Another post about UI Automation by CELTIC88.
https://www.purebasic.fr/english/viewto ... 91#p544591

Code: Select all

;UIA_PropertyIds
#UIA_AcceleratorKeyPropertyId = 30006
#UIA_AccessKeyPropertyId = 30007
#UIA_AnnotationAnnotationTypeIdPropertyId = 30113
#UIA_AnnotationAnnotationTypeNamePropertyId = 30114
#UIA_AnnotationAuthorPropertyId = 30115
#UIA_AnnotationDateTimePropertyId = 30116
#UIA_AnnotationObjectsPropertyId = 30156
#UIA_AnnotationTargetPropertyId = 30117
#UIA_AnnotationTypesPropertyId = 30155
#UIA_AriaPropertiesPropertyId = 30102
#UIA_AriaRolePropertyId = 30101
#UIA_AutomationIdPropertyId = 30011
#UIA_BoundingRectanglePropertyId = 30001
#UIA_CenterPointPropertyId = 30165
#UIA_ClassNamePropertyId = 30012
#UIA_ClickablePointPropertyId = 30014
#UIA_ControllerForPropertyId = 30104
#UIA_ControlTypePropertyId = 30003
#UIA_CulturePropertyId = 30015
#UIA_DescribedByPropertyId = 30105
#UIA_DockDockPositionPropertyId = 30069
#UIA_DragDropEffectPropertyId = 30139
#UIA_DragDropEffectsPropertyId = 30140
#UIA_DragGrabbedItemsPropertyId = 30144
#UIA_DragIsGrabbedPropertyId = 30138
#UIA_DropTargetDropTargetEffectPropertyId = 30142
#UIA_DropTargetDropTargetEffectsPropertyId = 30143
#UIA_ExpandCollapseExpandCollapseStatePropertyId = 30070
#UIA_FillColorPropertyId = 30160
#UIA_FillTypePropertyId = 30162
#UIA_FlowsFromPropertyId = 30148
#UIA_FlowsToPropertyId = 30106
#UIA_FrameworkIdPropertyId = 30024
#UIA_FullDescriptionPropertyId = 30159
#UIA_GridColumnCountPropertyId = 30063
#UIA_GridItemColumnPropertyId = 30065
#UIA_GridItemColumnSpanPropertyId = 30067
#UIA_GridItemContainingGridPropertyId = 30068
#UIA_GridItemRowPropertyId = 30064
#UIA_GridItemRowSpanPropertyId = 30066
#UIA_GridRowCountPropertyId = 30062
#UIA_HasKeyboardFocusPropertyId = 30008
#UIA_HeadingLevelPropertyId = 30173
#UIA_HelpTextPropertyId = 30013
#UIA_IsAnnotationPatternAvailablePropertyId = 30118
#UIA_IsContentElementPropertyId = 30017
#UIA_IsControlElementPropertyId = 30016
#UIA_IsCustomNavigationPatternAvailablePropertyId = 30151
#UIA_IsDataValidForFormPropertyId = 30103
#UIA_IsDialogPropertyId = 30174
#UIA_IsDockPatternAvailablePropertyId = 30027
#UIA_IsDragPatternAvailablePropertyId = 30137
#UIA_IsDropTargetPatternAvailablePropertyId = 30141
#UIA_IsEnabledPropertyId = 30010
#UIA_IsExpandCollapsePatternAvailablePropertyId = 30028
#UIA_IsGridItemPatternAvailablePropertyId = 30029
#UIA_IsGridPatternAvailablePropertyId = 30030
#UIA_IsInvokePatternAvailablePropertyId = 30031
#UIA_IsItemContainerPatternAvailablePropertyId = 30108
#UIA_IsKeyboardFocusablePropertyId = 30009
#UIA_IsLegacyIAccessiblePatternAvailablePropertyId = 30090
#UIA_IsMultipleViewPatternAvailablePropertyId = 30032
#UIA_IsObjectModelPatternAvailablePropertyId = 30112
#UIA_IsOffscreenPropertyId = 30022
#UIA_IsPasswordPropertyId = 30019
#UIA_IsPeripheralPropertyId = 30150
#UIA_IsRangeValuePatternAvailablePropertyId = 30033
#UIA_IsRequiredForFormPropertyId = 30025
#UIA_IsScrollItemPatternAvailablePropertyId = 30035
#UIA_IsScrollPatternAvailablePropertyId = 30034
#UIA_IsSelectionItemPatternAvailablePropertyId = 30036
#UIA_IsSelectionPattern2AvailablePropertyId = 30168
#UIA_IsSelectionPatternAvailablePropertyId = 30037
#UIA_IsSpreadsheetItemPatternAvailablePropertyId = 30132
#UIA_IsSpreadsheetPatternAvailablePropertyId = 30128
#UIA_IsStylesPatternAvailablePropertyId = 30127
#UIA_IsSynchronizedInputPatternAvailablePropertyId = 30110
#UIA_IsTableItemPatternAvailablePropertyId = 30039
#UIA_IsTablePatternAvailablePropertyId = 30038
#UIA_IsTextChildPatternAvailablePropertyId = 30136
#UIA_IsTextEditPatternAvailablePropertyId = 30149
#UIA_IsTextPattern2AvailablePropertyId = 30119
#UIA_IsTextPatternAvailablePropertyId = 30040
#UIA_IsTogglePatternAvailablePropertyId = 30041
#UIA_IsTransformPattern2AvailablePropertyId = 30134
#UIA_IsTransformPatternAvailablePropertyId = 30042
#UIA_IsValuePatternAvailablePropertyId = 30043
#UIA_IsVirtualizedItemPatternAvailablePropertyId = 30109
#UIA_IsWindowPatternAvailablePropertyId = 30044
#UIA_ItemStatusPropertyId = 30026
#UIA_ItemTypePropertyId = 30021
#UIA_LabeledByPropertyId = 30018
#UIA_LandmarkTypePropertyId = 30157
#UIA_LegacyIAccessibleChildIdPropertyId = 30091
#UIA_LegacyIAccessibleDefaultActionPropertyId = 30100
#UIA_LegacyIAccessibleDescriptionPropertyId = 30094
#UIA_LegacyIAccessibleHelpPropertyId = 30097
#UIA_LegacyIAccessibleKeyboardShortcutPropertyId = 30098
#UIA_LegacyIAccessibleNamePropertyId = 30092
#UIA_LegacyIAccessibleRolePropertyId = 30095
#UIA_LegacyIAccessibleSelectionPropertyId = 30099
#UIA_LegacyIAccessibleStatePropertyId = 30096
#UIA_LegacyIAccessibleValuePropertyId = 30093
#UIA_LevelPropertyId = 30154
#UIA_LiveSettingPropertyId = 30135
#UIA_LocalizedControlTypePropertyId = 30004
#UIA_LocalizedLandmarkTypePropertyId = 30158
#UIA_MultipleViewCurrentViewPropertyId = 30071
#UIA_MultipleViewSupportedViewsPropertyId = 30072
#UIA_NamePropertyId = 30005
#UIA_NativeWindowHandlePropertyId = 30020
#UIA_OptimizeForVisualContentPropertyId = 30111
#UIA_OrientationPropertyId = 30023
#UIA_OutlineColorPropertyId = 30161
#UIA_OutlineThicknessPropertyId = 30164
#UIA_PositionInSetPropertyId = 30152
#UIA_ProcessIdPropertyId = 30002
#UIA_ProviderDescriptionPropertyId = 30107
#UIA_RangeValueIsReadOnlyPropertyId = 30048
#UIA_RangeValueLargeChangePropertyId = 30051
#UIA_RangeValueMaximumPropertyId = 30050
#UIA_RangeValueMinimumPropertyId = 30049
#UIA_RangeValueSmallChangePropertyId = 30052
#UIA_RangeValueValuePropertyId = 30047
#UIA_RotationPropertyId = 30166
#UIA_RuntimeIdPropertyId = 30000
#UIA_ScrollHorizontallyScrollablePropertyId = 30057
#UIA_ScrollHorizontalScrollPercentPropertyId = 30053
#UIA_ScrollHorizontalViewSizePropertyId = 30054
#UIA_ScrollVerticallyScrollablePropertyId = 30058
#UIA_ScrollVerticalScrollPercentPropertyId = 30055
#UIA_ScrollVerticalViewSizePropertyId = 30056
#UIA_Selection2CurrentSelectedItemPropertyId = 30171
#UIA_Selection2FirstSelectedItemPropertyId = 30169
#UIA_Selection2ItemCountPropertyId = 30172
#UIA_Selection2LastSelectedItemPropertyId = 30170
#UIA_SelectionCanSelectMultiplePropertyId = 30060
#UIA_SelectionIsSelectionRequiredPropertyId = 30061
#UIA_SelectionItemIsSelectedPropertyId = 30079
#UIA_SelectionItemSelectionContainerPropertyId = 30080
#UIA_SelectionSelectionPropertyId = 30059
#UIA_SizeOfSetPropertyId = 30153
#UIA_SizePropertyId = 30167
#UIA_SpreadsheetItemAnnotationObjectsPropertyId = 30130
#UIA_SpreadsheetItemAnnotationTypesPropertyId = 30131
#UIA_SpreadsheetItemFormulaPropertyId = 30129
#UIA_StylesExtendedPropertiesPropertyId = 30126
#UIA_StylesFillColorPropertyId = 30122
#UIA_StylesFillPatternColorPropertyId = 30125
#UIA_StylesFillPatternStylePropertyId = 30123
#UIA_StylesShapePropertyId = 30124
#UIA_StylesStyleIdPropertyId = 30120
#UIA_StylesStyleNamePropertyId = 30121
#UIA_TableColumnHeadersPropertyId = 30082
#UIA_TableItemColumnHeaderItemsPropertyId = 30085
#UIA_TableItemRowHeaderItemsPropertyId = 30084
#UIA_TableRowHeadersPropertyId = 30081
#UIA_TableRowOrColumnMajorPropertyId = 30083
#UIA_ToggleToggleStatePropertyId = 30086
#UIA_Transform2CanZoomPropertyId = 30133
#UIA_Transform2ZoomLevelPropertyId = 30145
#UIA_Transform2ZoomMaximumPropertyId = 30147
#UIA_Transform2ZoomMinimumPropertyId = 30146
#UIA_TransformCanMovePropertyId = 30087
#UIA_TransformCanResizePropertyId = 30088
#UIA_TransformCanRotatePropertyId = 30089
#UIA_ValueIsReadOnlyPropertyId = 30046
#UIA_ValueValuePropertyId = 30045
#UIA_VisualEffectsPropertyId = 30163
#UIA_WindowCanMaximizePropertyId = 30073
#UIA_WindowCanMinimizePropertyId = 30074
#UIA_WindowIsModalPropertyId = 30077
#UIA_WindowIsTopmostPropertyId = 30078
#UIA_WindowWindowInteractionStatePropertyId = 30076
#UIA_WindowWindowVisualStatePropertyId = 30075

;UIA_PatternIds
#UIA_InvokePatternId = 10000
#UIA_SelectionPatternId = 10001
#UIA_ValuePatternId = 10002
#UIA_RangeValuePatternId = 10003
#UIA_ScrollPatternId = 10004
#UIA_ExpandCollapsePatternId = 10005
#UIA_GridPatternId = 10006
#UIA_GridItemPatternId = 10007
#UIA_MultipleViewPatternId = 10008
#UIA_WindowPatternId = 10009
#UIA_SelectionItemPatternId = 10010
#UIA_DockPatternId = 10011
#UIA_TablePatternId = 10012
#UIA_TableItemPatternId = 10013
#UIA_TextPatternId = 10014
#UIA_TogglePatternId = 10015
#UIA_TransformPatternId = 10016
#UIA_ScrollItemPatternId = 10017
#UIA_LegacyIAccessiblePatternId = 10018
#UIA_ItemContainerPatternId = 10019
#UIA_VirtualizedItemPatternId = 10020
#UIA_SynchronizedInputPatternId = 10021
#UIA_ObjectModelPatternId = 10022
#UIA_AnnotationPatternId = 10023
#UIA_TextPattern2Id = 10024
#UIA_StylesPatternId = 10025
#UIA_SpreadsheetPatternId = 10026
#UIA_SpreadsheetItemPatternId = 10027
#UIA_TransformPattern2Id = 10028
#UIA_TextChildPatternId = 10029
#UIA_DragPatternId = 10030
#UIA_DropTargetPatternId = 10031
#UIA_TextEditPatternId = 10032
#UIA_CustomNavigationPatternId = 10033
#UIA_SelectionPattern2Id = 10034

;UIA_EventIds
#UIA_ActiveTextPositionChangedEventId = 20036
#UIA_AsyncContentLoadedEventId = 20006
#UIA_AutomationFocusChangedEventId = 20005
#UIA_AutomationPropertyChangedEventId = 20004
#UIA_ChangesEventId = 20034
#UIA_Drag_DragCancelEventId = 20027
#UIA_Drag_DragCompleteEventId = 20028
#UIA_Drag_DragStartEventId = 20026
#UIA_DropTarget_DragEnterEventId = 20029
#UIA_DropTarget_DragLeaveEventId = 20030
#UIA_DropTarget_DroppedEventId = 20031
#UIA_HostedFragmentRootsInvalidatedEventId = 20025
#UIA_InputDiscardedEventId = 20022
#UIA_InputReachedOtherElementEventId = 20021
#UIA_InputReachedTargetEventId = 20020
#UIA_Invoke_InvokedEventId = 20009
#UIA_LayoutInvalidatedEventId = 20008
#UIA_LiveRegionChangedEventId = 20024
#UIA_MenuClosedEventId = 20007
#UIA_MenuModeEndEventId = 20019
#UIA_MenuModeStartEventId = 20018
#UIA_MenuOpenedEventId = 20003
#UIA_NotificationEventId = 20035
#UIA_Selection_InvalidatedEventId = 20013
#UIA_SelectionItem_ElementAddedToSelectionEventId = 20010
#UIA_SelectionItem_ElementRemovedFromSelectionEventId = 20011
#UIA_SelectionItem_ElementSelectedEventId = 20012
#UIA_StructureChangedEventId = 20002
#UIA_SystemAlertEventId = 20023
#UIA_Text_TextChangedEventId = 20015
#UIA_Text_TextSelectionChangedEventId = 20014
#UIA_TextEdit_ConversionTargetChangedEventId = 20033
#UIA_TextEdit_TextChangedEventId = 20032
#UIA_ToolTipClosedEventId = 20001
#UIA_ToolTipOpenedEventId = 20000
#UIA_Window_WindowClosedEventId = 20017
#UIA_Window_WindowOpenedEventId = 20016

;TreeScope
#TreeScope_None = 0
#TreeScope_Element = 1
#TreeScope_Children = 2
#TreeScope_Descendants = 4
#TreeScope_Parent = 8
#TreeScope_Ancestors = $10
#TreeScope_Subtree = #TreeScope_Element | #TreeScope_Children | #TreeScope_Descendants

Interface IUIAutomationElementArray Extends IUnknown
  ;get_Length(/* [retval][out] */ __RPC__out int *length)
  ;GetElement(/* [in] */ int index, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  
  get_Length.l(*length)
  GetElement.l(index, *pElement)
EndInterface

Interface IUIAutomation Extends IUnknown
  ;CompareElements(/* [in] */ __RPC__in_opt IUIAutomationElement *el1, /* [in] */ __RPC__in_opt IUIAutomationElement *el2, /* [retval][out] */ __RPC__out BOOL *areSame)
  ;CompareRuntimeIds(/* [in] */ __RPC__in SAFEARRAY * runtimeId1, /* [in] */ __RPC__in SAFEARRAY * runtimeId2, /* [retval][out] */ __RPC__out BOOL *areSame)
  ;GetRootElement(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **root)
  ;ElementFromHandle(/* [in] */ __RPC__in UIA_HWND hwnd, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;ElementFromPoint(/* [in] */ POINT pt, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;GetFocusedElement(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;GetRootElementBuildCache(/* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **root)
  ;ElementFromHandleBuildCache(/* [in] */ __RPC__in UIA_HWND hwnd, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;ElementFromPointBuildCache(/* [in] */ POINT pt, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;GetFocusedElementBuildCache(/* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;CreateTreeWalker(/* [in] */ __RPC__in_opt IUIAutomationCondition *pCondition, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationTreeWalker **walker)
  ;get_ControlViewWalker(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTreeWalker **walker)
  ;get_ContentViewWalker(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTreeWalker **walker)
  ;get_RawViewWalker(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTreeWalker **walker)
  ;get_RawViewCondition(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **condition)
  ;get_ControlViewCondition(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **condition)
  ;get_ContentViewCondition(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **condition)
  ;CreateCacheRequest(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCacheRequest **cacheRequest)
  ;CreateTrueCondition(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateFalseCondition(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreatePropertyCondition(/* [in] */ PROPERTYID propertyId, /* [in] */ VARIANT value, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreatePropertyConditionEx(/* [in] */ PROPERTYID propertyId, /* [in] */ VARIANT value, /* [in] */ enum PropertyConditionFlags flags, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateAndCondition(/* [in] */ __RPC__in_opt IUIAutomationCondition *condition1, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition2, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateAndConditionFromArray(/* [in] */ __RPC__in_opt SAFEARRAY * conditions, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateAndConditionFromNativeArray(/* [size_is][in] */ __RPC__in_ecount_full(conditionCount) IUIAutomationCondition **conditions, /* [in] */ int conditionCount, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateOrCondition(/* [in] */ __RPC__in_opt IUIAutomationCondition *condition1, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition2, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateOrConditionFromArray(/* [in] */ __RPC__in_opt SAFEARRAY * conditions, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateOrConditionFromNativeArray(/* [size_is][in] */ __RPC__in_ecount_full(conditionCount) IUIAutomationCondition **conditions, /* [in] */ int conditionCount, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;CreateNotCondition(/* [in] */ __RPC__in_opt IUIAutomationCondition *condition, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationCondition **newCondition)
  ;AddAutomationEventHandler(/* [in] */ EVENTID eventId, /* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [in] */ __RPC__in_opt IUIAutomationEventHandler *handler)
  ;RemoveAutomationEventHandler(/* [in] */ EVENTID eventId, /* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ __RPC__in_opt IUIAutomationEventHandler *handler)
  ;AddPropertyChangedEventHandlerNativeArray(/* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [in] */ __RPC__in_opt IUIAutomationPropertyChangedEventHandler *handler, /* [size_is][in] */ __RPC__in_ecount_full(propertyCount) PROPERTYID *propertyArray, /* [in] */ int propertyCount)
  ;AddPropertyChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [in] */ __RPC__in_opt IUIAutomationPropertyChangedEventHandler *handler, /* [in] */ __RPC__in SAFEARRAY * propertyArray)
  ;RemovePropertyChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ __RPC__in_opt IUIAutomationPropertyChangedEventHandler *handler)
  ;AddStructureChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [in] */ __RPC__in_opt IUIAutomationStructureChangedEventHandler *handler)
  ;RemoveStructureChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationElement *element, /* [in] */ __RPC__in_opt IUIAutomationStructureChangedEventHandler *handler)
  ;AddFocusChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [in] */ __RPC__in_opt IUIAutomationFocusChangedEventHandler *handler)
  ;RemoveFocusChangedEventHandler(/* [in] */ __RPC__in_opt IUIAutomationFocusChangedEventHandler *handler)
  ;RemoveAllEventHandlers()
  ;IntNativeArrayToSafeArray(/* [size_is][in] */ __RPC__in_ecount_full(arrayCount) int *array, /* [in] */ int arrayCount, /* [retval][out] */ __RPC__deref_out_opt SAFEARRAY * *safeArray)
  ;IntSafeArrayToNativeArray(/* [in] */ __RPC__in SAFEARRAY * intArray, /* [size_is][size_is][out] */ __RPC__deref_out_ecount_full_opt(*arrayCount) int **array, /* [retval][out] */ __RPC__out int *arrayCount)
  ;RectToVariant(/* [in] */ RECT rc, /* [retval][out] */ __RPC__out VARIANT *var)
  ;VariantToRect(/* [in] */ VARIANT var, /* [retval][out] */ __RPC__out RECT *rc)
  ;SafeArrayToRectNativeArray(/* [in] */ __RPC__in SAFEARRAY * rects, /* [size_is][size_is][out] */ __RPC__deref_out_ecount_full_opt(*rectArrayCount) RECT **rectArray, /* [retval][out] */ __RPC__out int *rectArrayCount)
  ;CreateProxyFactoryEntry(/* [in] */ __RPC__in_opt IUIAutomationProxyFactory *factory, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationProxyFactoryEntry **factoryEntry)
  ;get_ProxyFactoryMapping(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationProxyFactoryMapping **factoryMapping)
  ;GetPropertyProgrammaticName(/* [in] */ PROPERTYID property, /* [retval][out] */ __RPC__deref_out_opt BSTR *name)
  ;GetPatternProgrammaticName(/* [in] */ PATTERNID pattern, /* [retval][out] */ __RPC__deref_out_opt BSTR *name)
  ;PollForPotentialSupportedPatterns(/* [in] */ __RPC__in_opt IUIAutomationElement *pElement, /* [out] */ __RPC__deref_out_opt SAFEARRAY * *patternIds, /* [out] */ __RPC__deref_out_opt SAFEARRAY * *patternNames)
  ;PollForPotentialSupportedProperties(/* [in] */ __RPC__in_opt IUIAutomationElement *pElement, /* [out] */ __RPC__deref_out_opt SAFEARRAY * *propertyIds, /* [out] */ __RPC__deref_out_opt SAFEARRAY * *propertyNames)
  ;CheckNotSupported(/* [in] */ VARIANT value, /* [retval][out] */ __RPC__out BOOL *isNotSupported)
  ;get_ReservedNotSupportedValue(/* [retval][out] */ __RPC__deref_out_opt IUnknown **notSupportedValue)
  ;get_ReservedMixedAttributeValue(/* [retval][out] */ __RPC__deref_out_opt IUnknown **mixedAttributeValue)
  ;ElementFromIAccessible(/* [in] */ __RPC__in_opt IAccessible *accessible, /* [in] */ int childId, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  ;ElementFromIAccessibleBuildCache(/* [in] */ __RPC__in_opt IAccessible *accessible, /* [in] */ int childId, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **element)
  
  CompareElements.l(*el1, *el2, *areSame)
  CompareRuntimeIds.l(*runtimeId1, *runtimeId2, *areSame)
  GetRootElement.l(*pRoot)
  ElementFromHandle.l(hwnd, *pElement)
  ElementFromPoint.l(pt.q, *pElement)
  GetFocusedElement.l(*pElement)
  GetRootElementBuildCache.l(*cacheRequest, *pRoot)
  ElementFromHandleBuildCache.l(hwnd, *cacheRequest, *pElement)
  ElementFromPointBuildCache.l(pt.q, *cacheRequest, *pElement)
  GetFocusedElementBuildCache.l(*cacheRequest, *pElement)
  CreateTreeWalker.l(*pCondition, *pWalker)
  get_ControlViewWalker.l(*pWalker)
  get_ContentViewWalker.l(*pWalker)
  get_RawViewWalker.l(*pWalker)
  get_RawViewCondition.l(*pCondition)
  get_ControlViewCondition.l(*pCondition)
  get_ContentViewCondition.l(*pCondition)
  CreateCacheRequest.l(*pCacheRequest)
  CreateTrueCondition.l(*pNewCondition)
  CreateFalseCondition.l(*pNewCondition)
  CreatePropertyCondition.l(propertyId, value.p-variant, *pNewCondition)
  CreatePropertyConditionEx.l(propertyId, value.p-variant, flags, *pNewCondition)
  CreateAndCondition.l(*condition1, *condition2, *pNewCondition)
  CreateAndConditionFromArray.l(*conditions, *pNewCondition)
  CreateAndConditionFromNativeArray.l(*pConditions, conditionCount, *pNewCondition)
  CreateOrCondition.l(*condition1, *condition2, *pNewCondition)
  CreateOrConditionFromArray.l(*conditions, *pNewCondition)
  CreateOrConditionFromNativeArray.l(*pConditions, conditionCount, *pNewCondition)
  CreateNotCondition.l(*condition, *pNewCondition)
  AddAutomationEventHandler.l(eventId, *element, scope, *cacheRequest, *handler)
  RemoveAutomationEventHandler.l(eventId, *element, *handler)
  AddPropertyChangedEventHandlerNativeArray.l(*element, scope, *cacheRequest, *handler, *propertyArray, propertyCount)
  AddPropertyChangedEventHandler.l(*element, scope, *cacheRequest, *handler, *propertyArray)
  RemovePropertyChangedEventHandler.l(*element, *handler)
  AddStructureChangedEventHandler.l(*element, scope, *cacheRequest, *handler)
  RemoveStructureChangedEventHandler.l(*element, *handler)
  AddFocusChangedEventHandler.l(*cacheRequest, *handler)
  RemoveFocusChangedEventHandler.l(*handler)
  RemoveAllEventHandlers.l()
  IntNativeArrayToSafeArray.l(*array, arrayCount, *safeArray)
  IntSafeArrayToNativeArray.l(*intArray, *pArray, *arrayCount)
  RectToVariant.l(rc, *var)
  VariantToRect.l(var.p-variant, *rc)
  SafeArrayToRectNativeArray.l(*rects, *pRectArray, *rectArrayCount)
  CreateProxyFactoryEntry.l(*factory, *pFactoryEntry)
  get_ProxyFactoryMapping.l(*pFactoryMapping)
  GetPropertyProgrammaticName.l(property, *name)
  GetPatternProgrammaticName.l(pattern, *name)
  PollForPotentialSupportedPatterns.l(*pElement, *pPatternIds, *pPatternNames)
  PollForPotentialSupportedProperties.l(*pElement, *pPropertyIds, *pPropertyNames)
  CheckNotSupported.l(value.p-variant, *isNotSupported)
  get_ReservedNotSupportedValue.l(*pNotSupportedValue)
  get_ReservedMixedAttributeValue.l(*pMixedAttributeValue)
  ElementFromIAccessible.l(*accessible, childId, *pElement)
  ElementFromIAccessibleBuildCache.l(*accessible, childId, *cacheRequest, *pElement)
EndInterface

Interface IUIAutomation2 Extends IUIAutomation
  ;get_AutoSetFocus(/* [retval][out] */ __RPC__out BOOL *autoSetFocus)
  ;put_AutoSetFocus(/* [in] */ BOOL autoSetFocus)
  ;get_ConnectionTimeout(/* [retval][out] */ __RPC__out DWORD *timeout)
  ;put_ConnectionTimeout(/* [in] */ DWORD timeout)
  ;get_TransactionTimeout(/* [retval][out] */ __RPC__out DWORD *timeout)
  ;put_TransactionTimeout(/* [in] */ DWORD timeout)
  
  get_AutoSetFocus.l(*autoSetFocus)
  put_AutoSetFocus.l(autoSetFocus.l)
  get_ConnectionTimeout.l(*timeout)
  put_ConnectionTimeout.l(timeout.l)
  get_TransactionTimeout.l(*timeout)
  put_TransactionTimeout.l(timeout.l)
EndInterface

Interface IUIAutomationElement Extends IUnknown
  ;SetFocus()
  ;GetRuntimeId(/* [retval][out] */ __RPC__deref_out_opt SAFEARRAY * *runtimeId)
  ;FindFirst(/* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **found)
  ;FindAll(/* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **found)
  ;FindFirstBuildCache(/* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **found)
  ;FindAllBuildCache(/* [in] */ enum TreeScope scope, /* [in] */ __RPC__in_opt IUIAutomationCondition *condition, /* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **found)
  ;BuildUpdatedCache(/* [in] */ __RPC__in_opt IUIAutomationCacheRequest *cacheRequest, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **updatedElement)
  ;GetCurrentPropertyValue(/* [in] */ PROPERTYID propertyId, /* [retval][out] */ __RPC__out VARIANT *retVal)
  ;GetCurrentPropertyValueEx(/* [in] */ PROPERTYID propertyId, /* [in] */ BOOL ignoreDefaultValue, /* [retval][out] */ __RPC__out VARIANT *retVal)
  ;GetCachedPropertyValue(/* [in] */ PROPERTYID propertyId, /* [retval][out] */ __RPC__out VARIANT *retVal)
  ;GetCachedPropertyValueEx(/* [in] */ PROPERTYID propertyId, /* [in] */ BOOL ignoreDefaultValue, /* [retval][out] */ __RPC__out VARIANT *retVal)
  ;GetCurrentPatternAs(/* [in] */ PATTERNID patternId, /* [in] */ __RPC__in REFIID riid, /* [retval][iid_is][out] */ __RPC__deref_out_opt void **patternObject)
  ;GetCachedPatternAs(/* [in] */ PATTERNID patternId, /* [in] */ __RPC__in REFIID riid, /* [retval][iid_is][out] */ __RPC__deref_out_opt void **patternObject)
  ;GetCurrentPattern(/* [in] */ PATTERNID patternId, /* [retval][out] */ __RPC__deref_out_opt IUnknown **patternObject)
  ;GetCachedPattern(/* [in] */ PATTERNID patternId, /* [retval][out] */ __RPC__deref_out_opt IUnknown **patternObject)
  ;GetCachedParent(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **parent)
  ;GetCachedChildren(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **children)
  ;get_CurrentProcessId(/* [retval][out] */ __RPC__out int *retVal)
  ;get_CurrentControlType(/* [retval][out] */ __RPC__out CONTROLTYPEID *retVal)
  ;get_CurrentLocalizedControlType(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentName(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentAcceleratorKey(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentAccessKey(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentHasKeyboardFocus(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentIsKeyboardFocusable(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentIsEnabled(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentAutomationId(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentClassName(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentHelpText(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentCulture(/* [retval][out] */ __RPC__out int *retVal)
  ;get_CurrentIsControlElement(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentIsContentElement(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentIsPassword(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentNativeWindowHandle(/* [retval][out] */ __RPC__deref_out_opt UIA_HWND *retVal)
  ;get_CurrentItemType(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentIsOffscreen(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentOrientation(/* [retval][out] */ __RPC__out enum OrientationType *retVal)
  ;get_CurrentFrameworkId(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentIsRequiredForForm(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentItemStatus(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentBoundingRectangle(/* [retval][out] */ __RPC__out RECT *retVal)
  ;get_CurrentLabeledBy(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **retVal)
  ;get_CurrentAriaRole(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentAriaProperties(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentIsDataValidForForm(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CurrentControllerFor(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CurrentDescribedBy(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CurrentFlowsTo(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CurrentProviderDescription(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedProcessId(/* [retval][out] */ __RPC__out int *retVal)
  ;get_CachedControlType(/* [retval][out] */ __RPC__out CONTROLTYPEID *retVal)
  ;get_CachedLocalizedControlType(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedName(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedAcceleratorKey(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedAccessKey(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedHasKeyboardFocus(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedIsKeyboardFocusable(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedIsEnabled(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedAutomationId(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedClassName(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedHelpText(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedCulture(/* [retval][out] */ __RPC__out int *retVal)
  ;get_CachedIsControlElement(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedIsContentElement(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedIsPassword(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedNativeWindowHandle(/* [retval][out] */ __RPC__deref_out_opt UIA_HWND *retVal)
  ;get_CachedItemType(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedIsOffscreen(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedOrientation(/* [retval][out] */ __RPC__out enum OrientationType *retVal)
  ;get_CachedFrameworkId(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedIsRequiredForForm(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedItemStatus(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedBoundingRectangle(/* [retval][out] */ __RPC__out RECT *retVal)
  ;get_CachedLabeledBy(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **retVal)
  ;get_CachedAriaRole(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedAriaProperties(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedIsDataValidForForm(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedControllerFor(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CachedDescribedBy(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CachedFlowsTo(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **retVal)
  ;get_CachedProviderDescription(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;GetClickablePoint(/* [out] */ __RPC__out POINT *clickable, /* [retval][out] */ __RPC__out BOOL *gotClickable)
  
  SetFocus.l()
  GetRuntimeId.l(*runtimeId)
  FindFirst.l(scope, *condition, *pFound)
  FindAll.l(scope, *condition, *pFound)
  FindFirstBuildCache.l(scope, *condition, *cacheRequest, *pFound)
  FindAllBuildCache.l(scope, *condition, *cacheRequest, *pFound)
  BuildUpdatedCache.l(*cacheRequest, *pUpdatedElement)
  GetCurrentPropertyValue.l(propertyId, *retVal)
  GetCurrentPropertyValueEx.l(propertyId, ignoreDefaultValue, *retVal)
  GetCachedPropertyValue.l(propertyId, *retVal)
  GetCachedPropertyValueEx.l(propertyId, ignoreDefaultValue, *retVal)
  GetCurrentPatternAs.l(patternId, riid, *pPatternObject)
  GetCachedPatternAs.l(patternId, riid, *pPatternObject)
  GetCurrentPattern.l(patternId, *pPatternObject)
  GetCachedPattern.l(patternId, *pPatternObject)
  GetCachedParent.l(*pParent)
  GetCachedChildren.l(*pChildren)
  get_CurrentProcessId.l(*retVal)
  get_CurrentControlType.l(*retVal)
  get_CurrentLocalizedControlType.l(*retVal)
  get_CurrentName.l(*retVal)
  get_CurrentAcceleratorKey.l(*retVal)
  get_CurrentAccessKey.l(*retVal)
  get_CurrentHasKeyboardFocus.l(*retVal)
  get_CurrentIsKeyboardFocusable.l(*retVal)
  get_CurrentIsEnabled.l(*retVal)
  get_CurrentAutomationId.l(*retVal)
  get_CurrentClassName.l(*retVal)
  get_CurrentHelpText.l(*retVal)
  get_CurrentCulture.l(*retVal)
  get_CurrentIsControlElement.l(*retVal)
  get_CurrentIsContentElement.l(*retVal)
  get_CurrentIsPassword.l(*retVal)
  get_CurrentNativeWindowHandle.l(*retVal)
  get_CurrentItemType.l(*retVal)
  get_CurrentIsOffscreen.l(*retVal)
  get_CurrentOrientation.l(*retVal)
  get_CurrentFrameworkId.l(*retVal)
  get_CurrentIsRequiredForForm.l(*retVal)
  get_CurrentItemStatus.l(*retVal)
  get_CurrentBoundingRectangle.l(*retVal)
  get_CurrentLabeledBy.l(*pRetVal)
  get_CurrentAriaRole.l(*retVal)
  get_CurrentAriaProperties.l(*retVal)
  get_CurrentIsDataValidForForm.l(*retVal)
  get_CurrentControllerFor.l(*pRetVal)
  get_CurrentDescribedBy.l(*pRetVal)
  get_CurrentFlowsTo.l(*pRetVal)
  get_CurrentProviderDescription.l(*retVal)
  get_CachedProcessId.l(*retVal)
  get_CachedControlType.l(*retVal)
  get_CachedLocalizedControlType.l(*retVal)
  get_CachedName.l(*retVal)
  get_CachedAcceleratorKey.l(*retVal)
  get_CachedAccessKey.l(*retVal)
  get_CachedHasKeyboardFocus.l(*retVal)
  get_CachedIsKeyboardFocusable.l(*retVal)
  get_CachedIsEnabled.l(*retVal)
  get_CachedAutomationId.l(*retVal)
  get_CachedClassName.l(*retVal)
  get_CachedHelpText.l(*retVal)
  get_CachedCulture.l(*retVal)
  get_CachedIsControlElement.l(*retVal)
  get_CachedIsContentElement.l(*retVal)
  get_CachedIsPassword.l(*retVal)
  get_CachedNativeWindowHandle.l(*retVal)
  get_CachedItemType.l(*retVal)
  get_CachedIsOffscreen.l(*retVal)
  get_CachedOrientation.l(*retVal)
  get_CachedFrameworkId.l(*retVal)
  get_CachedIsRequiredForForm.l(*retVal)
  get_CachedItemStatus.l(*retVal)
  get_CachedBoundingRectangle.l(*retVal)
  get_CachedLabeledBy.l(*pRetVal)
  get_CachedAriaRole.l(*retVal)
  get_CachedAriaProperties.l(*retVal)
  get_CachedIsDataValidForForm.l(*retVal)
  get_CachedControllerFor.l(*pRetVal)
  get_CachedDescribedBy.l(*pRetVal)
  get_CachedFlowsTo.l(*pRetVal)
  get_CachedProviderDescription.l(*retVal)
  GetClickablePoint.l(*clickable, *gotClickable)
EndInterface

Interface IUIAutomationValuePattern Extends IUnknown
  ;SetValue(/* [in] */ __RPC__in BSTR val)
  ;get_CurrentValue(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CurrentIsReadOnly(/* [retval][out] */ __RPC__out BOOL *retVal)
  ;get_CachedValue(/* [retval][out] */ __RPC__deref_out_opt BSTR *retVal)
  ;get_CachedIsReadOnly(/* [retval][out] */ __RPC__out BOOL *retVal)
  
  SetValue.l(val.p-bstr)
  get_CurrentValue.l(*retVal)
  get_CurrentIsReadOnly.l(*retVal)
  get_CachedValue.l(*retVal)
  get_CachedIsReadOnly.l(*retVal)
EndInterface

Interface IUIAutomationTextPattern Extends IUnknown
  ;RangeFromPoint(/* [in] */ POINT pt, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **range)
  ;RangeFromChild(/* [in] */ __RPC__in_opt IUIAutomationElement *child, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **range)
  ;GetSelection(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRangeArray **ranges)
  ;GetVisibleRanges(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRangeArray **ranges)
  ;get_DocumentRange(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **range)
  ;get_SupportedTextSelection(/* [retval][out] */ __RPC__out enum SupportedTextSelection *supportedTextSelection)
  
  RangeFromPoint.l(pt.q, *pRange)
  RangeFromChild.l(*child, *pRange)
  GetSelection.l(*pRanges)
  GetVisibleRanges.l(*pRanges)
  get_DocumentRange.l(*pRange)
  get_SupportedTextSelection.l(*supportedTextSelection)
EndInterface

Interface IUIAutomationTextRange Extends IUnknown
  ;Clone(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **clonedRange)
  ;Compare(/* [in] */ __RPC__in_opt IUIAutomationTextRange *range, /* [retval][out] */ __RPC__out BOOL *areSame)
  ;CompareEndpoints(/* [in] */ enum TextPatternRangeEndpoint srcEndPoint, /* [in] */ __RPC__in_opt IUIAutomationTextRange *range, /* [in] */ enum TextPatternRangeEndpoint targetEndPoint, /* [retval][out] */ __RPC__out int *compValue)
  ;ExpandToEnclosingUnit(/* [in] */ enum TextUnit textUnit)
  ;FindAttribute(/* [in] */ TEXTATTRIBUTEID attr, /* [in] */ VARIANT val, /* [in] */ BOOL backward, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **found)
  ;FindText(/* [in] */ __RPC__in BSTR text, /* [in] */ BOOL backward, /* [in] */ BOOL ignoreCase, /* [retval][out] */ __RPC__deref_out_opt IUIAutomationTextRange **found)
  ;GetAttributeValue(/* [in] */ TEXTATTRIBUTEID attr, /* [retval][out] */ __RPC__out VARIANT *value)
  ;GetBoundingRectangles(/* [retval][out] */ __RPC__deref_out_opt SAFEARRAY * *boundingRects)
  ;GetEnclosingElement(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElement **enclosingElement)
  ;GetText(/* [in] */ int maxLength, /* [retval][out] */ __RPC__deref_out_opt BSTR *text)
  ;Move(/* [in] */ enum TextUnit unit, /* [in] */ int count, /* [retval][out] */ __RPC__out int *moved)
  ;MoveEndpointByUnit(/* [in] */ enum TextPatternRangeEndpoint endpoint, /* [in] */ enum TextUnit unit, /* [in] */ int count, /* [retval][out] */ __RPC__out int *moved)
  ;MoveEndpointByRange(/* [in] */ enum TextPatternRangeEndpoint srcEndPoint, /* [in] */ __RPC__in_opt IUIAutomationTextRange *range, /* [in] */ enum TextPatternRangeEndpoint targetEndPoint)
  ;Select()
  ;AddToSelection()
  ;RemoveFromSelection()
  ;ScrollIntoView(/* [in] */ BOOL alignToTop)
  ;GetChildren(/* [retval][out] */ __RPC__deref_out_opt IUIAutomationElementArray **children)
  
  Clone.l(*pClonedRange)
  Compare.l(*range, *areSame)
  CompareEndpoints.l(srcEndPoint, *range, targetEndPoint, *compValue)
  ExpandToEnclosingUnit.l(textUnit)
  FindAttribute.l(attr, val.p-variant, backward, *pFound)
  FindText.l(text, backward, ignoreCase, *pFound)
  GetAttributeValue.l(attr, *value)
  GetBoundingRectangles.l(*pBoundingRects)
  GetEnclosingElement.l(*pEnclosingElement)
  GetText.l(maxLength, *text)
  Move.l(unit, count, *moved)
  MoveEndpointByUnit.l(endpoint, unit, count, *moved)
  MoveEndpointByRange.l(srcEndPoint, *range, targetEndPoint)
  Select__.l()
  AddToSelection.l()
  RemoveFromSelection.l()
  ScrollIntoView.l(alignToTop)
  GetChildren.l(*pChildren)
EndInterface

Interface IUIAutomationCondition Extends IUnknown
EndInterface

Interface IUIAutomationPropertyChangedEventHandler Extends IUnknown
  ;HandlePropertyChangedEvent(/* [in] */ __RPC__in_opt IUIAutomationElement *sender, /* [in] */ PROPERTYID propertyId, /* [in] */ VARIANT newValue)
  HandlePropertyChangedEvent.l(*sender, propertyId, newValue.p-variant)
EndInterface

Interface IUIAutomationEventHandler Extends IUnknown
  ;HandleAutomationEvent(/* [in] */ __RPC__in_opt IUIAutomationElement *sender, /* [in] */ EVENTID eventId)
  HandleAutomationEvent.l(*sender, eventId)
EndInterface

DataSection
  CLSID_CUIAutomation:
  Data.l $ff48dba4
  Data.w $60ef, $4201
  Data.b $aa, $87, $54, $10, $3e, $ef, $59, $4e
  
  CLSID_CUIAutomation8:
  Data.l $e22ad333
  Data.w $b25f, $460c
  Data.b $83, $d0, $05, $81, $10, $73, $95, $c9
  
  IID_IUnknown:
  Data.l $00000000
  Data.w $0000, $0000
  Data.b $C0, $00, $00, $00, $00, $00, $00, $46
  
  IID_IUIAutomation:
  Data.l $30cbe57d
  Data.w $d9d0, $452a
  Data.b $ab, $13, $7a, $c5, $ac, $48, $25, $ee
  
  IID_IUIAutomation2:
  Data.l $34723aff
  Data.w $0c9d, $49d0
  Data.b $98, $96, $7a, $b5, $2d, $f8, $cd, $8a
  
  IID_IUIAutomationPropertyChangedEventHandler:
  Data.l $40cd37d4
  Data.w $c756, $4b0c
  Data.b $8c, $6f, $bd, $df, $ee, $b1, $3b, $50
  
  IID_IUIAutomationEventHandler:
  Data.l $146c3c17
  Data.w $f12e, $4e22
  Data.b $8c, $27, $f8, $94, $b9, $b7, $9c, $69
EndDataSection

Example:
Get the speed information from a copy status window of Windows Explorer.

Code: Select all

Prototype ptQueryFullProcessImageName(hProcess, dwFlags.l, *lpExeName, *lpdwSize)
Global QueryFullProcessImageName.ptQueryFullProcessImageName

Global hEventExit

Procedure.s GetFileNameFromWindow(hWnd)
  Protected LibKernel32 = OpenLibrary(#PB_Any, "Kernel32.dll")
  Protected Name.s{1024}, Size.l = 1023
  
  If LibKernel32
    QueryFullProcessImageName = GetFunction(LibKernel32, "QueryFullProcessImageNameW")
    If QueryFullProcessImageName
      If GetWindowThreadProcessId_(hWnd, @PID)
        If PID
          hProcess = OpenProcess_(#PROCESS_QUERY_INFORMATION, 0, PID)
          If hProcess
            QueryFullProcessImageName(hProcess, 0, @Name, @Size)
            CloseHandle_(hProcess)
          EndIf
        EndIf
      EndIf
    EndIf
    CloseLibrary(LibKernel32)
  EndIf
  
  ProcedureReturn Name
EndProcedure

Structure STRUC_IUnknownBase
  *pVtbl
  lRefCount.l
EndStructure

Structure STRUC_IUnknownVtbl Extends STRUC_IUnknownBase
  *pQueryInterface
  *pAddRef
  *pRelease
EndStructure

Structure STRUC_IUIAutomationPropertyChangedEventHandlerVtbl Extends STRUC_IUnknownVtbl
  *pHandlePropertyChangedEvent
EndStructure

Structure STRUC_IUIAutomationEventHandlerVtbl Extends STRUC_IUnknownVtbl
  *pHandleAutomationEvent
EndStructure


;-*** How to Implement Event Handlers
; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-howto-implement-event-handlers

Procedure.l IUIAutomationPropertyChangedEventHandler_QueryInterface(*this.STRUC_IUnknownBase, *riid.IID, *ppvObject.Integer)
  If *ppvObject And *riid
    If CompareMemory(*riid, ?IID_IUnknown, SizeOf(IID)) Or CompareMemory(*riid, ?IID_IUIAutomationPropertyChangedEventHandler, SizeOf(IID))
      *this\lRefCount + 1
      *ppvObject\i = *this
    Else
      *ppvObject\i = 0
      ProcedureReturn #E_NOINTERFACE
    EndIf
  Else
    ProcedureReturn #E_POINTER
  EndIf
  
  ProcedureReturn #S_OK
EndProcedure

Procedure.l IUIAutomationEventHandler_QueryInterface(*this.STRUC_IUnknownBase, *riid.IID, *ppvObject.Integer)
  If *ppvObject And *riid
    If CompareMemory(*riid, ?IID_IUnknown, SizeOf(IID)) Or CompareMemory(*riid, ?IID_IUIAutomationEventHandler, SizeOf(IID))
      *this\lRefCount + 1
      *ppvObject\i = *this
    Else
      *ppvObject\i = 0
      ProcedureReturn #E_NOINTERFACE
    EndIf
  Else
    ProcedureReturn #E_POINTER
  EndIf
  
  ProcedureReturn #S_OK
EndProcedure

Procedure.l IUnknown_AddRef(*this.STRUC_IUnknownBase)
  *this\lRefCount + 1
  ProcedureReturn *this\lRefCount
EndProcedure

Procedure.l IUnknown_Release(*this.STRUC_IUnknownBase)
  *this\lRefCount - 1
  ProcedureReturn *this\lRefCount
EndProcedure

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  Procedure.l IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent(*this, *sender.IUIAutomationElement, propertyId, *newValue.VARIANT)
    ;Debug propertyId
    
    If propertyId = #UIA_NamePropertyId
      If *newValue
        If *newValue\vt = #VT_BSTR And *newValue\bstrVal
          Name$ = PeekS(*newValue\bstrVal)
          ;VariantClear_(*newValue)   ;necessary?
          
          Debug Name$
        EndIf
      EndIf
    EndIf
    
    ProcedureReturn #S_OK
  EndProcedure
  
CompilerElse
  
  Procedure.l IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent(*this, *sender.IUIAutomationElement, propertyId, newValue1.q, newValue2.q)
    Protected *newValue.VARIANT
    
    If propertyId = #UIA_NamePropertyId
      *newValue = @newValue1
      If *newValue
        If *newValue\vt = #VT_BSTR And *newValue\bstrVal
          Name$ = PeekS(*newValue\bstrVal)
          ;VariantClear_(*newValue)   ;necessary?
          
          Debug Name$
        EndIf
      EndIf
    EndIf
    
    ProcedureReturn #S_OK
  EndProcedure
CompilerEndIf

Procedure.l IUIAutomationEventHandler_HandleAutomationEvent(*this, *sender.IUIAutomationElement, eventId)
  ;Debug eventId
  
  If eventId = #UIA_Window_WindowClosedEventId
    SetEvent_(hEventExit)
  EndIf
  ProcedureReturn #S_OK
EndProcedure

Procedure EnumChildProc(hWnd, *lParam.Integer)
  Protected sClassName.s{128}
  If hWnd
    If GetClassName_(hWnd, @sClassName, 127)
      ;If sClassName = "RateChartOverlayWindow"
      If sClassName = "CvChartWindow"
        *lParam\i = hWnd
        ProcedureReturn 0
      EndIf
    EndIf
  EndIf
  ProcedureReturn 1
EndProcedure

Procedure GetExplorerStatusInfo()
  Protected *NameString, Name$
  Protected hWndParent, hWnd
  Protected sFilename.s
  Protected *UIAutomationEventHandler.STRUC_IUIAutomationEventHandlerVtbl
  Protected *UIAutomationPropertyChangedEventHandler.STRUC_IUIAutomationPropertyChangedEventHandlerVtbl
  Protected Value.VARIANT
  Protected PropertyID = #UIA_NamePropertyId
  
  ;Search a status window of Explorer. 
  Repeat
    hWndParent = FindWindowEx_(0, hWndParent, "OperationStatusWindow", 0)
    If hWndParent
      sFilename = GetFileNameFromWindow(hWndParent)
      If sFilename
        If FindString(sFilename, "explorer.exe", 1, #PB_String_NoCase)
          EnumChildWindows_(hWndParent, @EnumChildProc(), @hWnd)
          If hWnd
            hWnd = hWndParent
            Break
          EndIf
        EndIf
      EndIf
    EndIf
  Until hWndParent = 0
  
  If hWnd = 0
    Debug "Explorer status window not found."
    ProcedureReturn 0
  EndIf
  
  Repeat
    
    ;If CoCreateInstance_(?CLSID_CUIAutomation8, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation2, @*Automation.IUIAutomation2) <> #S_OK : Break : EndIf
    
    If CoCreateInstance_(?CLSID_CUIAutomation, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation, @*Automation.IUIAutomation) <> #S_OK : Break : EndIf
    If *Automation = 0 : Break : EndIf
    
    
    ;-*** Obtaining UI Automation Elements
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-obtainingelements
    
    If *Automation\ElementFromHandle(hWnd, @*Element.IUIAutomationElement) <> #S_OK : Break : EndIf
    If *Element = 0 : Break : EndIf
    
    
    ; The name of an object can be obtained by "Inspect" in the Windows SDK.
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects
    Value\vt = #VT_BSTR
    Value\bstrVal = SysAllocString_("eltRateChart")
    
    
    ;-*** Retrieving Properties from UI Automation Elements
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-propertiesforclients
    
    If *Automation\CreatePropertyCondition(#UIA_AutomationIdPropertyId, Value, @*PatternCondition.IUIAutomationCondition) <> #S_OK : Break : EndIf
    If *PatternCondition = 0 : Break : EndIf
    
    If *Element\FindFirst(#TreeScope_Subtree, *PatternCondition, @*RateElement.IUIAutomationElement) = #S_OK
      
      If *RateElement
        
        ;-*** UI Automation Control Patterns Overview
        ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-controlpatternsoverview
        
        
        ; In this case, it is the same string as the string in the "*NameString".
        
;         If *RateElement\GetCurrentPattern(#UIA_ValuePatternId, @*ValuePattern.IUIAutomationValuePattern) = #S_OK
;           If *ValuePattern
;             
;             If *ValuePattern\get_CurrentValue(@*ValueString) = #S_OK
;               If *ValueString
;                 Debug PeekS(*ValueString)
;                 SysFreeString_(*ValueString)
;               EndIf
;             EndIf
;             
;             *ValuePattern\Release()
;           EndIf
;         EndIf
        
        
        ; Attach the event handlers.
        *UIAutomationEventHandler = AllocateMemory(SizeOf(STRUC_IUIAutomationEventHandlerVtbl))
        If *UIAutomationEventHandler
          With *UIAutomationEventHandler
            \pVtbl = *UIAutomationEventHandler + OffsetOf(STRUC_IUIAutomationEventHandlerVtbl\pQueryInterface)
            \pQueryInterface = @IUIAutomationEventHandler_QueryInterface()
            \pAddRef = @IUnknown_AddRef()
            \pRelease = @IUnknown_Release()
            \pHandleAutomationEvent = @IUIAutomationEventHandler_HandleAutomationEvent()
          EndWith
          
          hEventExit = CreateEvent_(0, 0, 0, 0)
          
          If *Automation\AddAutomationEventHandler(#UIA_Window_WindowClosedEventId, *Element, #TreeScope_Subtree, 0, *UIAutomationEventHandler) = #S_OK
            Debug "OK 1"
            
            *UIAutomationPropertyChangedEventHandler = AllocateMemory(SizeOf(STRUC_IUIAutomationPropertyChangedEventHandlerVtbl))
            If *UIAutomationPropertyChangedEventHandler
              
              With *UIAutomationPropertyChangedEventHandler
                \pVtbl = *UIAutomationPropertyChangedEventHandler + OffsetOf(STRUC_IUIAutomationPropertyChangedEventHandlerVtbl\pQueryInterface)
                \pQueryInterface = @IUIAutomationPropertyChangedEventHandler_QueryInterface()
                \pAddRef = @IUnknown_AddRef()
                \pRelease = @IUnknown_Release()
                \pHandlePropertyChangedEvent = @IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent()
              EndWith
              
              If *Automation\AddPropertyChangedEventHandlerNativeArray(*RateElement, #TreeScope_Subtree, 0, *UIAutomationPropertyChangedEventHandler, @PropertyID, 1) = #S_OK
                Debug "OK 2"
                
                If *RateElement\get_CurrentName(@*NameString) = #S_OK
                  If *NameString
                    Name$ = PeekS(*NameString)
                    SysFreeString_(*NameString)
                    
                    Debug Name$
                  EndIf
                EndIf
                
                ; Another way to get the name string.
                
;                 If *RateElement\GetCurrentPropertyValue(#UIA_NamePropertyId, Value2.VARIANT) = #S_OK
;                   If Value2\bstrVal
;                     Name$ = PeekS(Value2\bstrVal)
;                     SysFreeString_(Value2\bstrVal)
;                     
;                     Debug Name$
;                   EndIf
;                 EndIf
                
                
                ; Wait for the status window to close.
                WaitForSingleObject_(hEventExit, #INFINITE)
                
                Debug "Exit"
                
                *Automation\RemovePropertyChangedEventHandler(*RateElement, *UIAutomationPropertyChangedEventHandler)
              EndIf
              
              FreeMemory(*UIAutomationPropertyChangedEventHandler)
            EndIf
            
            *Automation\RemoveAutomationEventHandler(#UIA_Window_WindowClosedEventId, *Element, *UIAutomationEventHandler)
          EndIf
          
          CloseHandle_(hEventExit)
          
          FreeMemory(*UIAutomationEventHandler)
        EndIf
        
        *RateElement\Release()
      EndIf
    EndIf
    
  Until 1
  
  If Value\bstrVal : SysFreeString_(Value\bstrVal) : EndIf
  If *PatternCondition : *PatternCondition\Release() : EndIf
  If *Element : *Element\Release() : EndIf
  If *Automation : *Automation\Release() : EndIf
EndProcedure

Procedure CoInitUninit()
  Static CoInit, UnInit
  
  If UnInit = 0
    UnInit = 1
    
    CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)      ; #COINIT_MULTITHREADED is essential. If not, the event handlers will not be invoked.
    If CoInit = #RPC_E_CHANGED_MODE
      CoUninitialize_()
      CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)
    EndIf
  Else
    If CoInit = #S_OK Or CoInit = #S_FALSE
      CoUninitialize_()
    EndIf
  EndIf
EndProcedure


CoInitUninit()

GetExplorerStatusInfo()

CoInitUninit()
Last edited by breeze4me on Sun Jul 24, 2022 6:07 pm, edited 1 time in total.
breeze4me
Enthusiast
Enthusiast
Posts: 511
Joined: Thu Mar 09, 2006 9:24 am
Location: S. Kor

Re: [Windows] UI Automation Interface example

Post by breeze4me »

How to get all the information in a status window of Windows Explorer.

Code: Select all

Prototype ptQueryFullProcessImageName(hProcess, dwFlags.l, *lpExeName, *lpdwSize)
Global QueryFullProcessImageName.ptQueryFullProcessImageName

Procedure.s GetFileNameFromWindow(hWnd)
  Protected LibKernel32 = OpenLibrary(#PB_Any, "Kernel32.dll")
  Protected Name.s{1024}, Size.l = 1023
  
  If LibKernel32
    QueryFullProcessImageName = GetFunction(LibKernel32, "QueryFullProcessImageNameW")
    If QueryFullProcessImageName
      If GetWindowThreadProcessId_(hWnd, @PID)
        If PID
          hProcess = OpenProcess_(#PROCESS_QUERY_INFORMATION, 0, PID)
          If hProcess
            QueryFullProcessImageName(hProcess, 0, @Name, @Size)
            CloseHandle_(hProcess)
          EndIf
        EndIf
      EndIf
    EndIf
    CloseLibrary(LibKernel32)
  EndIf
  
  ProcedureReturn Name
EndProcedure

Procedure EnumChildProc(hWnd, *lParam.Integer)
  Protected sClassName.s{128}
  If hWnd
    If GetClassName_(hWnd, @sClassName, 127)
      ;If sClassName = "RateChartOverlayWindow"
      If sClassName = "CvChartWindow"
        *lParam\i = hWnd
        ProcedureReturn 0
      EndIf
    EndIf
  EndIf
  ProcedureReturn 1
EndProcedure

Procedure GetExplorerAllStatusInfos()
  Protected *String
  Protected hWndParent, hWnd
  Protected sFilename.s
  
  ;Search a status window of Explorer. 
  Repeat
    hWndParent = FindWindowEx_(0, hWndParent, "OperationStatusWindow", 0)
    If hWndParent
      sFilename = GetFileNameFromWindow(hWndParent)
      If sFilename
        If FindString(sFilename, "explorer.exe", 1, #PB_String_NoCase)
          EnumChildWindows_(hWndParent, @EnumChildProc(), @hWnd)
          If hWnd
            hWnd = hWndParent
            Break
          EndIf
        EndIf
      EndIf
    EndIf
  Until hWndParent = 0
  
  If hWnd = 0
    Debug "Explorer status window not found."
    ProcedureReturn 0
  EndIf
  
  Repeat
    If CoCreateInstance_(?CLSID_CUIAutomation, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation, @*Automation.IUIAutomation) <> #S_OK : Break : EndIf
    If *Automation = 0 : Break : EndIf
    
    If *Automation\ElementFromHandle(hWnd, @*Element.IUIAutomationElement) <> #S_OK : Break : EndIf
    If *Element = 0 : Break : EndIf
    
    If *Automation\CreateTrueCondition(@*PatternCondition.IUIAutomationCondition) <> #S_OK : Break : EndIf
    If *PatternCondition = 0 : Break : EndIf
    
    If *Element\FindAll(#TreeScope_Subtree, *PatternCondition, @*AllElements.IUIAutomationElementArray) <> #S_OK : Break : EndIf
    If *AllElements = 0 : Break : EndIf
    
    If *AllElements\get_Length(@Count) <> #S_OK : Break : EndIf
    
    Debug "number of all elements: " + Count
    
    Count - 1
    
    For i = 0 To Count
      If *AllElements\GetElement(i, @*EachElement.IUIAutomationElement) = #S_OK
        If *EachElement
          
          If *EachElement\get_CurrentClassName(@*String) = #S_OK
            If *String
              ClassName$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Class Name: " + ClassName$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentAutomationId(@*String) = #S_OK
            If *String
              AutomationId$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Automation Id: " + AutomationId$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentName(@*String) = #S_OK
            If *String
              Name$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Name: " + Name$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentControlType(@ControlType) = #S_OK
            Debug "Control Type: " + ControlType
          EndIf
          
          If *EachElement\get_CurrentHelpText(@*String) = #S_OK
            If *String
              Text$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Help Text: " + Text$
            EndIf
          EndIf
          
          Debug "----------------------------------"
          
          *EachElement\Release()
        EndIf
      EndIf
    Next
    
  Until 1
  
  If *AllElements : *AllElements\Release() : EndIf
  If *PatternCondition : *PatternCondition\Release() : EndIf
  If *Element : *Element\Release() : EndIf
  If *Automation : *Automation\Release() : EndIf
EndProcedure

Procedure CoInitUninit()
  Static CoInit, UnInit
  
  If UnInit = 0
    UnInit = 1
    
    CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)      ; #COINIT_MULTITHREADED is essential. If not, the event handlers will not be invoked.
    If CoInit = #RPC_E_CHANGED_MODE
      CoUninitialize_()
      CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)
    EndIf
  Else
    If CoInit = #S_OK Or CoInit = #S_FALSE
      CoUninitialize_()
    EndIf
  EndIf
EndProcedure


CoInitUninit()

GetExplorerAllStatusInfos()

CoInitUninit()

User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: [Windows] UI Automation Interface example

Post by Kwai chang caine »

Really nice code :shock:
Works nice here :wink:
Really thanks for this sharing 8)

I have a little question :oops:
"AUTOMATION UI" is there a link with the "Automation" (OLE) of Internet explorer or Office for remote Excel, Word, etc ... ?
Believe you it's possible to send order to the objects in addition to read it ?
ImageThe happiness is a road...
Not a destination
breeze4me
Enthusiast
Enthusiast
Posts: 511
Joined: Thu Mar 09, 2006 9:24 am
Location: S. Kor

Re: [Windows] UI Automation Interface example

Post by breeze4me »

Kwai chang caine wrote: Sun Jul 31, 2022 7:22 pm "AUTOMATION UI" is there a link with the "Automation" (OLE) of Internet explorer or Office for remote Excel, Word, etc ... ?
Believe you it's possible to send order to the objects in addition to read it ?
It's "UI" (User interface) Automation. As the name suggests, it is for the user interface, which is different from OLE.

https://en.wikipedia.org/wiki/Microsoft_UI_Automation
Microsoft UI Automation (UIA) is an application programming interface (API) that allows one to access, identify, and manipulate the user interface (UI) elements of another application.
User avatar
chi
Addict
Addict
Posts: 1028
Joined: Sat May 05, 2007 5:31 pm
Location: Linz, Austria

Re: [Windows] UI Automation Interface example

Post by chi »

Hmm, I get the message "Explorer status window not found" for both examples. I have tried copying large files to the desktop or doing some things in Explorer or Notepad, but nothing. I can use any tool on this website to get information about the UI, so do I need to do anything special? I am using Win7 btw.
Et cetera is my worst enemy
breeze4me
Enthusiast
Enthusiast
Posts: 511
Joined: Thu Mar 09, 2006 9:24 am
Location: S. Kor

Re: [Windows] UI Automation Interface example

Post by breeze4me »

The copy status window in Windows 7 has a different class name.
Here is the code that changed the examples for Windows 7.

Code: Select all

Prototype ptQueryFullProcessImageName(hProcess, dwFlags.l, *lpExeName, *lpdwSize)
Global QueryFullProcessImageName.ptQueryFullProcessImageName

Global hEventExit

Procedure.s GetFileNameFromWindow(hWnd)
  Protected LibKernel32 = OpenLibrary(#PB_Any, "Kernel32.dll")
  Protected Name.s{1024}, Size.l = 1023
  
  If LibKernel32
    QueryFullProcessImageName = GetFunction(LibKernel32, "QueryFullProcessImageNameW")
    If QueryFullProcessImageName
      If GetWindowThreadProcessId_(hWnd, @PID)
        If PID
          hProcess = OpenProcess_(#PROCESS_QUERY_INFORMATION, 0, PID)
          If hProcess
            QueryFullProcessImageName(hProcess, 0, @Name, @Size)
            CloseHandle_(hProcess)
          EndIf
        EndIf
      EndIf
    EndIf
    CloseLibrary(LibKernel32)
  EndIf
  
  ProcedureReturn Name
EndProcedure

Structure STRUC_IUnknownBase
  *pVtbl
  lRefCount.l
EndStructure

Structure STRUC_IUnknownVtbl Extends STRUC_IUnknownBase
  *pQueryInterface
  *pAddRef
  *pRelease
EndStructure

Structure STRUC_IUIAutomationPropertyChangedEventHandlerVtbl Extends STRUC_IUnknownVtbl
  *pHandlePropertyChangedEvent
EndStructure

Structure STRUC_IUIAutomationEventHandlerVtbl Extends STRUC_IUnknownVtbl
  *pHandleAutomationEvent
EndStructure


;-*** How to Implement Event Handlers
; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-howto-implement-event-handlers

Procedure.l IUIAutomationPropertyChangedEventHandler_QueryInterface(*this.STRUC_IUnknownBase, *riid.IID, *ppvObject.Integer)
  If *ppvObject And *riid
    If CompareMemory(*riid, ?IID_IUnknown, SizeOf(IID)) Or CompareMemory(*riid, ?IID_IUIAutomationPropertyChangedEventHandler, SizeOf(IID))
      *this\lRefCount + 1
      *ppvObject\i = *this
    Else
      *ppvObject\i = 0
      ProcedureReturn #E_NOINTERFACE
    EndIf
  Else
    ProcedureReturn #E_POINTER
  EndIf
  
  ProcedureReturn #S_OK
EndProcedure

Procedure.l IUIAutomationEventHandler_QueryInterface(*this.STRUC_IUnknownBase, *riid.IID, *ppvObject.Integer)
  If *ppvObject And *riid
    If CompareMemory(*riid, ?IID_IUnknown, SizeOf(IID)) Or CompareMemory(*riid, ?IID_IUIAutomationEventHandler, SizeOf(IID))
      *this\lRefCount + 1
      *ppvObject\i = *this
    Else
      *ppvObject\i = 0
      ProcedureReturn #E_NOINTERFACE
    EndIf
  Else
    ProcedureReturn #E_POINTER
  EndIf
  
  ProcedureReturn #S_OK
EndProcedure

Procedure.l IUnknown_AddRef(*this.STRUC_IUnknownBase)
  *this\lRefCount + 1
  ProcedureReturn *this\lRefCount
EndProcedure

Procedure.l IUnknown_Release(*this.STRUC_IUnknownBase)
  *this\lRefCount - 1
  ProcedureReturn *this\lRefCount
EndProcedure

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
  Procedure.l IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent(*this, *sender.IUIAutomationElement, propertyId, *newValue.VARIANT)
    ;Debug propertyId
    
    If propertyId = #UIA_NamePropertyId
      If *newValue
        If *newValue\vt = #VT_BSTR And *newValue\bstrVal
          Name$ = PeekS(*newValue\bstrVal)
          ;VariantClear_(*newValue)   ;necessary?
          
          Debug Name$
        EndIf
      EndIf
    EndIf
    
    ProcedureReturn #S_OK
  EndProcedure
  
CompilerElse
  
  Procedure.l IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent(*this, *sender.IUIAutomationElement, propertyId, newValue1.q, newValue2.q)
    Protected *newValue.VARIANT
    
    If propertyId = #UIA_NamePropertyId
      *newValue = @newValue1
      If *newValue
        If *newValue\vt = #VT_BSTR And *newValue\bstrVal
          Name$ = PeekS(*newValue\bstrVal)
          ;VariantClear_(*newValue)   ;necessary?
          
          Debug Name$
        EndIf
      EndIf
    EndIf
    
    ProcedureReturn #S_OK
  EndProcedure
CompilerEndIf

Procedure.l IUIAutomationEventHandler_HandleAutomationEvent(*this, *sender.IUIAutomationElement, eventId)
  ;Debug eventId
  
  If eventId = #UIA_Window_WindowClosedEventId
    SetEvent_(hEventExit)
  EndIf
  ProcedureReturn #S_OK
EndProcedure

Procedure EnumChildProc(hWnd, *lParam.Integer)
  Protected sClassName.s{128}
  If hWnd
    If GetClassName_(hWnd, @sClassName, 127)
      ;If sClassName = "RateChartOverlayWindow"
      If sClassName = "CvChartWindow"
        *lParam\i = hWnd
        ProcedureReturn 0
      EndIf
    EndIf
  EndIf
  ProcedureReturn 1
EndProcedure

Procedure GetExplorerStatusInfo()
  Protected *NameString, Name$
  Protected hWndParent, hWnd
  Protected sFilename.s
  Protected *UIAutomationEventHandler.STRUC_IUIAutomationEventHandlerVtbl
  Protected *UIAutomationPropertyChangedEventHandler.STRUC_IUIAutomationPropertyChangedEventHandlerVtbl
  Protected Value.VARIANT
  Protected PropertyID = #UIA_NamePropertyId
  
  ;Search a status window of Explorer. 
  Repeat
    hWndParent = FindWindowEx_(0, hWndParent, "#32770", 0)
    If hWndParent
      sFilename = GetFileNameFromWindow(hWndParent)
      If sFilename
        If FindString(sFilename, "explorer.exe", 1, #PB_String_NoCase)
          hWnd = hWndParent
          Break
        EndIf
      EndIf
    EndIf
  Until hWndParent = 0
  
  If hWnd = 0
    Debug "Explorer status window not found."
    ProcedureReturn 0
  EndIf
  
  Repeat
    
    ;If CoCreateInstance_(?CLSID_CUIAutomation8, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation2, @*Automation.IUIAutomation2) <> #S_OK : Break : EndIf
    
    If CoCreateInstance_(?CLSID_CUIAutomation, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation, @*Automation.IUIAutomation) <> #S_OK : Break : EndIf
    If *Automation = 0 : Break : EndIf
    
    
    ;-*** Obtaining UI Automation Elements
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-obtainingelements
    
    If *Automation\ElementFromHandle(hWnd, @*Element.IUIAutomationElement) <> #S_OK : Break : EndIf
    If *Element = 0 : Break : EndIf
    
    
    ; The name of an object can be obtained by "Inspect" in the Windows SDK.
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects
    Value\vt = #VT_BSTR
    Value\bstrVal = SysAllocString_("XProgressSpeed")
    
    
    ;-*** Retrieving Properties from UI Automation Elements
    ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-propertiesforclients
    
    If *Automation\CreatePropertyCondition(#UIA_AutomationIdPropertyId, Value, @*PatternCondition.IUIAutomationCondition) <> #S_OK : Break : EndIf
    If *PatternCondition = 0 : Break : EndIf
    
    If *Element\FindFirst(#TreeScope_Subtree, *PatternCondition, @*RateElement.IUIAutomationElement) = #S_OK
      
      If *RateElement
        
        ;-*** UI Automation Control Patterns Overview
        ; https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-controlpatternsoverview
        
        
        ; In this case, it is the same string as the string in the "*NameString".
        
;         If *RateElement\GetCurrentPattern(#UIA_ValuePatternId, @*ValuePattern.IUIAutomationValuePattern) = #S_OK
;           If *ValuePattern
;             
;             If *ValuePattern\get_CurrentValue(@*ValueString) = #S_OK
;               If *ValueString
;                 Debug PeekS(*ValueString)
;                 SysFreeString_(*ValueString)
;               EndIf
;             EndIf
;             
;             *ValuePattern\Release()
;           EndIf
;         EndIf
        
        
        ; Attach the event handlers.
        *UIAutomationEventHandler = AllocateMemory(SizeOf(STRUC_IUIAutomationEventHandlerVtbl))
        If *UIAutomationEventHandler
          With *UIAutomationEventHandler
            \pVtbl = *UIAutomationEventHandler + OffsetOf(STRUC_IUIAutomationEventHandlerVtbl\pQueryInterface)
            \pQueryInterface = @IUIAutomationEventHandler_QueryInterface()
            \pAddRef = @IUnknown_AddRef()
            \pRelease = @IUnknown_Release()
            \pHandleAutomationEvent = @IUIAutomationEventHandler_HandleAutomationEvent()
          EndWith
          
          hEventExit = CreateEvent_(0, 0, 0, 0)
          
          If *Automation\AddAutomationEventHandler(#UIA_Window_WindowClosedEventId, *Element, #TreeScope_Subtree, 0, *UIAutomationEventHandler) = #S_OK
            Debug "OK 1"
            
            *UIAutomationPropertyChangedEventHandler = AllocateMemory(SizeOf(STRUC_IUIAutomationPropertyChangedEventHandlerVtbl))
            If *UIAutomationPropertyChangedEventHandler
              
              With *UIAutomationPropertyChangedEventHandler
                \pVtbl = *UIAutomationPropertyChangedEventHandler + OffsetOf(STRUC_IUIAutomationPropertyChangedEventHandlerVtbl\pQueryInterface)
                \pQueryInterface = @IUIAutomationPropertyChangedEventHandler_QueryInterface()
                \pAddRef = @IUnknown_AddRef()
                \pRelease = @IUnknown_Release()
                \pHandlePropertyChangedEvent = @IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent()
              EndWith
              
              If *Automation\AddPropertyChangedEventHandlerNativeArray(*RateElement, #TreeScope_Subtree, 0, *UIAutomationPropertyChangedEventHandler, @PropertyID, 1) = #S_OK
                Debug "OK 2"
                
                If *RateElement\get_CurrentName(@*NameString) = #S_OK
                  If *NameString
                    Name$ = PeekS(*NameString)
                    SysFreeString_(*NameString)
                    
                    Debug Name$
                  EndIf
                EndIf
                
                ; Another way to get the name string.
                
;                 If *RateElement\GetCurrentPropertyValue(#UIA_NamePropertyId, Value2.VARIANT) = #S_OK
;                   If Value2\bstrVal
;                     Name$ = PeekS(Value2\bstrVal)
;                     SysFreeString_(Value2\bstrVal)
;                     
;                     Debug Name$
;                   EndIf
;                 EndIf
                
                
                ; Wait for the status window to close.
                WaitForSingleObject_(hEventExit, #INFINITE)
                
                Debug "Exit"
                
                *Automation\RemovePropertyChangedEventHandler(*RateElement, *UIAutomationPropertyChangedEventHandler)
              EndIf
              
              FreeMemory(*UIAutomationPropertyChangedEventHandler)
            EndIf
            
            *Automation\RemoveAutomationEventHandler(#UIA_Window_WindowClosedEventId, *Element, *UIAutomationEventHandler)
          EndIf
          
          CloseHandle_(hEventExit)
          
          FreeMemory(*UIAutomationEventHandler)
        EndIf
        
        *RateElement\Release()
      EndIf
    EndIf
    
  Until 1
  
  If Value\bstrVal : SysFreeString_(Value\bstrVal) : EndIf
  If *PatternCondition : *PatternCondition\Release() : EndIf
  If *Element : *Element\Release() : EndIf
  If *Automation : *Automation\Release() : EndIf
EndProcedure

Procedure CoInitUninit()
  Static CoInit, UnInit
  
  If UnInit = 0
    UnInit = 1
    
    CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)      ; #COINIT_MULTITHREADED is essential. If not, the event handlers will not be invoked.
    If CoInit = #RPC_E_CHANGED_MODE
      CoUninitialize_()
      CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)
    EndIf
  Else
    If CoInit = #S_OK Or CoInit = #S_FALSE
      CoUninitialize_()
    EndIf
  EndIf
EndProcedure


CoInitUninit()

GetExplorerStatusInfo()

CoInitUninit()

Code: Select all

Prototype ptQueryFullProcessImageName(hProcess, dwFlags.l, *lpExeName, *lpdwSize)
Global QueryFullProcessImageName.ptQueryFullProcessImageName

Global hEventExit

Procedure.s GetFileNameFromWindow(hWnd)
  Protected LibKernel32 = OpenLibrary(#PB_Any, "Kernel32.dll")
  Protected Name.s{1024}, Size.l = 1023
  
  If LibKernel32
    QueryFullProcessImageName = GetFunction(LibKernel32, "QueryFullProcessImageNameW")
    If QueryFullProcessImageName
      If GetWindowThreadProcessId_(hWnd, @PID)
        If PID
          hProcess = OpenProcess_(#PROCESS_QUERY_INFORMATION, 0, PID)
          If hProcess
            QueryFullProcessImageName(hProcess, 0, @Name, @Size)
            CloseHandle_(hProcess)
          EndIf
        EndIf
      EndIf
    EndIf
    CloseLibrary(LibKernel32)
  EndIf
  
  ProcedureReturn Name
EndProcedure

Procedure GetExplorerAllStatusInfos()
  Protected *String
  Protected hWndParent, hWnd
  Protected sFilename.s
  
  ;Search a status window of Explorer. 
  Repeat
    hWndParent = FindWindowEx_(0, hWndParent, "#32770", 0)
    If hWndParent
      sFilename = GetFileNameFromWindow(hWndParent)
      If sFilename
        If FindString(sFilename, "explorer.exe", 1, #PB_String_NoCase)
          hWnd = hWndParent
          Break
        EndIf
      EndIf
    EndIf
  Until hWndParent = 0
  
  If hWnd = 0
    Debug "Explorer status window not found."
    ProcedureReturn 0
  EndIf
  
  Repeat
    If CoCreateInstance_(?CLSID_CUIAutomation, 0, #CLSCTX_INPROC_SERVER, ?IID_IUIAutomation, @*Automation.IUIAutomation) <> #S_OK : Break : EndIf
    If *Automation = 0 : Break : EndIf
    
    If *Automation\ElementFromHandle(hWnd, @*Element.IUIAutomationElement) <> #S_OK : Break : EndIf
    If *Element = 0 : Break : EndIf
    
    If *Automation\CreateTrueCondition(@*PatternCondition.IUIAutomationCondition) <> #S_OK : Break : EndIf
    If *PatternCondition = 0 : Break : EndIf
    
    If *Element\FindAll(#TreeScope_Subtree, *PatternCondition, @*AllElements.IUIAutomationElementArray) <> #S_OK : Break : EndIf
    If *AllElements = 0 : Break : EndIf
    
    If *AllElements\get_Length(@Count) <> #S_OK : Break : EndIf
    
    Debug "number of all elements: " + Count
    
    Count - 1
    
    For i = 0 To Count
      If *AllElements\GetElement(i, @*EachElement.IUIAutomationElement) = #S_OK
        If *EachElement
          
          If *EachElement\get_CurrentClassName(@*String) = #S_OK
            If *String
              ClassName$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Class Name: " + ClassName$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentAutomationId(@*String) = #S_OK
            If *String
              AutomationId$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Automation Id: " + AutomationId$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentName(@*String) = #S_OK
            If *String
              Name$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Name: " + Name$
            EndIf
          EndIf
          
          If *EachElement\get_CurrentControlType(@ControlType) = #S_OK
            Debug "Control Type: " + ControlType
          EndIf
          
          If *EachElement\get_CurrentHelpText(@*String) = #S_OK
            If *String
              Text$ = PeekS(*String)
              SysFreeString_(*String)
              
              Debug "Help Text: " + Text$
            EndIf
          EndIf
          
          Debug "----------------------------------"
          
          *EachElement\Release()
        EndIf
      EndIf
    Next
    
  Until 1
  
  If *AllElements : *AllElements\Release() : EndIf
  If *PatternCondition : *PatternCondition\Release() : EndIf
  If *Element : *Element\Release() : EndIf
  If *Automation : *Automation\Release() : EndIf
EndProcedure

Procedure CoInitUninit()
  Static CoInit, UnInit
  
  If UnInit = 0
    UnInit = 1
    
    CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)      ; #COINIT_MULTITHREADED is essential. If not, the event handlers will not be invoked.
    If CoInit = #RPC_E_CHANGED_MODE
      CoUninitialize_()
      CoInit = CoInitializeEx_(0, #COINIT_MULTITHREADED)
    EndIf
  Else
    If CoInit = #S_OK Or CoInit = #S_FALSE
      CoUninitialize_()
    EndIf
  EndIf
EndProcedure


CoInitUninit()

GetExplorerAllStatusInfos()

CoInitUninit()
User avatar
chi
Addict
Addict
Posts: 1028
Joined: Sat May 05, 2007 5:31 pm
Location: Linz, Austria

Re: [Windows] UI Automation Interface example

Post by chi »

Thanks a lot, breeze4me. Now it's (kinda) working :lol:

Example1:

Code: Select all

copy started...
running your code... 
. OK 1
. OK 2
. 150 MB/second (displayed once, doesn't update. It's suppose to, right? Repeat Until *Automation = 0 : Break)
copy dialog is closing...
. Exit
Example2:

Code: Select all

copy started...
running your code...
. number of all elements: 33
. Class Name: #32770
. Automation Id: 
. Name: Copying 1 item (1,95 GB)
. Control Type: 50032
. Help Text: 
. ----------------------------------
...
code execution finished... although copy is still in progress. 
Et cetera is my worst enemy
breeze4me
Enthusiast
Enthusiast
Posts: 511
Joined: Thu Mar 09, 2006 9:24 am
Location: S. Kor

Re: [Windows] UI Automation Interface example

Post by breeze4me »

Example 1:
On Windows 10 and later, the update interval is about 10 seconds, but I don't know on Windows 7.

Example 2:
Yes, correct operation. The code shows all the information only once.
User avatar
chi
Addict
Addict
Posts: 1028
Joined: Sat May 05, 2007 5:31 pm
Location: Linz, Austria

Re: [Windows] UI Automation Interface example

Post by chi »

Ok, thanks for clearing thing up for me! I think I can handle the rest myself, if I ever feel the need to work with UI automation... Thx for sharing
Et cetera is my worst enemy
Post Reply