How to access UI elements

Mac OSX specific forum
User avatar
Piero
Addict
Addict
Posts: 916
Joined: Sat Apr 29, 2023 6:04 pm
Location: Italy

How to access UI elements

Post by Piero »

Code: Select all

-- Applescript:
display dialog "You will have 5 seconds to point cursor" buttons {"GO!"} default button 1
delay 5 -- 5 seconds to point cursor
use application "System Events"
use scripting additions
set s to __string__({UIElement:it ¬
	, UIProperties:properties ¬
	, UIAttributes:name of attributes ¬
	, UIActions:name of actions} of ¬
	(click at my mouseLocation()))
set [a, b, c] to [length of "{UIElement:", ¬
	offset of "of application \"System Events\", " in s, ¬
	length of "of application \"System Events\", "]
set UIElement to text (1 + a) thru (b - 2) of s
set UIRecord to ["{", text (b + c) thru -1 of s]
set UIString to the contents of [¬
	"use application \"System Events\"", ¬
	linefeed, linefeed, UIElement, ¬
	linefeed, linefeed, UIRecord] as text
display dialog "Copy results (to paste and compile on Script Editor)?"
set the clipboard to the UIString
return the UIString
to __string__(object)
	local object
	try
		set s to object as missing value
	on error E --> "Can’t make %object% into type missing value."
		set tid to the text item delimiters
		set the text item delimiters to "Can’t make "
		set s to text items 2 thru -1 of E as text
		set the text item delimiters to " into type missing value."
		set s to text items 1 thru -2 of s as text
		set the text item delimiters to tid
	end try
	return s
end __string__
on mouseLocation()
	script
		use framework "Foundation"
		use framework "AppKit"
		property this : a reference to current application
		property parent : this
		property NSEvent : a reference to NSEvent of this
		property NSScreen : a reference to NSScreen of this
		on mouseLocation()
			set [list, [number, height]] to ¬
				NSScreen's ¬
				mainScreen()'s ¬
				frame()
			tell NSEvent's mouseLocation() to ¬
				{its x, height - (its y)}
		end mouseLocation
	end script
	result's mouseLocation()
end mouseLocation

…or you can use UI Browser
Hints:   ⌘⇧.   ~/Library/Logs/K6J1GI(IkG.plist   ~/Library/Application Support/UI Browser/

Edit/Note:
…there is also the Accessibility Inspector app (inside Xcode) but it's "not very easy to use"…