Page 1 of 1
Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 2:32 am
by BarryG
Hi guys, I'm trying to determine the dimensions (rectangle) of the item under the mouse using AccessibleObjectFromPoint. There's info about it here ->
https://docs.microsoft.com/en-us/window ... cclocation but I don't know how to fit that into the following code template. Can somebody please help if they know? Thank you.
Note: The goal of this is to get the dimensions of Google's "Search" box, which can't normally be obtained with GetWindowRect_() as it's not a Windows item. I've read online that AccessibleObjectFromPoint can probably do it, hence this request for help. It should be possible because I can get the text in the Google "Search" box with AccessibleObjectFromPoint, but not the dimensions of the box.
Code: Select all
CoInitialize_(0)
Prototype.l ProtoAccessibleObjectFromPoint(x.l,y.l,*ia,*var)
Global AccessibleObjectFromPoint.ProtoAccessibleObjectFromPoint
AccessibleObjectFromPoint=GetFunction(OpenLibrary(#PB_Any,"Oleacc.dll"),"AccessibleObjectFromPoint")
Procedure GetItemDimensions()
Protected CursorPos.Point
Protected handle
Protected *pIAcc.IAccessible
Protected vt.VARIANT
Protected s.s
GetCursorPos_(@CursorPos)
If AccessibleObjectFromPoint(CursorPos\x,CursorPos\y,@*pIAcc,@vt)=#S_OK
; Get the item dimensions here, somehow... :(
EndIf
*pIAcc\Release()
ProcedureReturn #Null
EndProcedure
Sleep_(2000) ; Give user time to hover over the "Search" box.
GetItemDimensions()
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 3:11 am
by IdeasVacuum
Hi Barry
Not sure it can do it because the Google Search Box is part of a web page (HTML) and does not have to comply with anything Microsoft.
You might be able to download the HTML and extract the box dimensions, but the height auto-expands to display recent searches.
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 5:05 am
by BarryG
There'd have to be a way to do it, because PhraseExpress can pop up an auto-complete box in any web browser field, such as Google's "Search" field; so somehow they know where the caret is. I'm trying to do it too, but GetCaretPos_() doesn't work with web browsers. I'm asking in relation to this post ->
viewtopic.php?f=13&t=72738
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 11:36 am
by IdeasVacuum
That's different Barry
You probably don't need the box size (or fake box size) or even the caret position, you probably do need to know the id of the box and "inject" your text into the box - it's javascript territory (sorry, I'm not a javascript person).
You might find something useful on the Spider Basic Forum, someone like Peter or Falsam that can help. I can link to something about injection - it's used by Android Apps to fill in Web form boxes:
https://puravidaapps.com/inject.php
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 11:40 am
by BarryG
I don't think it's an injection thing because PhraseExpress doesn't do anything like that and it works.
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 12:09 pm
by IdeasVacuum
How do you know how PhraseExpress does what it does?
It's said that Clive Sinclair, the man who developed the prototype of the Amstrad PC, once tried to explain the computer's technical specifications to Alan Sugar. 'I don't care if they have rubber bands in them,' replied Sugar, the Amstrad boss, 'as long as they work.'
By the way, it has nothing to do with sharp needles. Like a lot of javascript terms, it's just a different, more wacky name for something.
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 12:23 pm
by BarryG
IdeasVacuum wrote:How do you know how PhraseExpress does what it does?
Obviously I don't know for sure, but do you really think it's parsing the loaded web page source and getting the element info?
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 12:26 pm
by IdeasVacuum
... yes I do, if it's done on a smartphone in milliseconds, it can be done on a desktop.
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 1:16 pm
by BarryG
I may have to investigate that route, then. But can it get the web elements from an external browser? I'm not talking about using a PureBasic WebGadget(), because that link you gave (
https://puravidaapps.com/inject.php) is injecting code into
its own loaded webview into its own app, which of course is easy. I'm talking about a PureBasic exe reading Firefox.exe's web elements of any given page; a totally independent and separate process.
Wouldn't it be a whole lot easier just to use AccessibleObjectFromPoint to get the current focussed item, since Windows already knows what that is for everything on-screen (both apps and web-based items)? Like how the Windows Narrator knows.
Re: Get item dimensions with AccessibleObjectFromPoint
Posted: Mon Mar 16, 2020 6:18 pm
by IdeasVacuum
Wouldn't it be a whole lot easier just to use AccessibleObjectFromPoint to get the current focussed item, since Windows already knows what that is for everything on-screen (both apps and web-based items)? Like how the Windows Narrator knows.
As far as I can tell from the MS link, it does not know everything that is on-screen, it only knows the things that have been built to be compatible, which websites built with C# etc probably are but most things Google definitely not.