CocoaMessage

Mac OSX specific forum
spacebuddy
Enthusiast
Enthusiast
Posts: 346
Joined: Thu Jul 02, 2009 5:42 am

CocoaMessage

Post by spacebuddy »

I am trying to understand how the CocoaMessage works, I am trying to get a snapshot of
the desktop using CGDisplayCreateImage. In Xcode it works, but it is called
like this CGImageRef image = CGDisplayCreateImage(displays[displaysIndex]);

It uses Quartz to get a snapshot of the desktop. Just have no clue how to translate this to PB x64

Can it be translated? :D
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

CocoaMessage allows you to send messages to cocoa objects.
CGDisplayCreateImage is a normal C function that needs to be imported using ImportC.

Something like this

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageSize.NSSize

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageSize\width = CGImageGetWidth(CGImage)
ImageSize\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageSize); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageSize\width, ImageSize\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
DrawImage(NSImage, 0, 0); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
Be aware that this approach requires at least OS X 10.6
Last edited by wilbert on Fri Feb 01, 2013 8:04 pm, edited 1 time in total.
spacebuddy
Enthusiast
Enthusiast
Posts: 346
Joined: Thu Jul 02, 2009 5:42 am

Re: CocoaMessage

Post by spacebuddy »

Thanks Wilbert, that works perfectly :D
User avatar
Shardik
Addict
Addict
Posts: 1988
Joined: Thu Apr 21, 2005 2:38 pm
Location: Germany

Re: CocoaMessage

Post by Shardik »

Wilbert,

thank you for the nice example. But one small hint: the long path to ApplicationServices behind ImportC is not necessary. A simple ImportC "" is sufficient... :wink:
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

Thanks Shardik, I changed it :)
berbellll
New User
New User
Posts: 5
Joined: Wed May 10, 2017 1:54 am

Re: CocoaMessage

Post by berbellll »

wilbert wrote:CocoaMessage allows you to send messages to cocoa objects.
CGDisplayCreateImage is a normal C function that needs to be imported using ImportC.

Something like this

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageSize.NSSize

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageSize\width = CGImageGetWidth(CGImage)
ImageSize\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageSize); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageSize\width, ImageSize\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
DrawImage(NSImage, 0, 0); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
Be aware that this approach requires at least OS X 10.6



Works in pb 5.31
pb 5.40 - 5.60 black screen
can someone help ?
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageRect.NSRect

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageRect\size\width = CGImageGetWidth(CGImage)
ImageRect\size\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
Last edited by wilbert on Wed May 10, 2017 12:27 pm, edited 1 time in total.
Windows (x64)
Raspberry Pi OS (Arm64)
berbellll
New User
New User
Posts: 5
Joined: Wed May 10, 2017 1:54 am

Re: CocoaMessage

Post by berbellll »

is working.
Thank you !!!
WilliamL
Addict
Addict
Posts: 1214
Joined: Mon Aug 04, 2008 10:56 pm
Location: Seattle, USA

Re: CocoaMessage

Post by WilliamL »

Works fine here but the image is twice as big when copying a retina screen to an image.

I suppose you could reduce the image size by the scaling factor which would be 2 for the retina display.

Code: Select all

Define backingScaleFactor.CGFloat = 1.0

If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@backingScaleFactor, CocoaMessage(0, 0, "NSScreen mainScreen"), "backingScaleFactor")
EndIf

Debug backingScaleFactor
http://www.purebasic.fr/english/viewtop ... na#p407165
MacBook Pro-M1 (2021), Sonoma 14.3.1 (CLT 15.3), PB 6.10b7 M1
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

@William,
I don't remember if I asked this before but what kind of values does the code below return

Code: Select all

CocoaMessage(@Frame.NSRect, CocoaMessage(0, 0, "NSScreen mainScreen"), "frame")
Debug Frame\size\width
Debug Frame\size\height

ExamineDesktops()
Debug DesktopWidth(0)
Debug DesktopHeight(0)
Does it return the size in logical pixels or physical pixels ?
Windows (x64)
Raspberry Pi OS (Arm64)
WilliamL
Addict
Addict
Posts: 1214
Joined: Mon Aug 04, 2008 10:56 pm
Location: Seattle, USA

Re: CocoaMessage

Post by WilliamL »

@wilbert, here are the values.

In the past, I think I've just taken the x and y width/height and divided it by the ScaleFactor which will be either 1 or 2.

Code: Select all

CocoaMessage(@Frame.NSRect, CocoaMessage(0, 0, "NSScreen mainScreen"), "frame")
Debug Frame\size\width ; = 1440.0
Debug Frame\size\height ; = 900.0

ExamineDesktops()
Debug DesktopWidth(0) ; = 1440
Debug DesktopHeight(0) ; =900

Define backingScaleFactor.CGFloat = 1.0

If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@backingScaleFactor, CocoaMessage(0, 0, "NSScreen mainScreen"), "backingScaleFactor")
EndIf

Debug backingScaleFactor ; = 2
oddly, your first example above no longer works. This was your original version that works.

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageRect.NSRect

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageRect\size\width = CGImageGetWidth(CGImage)
ImageRect\size\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
MacBook Pro-M1 (2021), Sonoma 14.3.1 (CLT 15.3), PB 6.10b7 M1
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

If you have multiple screens attached, it probably is best to capture the screen the application is running on.
That could be done like this.

Code: Select all

; PB 5.40+

EnableExplicit

Define CGImage, NSImage, MainScreen, MainScreenID, Rect.NSRect, ZeroSize.NSSize

; Get id for main screen
MainScreen = CocoaMessage(0, 0, "NSScreen mainScreen")
MainScreenID = CocoaMessage(0, CocoaMessage(0, CocoaMessage(0, MainScreen, "deviceDescription"), 
                                            "objectForKey:$", @"NSScreenNumber"), "intValue")

; Create NSImage from main screen
CGImage = CGDisplayCreateImage_(MainScreenID)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ZeroSize)
CGImageRelease_(CGImage)

; Set rectangle (0, 0, width, height) for main screen
CocoaMessage(@Rect, MainScreen, "frame")
Rect\origin\x = 0
Rect\origin\y = 0

; Create a PureBasic image
CreateImage(0, Rect\size\width, Rect\size\height)
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @Rect)
StopDrawing()

; Release the NSImage
CocoaMessage(0, NSImage, "release")

If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0, 10, 10, 640, 480, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
If you only want to show the captured screen, you don't need to convert to a PB image.
In that case you can display a retina screen capture as sharp as the original.

Code: Select all

; PB 5.40+

EnableExplicit

Define CGImage, NSImage, MainScreen, MainScreenID, Scale.CGFloat, Size.NSSize

; Get id for main screen
MainScreen = CocoaMessage(0, 0, "NSScreen mainScreen")
MainScreenID = CocoaMessage(0, CocoaMessage(0, CocoaMessage(0, MainScreen, "deviceDescription"), 
                                            "objectForKey:$", @"NSScreenNumber"), "intValue")

; Create NSImage from main screen
CGImage = CGDisplayCreateImage_(MainScreenID)
Size\width = CGImageGetWidth_(CGImage)
Size\height = CGImageGetHeight_(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @Size)
CGImageRelease_(CGImage)

; Correct NSImage size for backingScaleFactor
If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@Scale, MainScreen, "backingScaleFactor")
  Size\width / Scale
  Size\height / Scale
  CocoaMessage(0, NSImage, "setSize:@", @Size)
EndIf

; Show NSImage
If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0, 10, 10, 640, 480, NSImage)
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf
Windows (x64)
Raspberry Pi OS (Arm64)
WilliamL
Addict
Addict
Posts: 1214
Joined: Mon Aug 04, 2008 10:56 pm
Location: Seattle, USA

Re: CocoaMessage

Post by WilliamL »

Both of your examples work fine but, you are right, the second one is much sharper and is identical to the actual screen. I see that you are drawing with cocoa in the second one. Interesting example!

Thanks wilbert!
MacBook Pro-M1 (2021), Sonoma 14.3.1 (CLT 15.3), PB 6.10b7 M1
berbellll
New User
New User
Posts: 5
Joined: Wed May 10, 2017 1:54 am

Re: CocoaMessage

Post by berbellll »

Code: Select all

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Procedure GetScreenX()
  
  Define CGImage, NSImage, ImageRect.NSRect
  
  CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
  ImageRect\size\width = CGImageGetWidth(CGImage)
  ImageRect\size\height = CGImageGetHeight(CGImage)
  NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
  CGImageRelease(CGImage)
  
  CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
  StartDrawing(ImageOutput(0))
  CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
  StopDrawing()
  
  CocoaMessage(0, NSImage, "release"); release the NSImage
  FreeImage(0)
  
  
;   ProcedureReturn ImageID(0)
EndProcedure




Delay(5000)
Repeat
  image = GetScreenX()
  x + 1
Until x = 400


; Show NSImage
; If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
;   ImageGadget(0, 10, 10, 640, 480, image)
;   Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
; EndIf

memory leak ?
[ERROR] NSImage: Insufficient memory To allocate pixel Data buffer of 5242880 bytes
wilbert
PureBasic Expert
PureBasic Expert
Posts: 3870
Joined: Sun Aug 08, 2004 5:21 am
Location: Netherlands

Re: CocoaMessage

Post by wilbert »

berbellll wrote:memory leak ?
[ERROR] NSImage: Insufficient memory To allocate pixel Data buffer of 5242880 bytes
Cocoa works with autorelease pools.
When you use a command within the PureBasic event loop, the objects marked for autorelease, are released regularly.
In this case you create a lot of objects outside of that event loop. You can solve the problem by using your own autorelease pool inside your procedure.
At the top of your GetScreenX procedure, you insert

Code: Select all

Pool = CocoaMessage(0, 0, "NSAutoreleasePool new")
and at the bottom of your procedure

Code: Select all

CocoaMessage(0, Pool, "release")
That should fix the problem.
Windows (x64)
Raspberry Pi OS (Arm64)
Post Reply