CocoaMessage

[spacebuddy]

I am trying to understand how the CocoaMessage works, I am trying to get a snapshot of
the desktop using CGDisplayCreateImage. In Xcode it works, but it is called
like this CGImageRef image = CGDisplayCreateImage(displays[displaysIndex]);

It uses Quartz to get a snapshot of the desktop. Just have no clue how to translate this to PB x64

Can it be translated?

[wilbert]

CocoaMessage allows you to send messages to cocoa objects.
CGDisplayCreateImage is a normal C function that needs to be imported using ImportC.

Something like this

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageSize.NSSize

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageSize\width = CGImageGetWidth(CGImage)
ImageSize\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageSize); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageSize\width, ImageSize\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
DrawImage(NSImage, 0, 0); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

Be aware that this approach requires at least OS X 10.6

[spacebuddy]

Thanks Wilbert, that works perfectly

[Shardik]

Wilbert,

thank you for the nice example. But one small hint: the long path to ApplicationServices behind ImportC is not necessary. A simple ImportC "" is sufficient...

[wilbert]

Thanks Shardik, I changed it

[berbellll]

CocoaMessage allows you to send messages to cocoa objects.
CGDisplayCreateImage is a normal C function that needs to be imported using ImportC.

Something like this

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageSize.NSSize

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageSize\width = CGImageGetWidth(CGImage)
ImageSize\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageSize); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageSize\width, ImageSize\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
DrawImage(NSImage, 0, 0); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

Be aware that this approach requires at least OS X 10.6

Works in pb 5.31
pb 5.40 - 5.60 black screen
can someone help ?

[wilbert]

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageRect.NSRect

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageRect\size\width = CGImageGetWidth(CGImage)
ImageRect\size\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

[berbellll]

is working.
Thank you !!!

[WilliamL]

Works fine here but the image is twice as big when copying a retina screen to an image.

I suppose you could reduce the image size by the scaling factor which would be 2 for the retina display.

Code: Select all

Define backingScaleFactor.CGFloat = 1.0

If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@backingScaleFactor, CocoaMessage(0, 0, "NSScreen mainScreen"), "backingScaleFactor")
EndIf

Debug backingScaleFactor

http://www.purebasic.fr/english/viewtop ... na#p407165

[wilbert]

@William,
I don't remember if I asked this before but what kind of values does the code below return

Code: Select all

CocoaMessage(@Frame.NSRect, CocoaMessage(0, 0, "NSScreen mainScreen"), "frame")
Debug Frame\size\width
Debug Frame\size\height

ExamineDesktops()
Debug DesktopWidth(0)
Debug DesktopHeight(0)

Does it return the size in logical pixels or physical pixels ?

[WilliamL]

@wilbert, here are the values.

In the past, I think I've just taken the x and y width/height and divided it by the ScaleFactor which will be either 1 or 2.

Code: Select all

CocoaMessage(@Frame.NSRect, CocoaMessage(0, 0, "NSScreen mainScreen"), "frame")
Debug Frame\size\width ; = 1440.0
Debug Frame\size\height ; = 900.0

ExamineDesktops()
Debug DesktopWidth(0) ; = 1440
Debug DesktopHeight(0) ; =900

Define backingScaleFactor.CGFloat = 1.0

If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@backingScaleFactor, CocoaMessage(0, 0, "NSScreen mainScreen"), "backingScaleFactor")
EndIf

Debug backingScaleFactor ; = 2

oddly, your first example above no longer works. This was your original version that works.

Code: Select all

EnableExplicit

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Define CGImage, NSImage, ImageRect.NSRect

CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
ImageRect\size\width = CGImageGetWidth(CGImage)
ImageRect\size\height = CGImageGetHeight(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
CGImageRelease(CGImage)

CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
StopDrawing()

CocoaMessage(0, NSImage, "release"); release the NSImage

If OpenWindow(0, 0, 0, 320, 220, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0,  10, 10, 300, 200, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

[wilbert]

If you have multiple screens attached, it probably is best to capture the screen the application is running on.
That could be done like this.

Code: Select all

; PB 5.40+

EnableExplicit

Define CGImage, NSImage, MainScreen, MainScreenID, Rect.NSRect, ZeroSize.NSSize

; Get id for main screen
MainScreen = CocoaMessage(0, 0, "NSScreen mainScreen")
MainScreenID = CocoaMessage(0, CocoaMessage(0, CocoaMessage(0, MainScreen, "deviceDescription"), 
                                            "objectForKey:$", @"NSScreenNumber"), "intValue")

; Create NSImage from main screen
CGImage = CGDisplayCreateImage_(MainScreenID)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ZeroSize)
CGImageRelease_(CGImage)

; Set rectangle (0, 0, width, height) for main screen
CocoaMessage(@Rect, MainScreen, "frame")
Rect\origin\x = 0
Rect\origin\y = 0

; Create a PureBasic image
CreateImage(0, Rect\size\width, Rect\size\height)
StartDrawing(ImageOutput(0))
CocoaMessage(0, NSImage, "drawInRect:@", @Rect)
StopDrawing()

; Release the NSImage
CocoaMessage(0, NSImage, "release")

If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0, 10, 10, 640, 480, ImageID(0))
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

If you only want to show the captured screen, you don't need to convert to a PB image.
In that case you can display a retina screen capture as sharp as the original.

Code: Select all

; PB 5.40+

EnableExplicit

Define CGImage, NSImage, MainScreen, MainScreenID, Scale.CGFloat, Size.NSSize

; Get id for main screen
MainScreen = CocoaMessage(0, 0, "NSScreen mainScreen")
MainScreenID = CocoaMessage(0, CocoaMessage(0, CocoaMessage(0, MainScreen, "deviceDescription"), 
                                            "objectForKey:$", @"NSScreenNumber"), "intValue")

; Create NSImage from main screen
CGImage = CGDisplayCreateImage_(MainScreenID)
Size\width = CGImageGetWidth_(CGImage)
Size\height = CGImageGetHeight_(CGImage)
NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @Size)
CGImageRelease_(CGImage)

; Correct NSImage size for backingScaleFactor
If OSVersion() >= #PB_OS_MacOSX_10_7
  CocoaMessage(@Scale, MainScreen, "backingScaleFactor")
  Size\width / Scale
  Size\height / Scale
  CocoaMessage(0, NSImage, "setSize:@", @Size)
EndIf

; Show NSImage
If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
  ImageGadget(0, 10, 10, 640, 480, NSImage)
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
EndIf

[WilliamL]

Both of your examples work fine but, you are right, the second one is much sharper and is identical to the actual screen. I see that you are drawing with cocoa in the second one. Interesting example!

Thanks wilbert!

[berbellll]

Code: Select all

ImportC ""
  CGDisplayCreateImage(displayID)
  CGImageGetHeight(image)
  CGImageGetWidth(image)
  CGImageRelease(image)
  CGMainDisplayID()
EndImport

Procedure GetScreenX()
  
  Define CGImage, NSImage, ImageRect.NSRect
  
  CGImage = CGDisplayCreateImage(CGMainDisplayID()); get CGImage from main display
  ImageRect\size\width = CGImageGetWidth(CGImage)
  ImageRect\size\height = CGImageGetHeight(CGImage)
  NSImage = CocoaMessage(0, CocoaMessage(0, 0, "NSImage alloc"), "initWithCGImage:", CGImage, "size:@", @ImageRect\size); convert CGImage into NSImage
  CGImageRelease(CGImage)
  
  CreateImage(0, ImageRect\size\width, ImageRect\size\height); Create a PureBasic image
  StartDrawing(ImageOutput(0))
  CocoaMessage(0, NSImage, "drawInRect:@", @ImageRect); draw the NSImage object
  StopDrawing()
  
  CocoaMessage(0, NSImage, "release"); release the NSImage
  FreeImage(0)
  
  
;   ProcedureReturn ImageID(0)
EndProcedure




Delay(5000)
Repeat
  image = GetScreenX()
  x + 1
Until x = 400


; Show NSImage
; If OpenWindow(0, 0, 0, 660, 500, "Image from display", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)
;   ImageGadget(0, 10, 10, 640, 480, image)
;   Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
; EndIf

memory leak ?
[ERROR] NSImage: Insufficient memory To allocate pixel Data buffer of 5242880 bytes

[wilbert]

memory leak ?
[ERROR] NSImage: Insufficient memory To allocate pixel Data buffer of 5242880 bytes

Cocoa works with autorelease pools.
When you use a command within the PureBasic event loop, the objects marked for autorelease, are released regularly.
In this case you create a lot of objects outside of that event loop. You can solve the problem by using your own autorelease pool inside your procedure.
At the top of your GetScreenX procedure, you insert

Code: Select all

Pool = CocoaMessage(0, 0, "NSAutoreleasePool new")

and at the bottom of your procedure

Code: Select all

CocoaMessage(0, Pool, "release")

That should fix the problem.