Auto Resize WebGadget [Windows]

[IdeasVacuum]

I think at the moment, the solution is a choice between Rashad's and Breezeforme's code, depending on what you need.

Breezeforme gets the web page size accurately, which is important if there is a way to capture the page as an image.

Rashad invokes an output method that captures the whole page, but the ideal page-as-a-single-file method (.mht) is not perfect because Microsoft have not kept their format up to date. With Rashad's method there is no need to re-size the WebGadget.

@Kwaï chang caïne: The code that fails to capture the whole page is mine, Breezforme didn't get it wrong.

[breeze4me]

"How to do a full website snapshot?"
viewtopic.php?f=13&t=35366


The code needs to be modified for some site.

Code: Select all

WebGadget(#frmSnapShot_WB, 0, 0, 0, 0, URL$)

to

Code: Select all

WebGadget(#frmSnapShot_WB, 0, 0, 1024, 0, URL$)

[ts-soft]

No of all this sources works an all websites!
On my Homepage (www.realsource.de) comes mostly a very narrow browser.

I think it is impossible to resize all websites.

[IdeasVacuum]

Well, Kiffi's code is a winner, it gets the size of the page accurately and outputs as a single image!

Edit: ...and with the Window/WebGadget hidden!

[RASHAD]

I have to admit it
ts-soft is right
Even Kiffi code failed with ts-soft WebPage (Width = 0)
I should search the forum before get in deep

Thanks breeze4me for Kiffi link

[IdeasVacuum]

Well, I changed Kiffi's code a little bit, striped-out all the image scaling (which might be the cause of the issue you guys see), changed all the longs to ints, changed the output to png, changed GetWindowLong to GetWindowLongPtr (which might also be important).

...it captures "www.realsource.de" perfectly on my WinXP 32bit PC.

EDITED 16/03/2012:

Code: Select all

UsePNGImageEncoder()

Enumeration
  #WinSnapShot
  #WebSnapShot
EndEnumeration

  #DVASPECT_CONTENT  = 1
  #DVASPECT_THUMBNAIL = 2
  #DVASPECT_ICON = 4
  #DVASPECT_DOCPRINT = 8


Procedure CaptureWebPage(URL$,sFilename.s)
;-----------------------------------------------------

  Define.IWebBrowser2   m_pWebBrowser

  Define.IHTMLDocument2 pDocument
  Define.IHTMLDocument3 pDocument3
  Define.IHTMLElement   pElement
  Define.IHTMLElement2  pElement2
  Define.iDispatch      pDispatch
  Define.IViewObject2   pViewObject

  Define.i bodyHeight
  Define.i bodyWidth
  Define.i rootHeight
  Define.i rootWidth

  Define.RECT rcBounds

  Define.i bolFlag
  Define.i IsBusy
  Define.i hr

  bolFlag = #False

  If OpenWindow(#WinSnapShot, 0, 0, 0, 0, "", #PB_Window_Invisible | #PB_Window_BorderLess)

      WebGadget(#WebSnapShot, 0, 0, 0, 0, URL$)

      Repeat
        If GetGadgetAttribute(#WebSnapShot, #PB_Web_Busy) = 0
          Break
        EndIf

        While WindowEvent(): Delay(1) : Wend
      ForEver

      m_pWebBrowser = GetWindowLongPtr_(GadgetID(#WebSnapShot), #GWL_USERDATA)

         hr = m_pWebBrowser\get_document(@pDispatch)
      If hr = #S_OK

        If pDispatch

             hr = pDispatch\QueryInterface(?IID_IHTMLDocument2, @pDocument)
          If hr = #S_OK

            If pDocument

                 hr = pDocument\get_body(@pElement)
              If hr = #S_OK
                If pElement

                     hr = pElement\QueryInterface(?IID_IHTMLElement2, @pElement2)
                  If hr = #S_OK
                    If pElement2

                         hr = pElement2\get_scrollHeight(@bodyHeight)
                      If hr = #S_OK

                           hr = pElement2\get_scrollWidth(@bodyWidth)
                        If hr = #S_OK

                             hr = pDispatch\QueryInterface(?IID_IHTMLDocument3, @pDocument3)
                          If hr = #S_OK

                            If pDocument3

                                 hr = pDocument3\get_documentElement(@pElement)
                              If hr <> #S_OK : ProcedureReturn #False : EndIf

                                 hr = pElement\QueryInterface(?IID_IHTMLElement2, @pElement2)
                              If hr <> #S_OK : ProcedureReturn #False : EndIf

                                 hr = pElement2\get_scrollHeight(@rootHeight)
                              If hr <> #S_OK : ProcedureReturn #False : EndIf


                                 hr = pElement2\get_scrollWidth(@rootWidth)
                              If hr <> #S_OK : ProcedureReturn #False : EndIf

                              Define.i width
                              Define.i height
                                                         width = bodyWidth
                              If rootWidth > bodyWidth : width = rootWidth : EndIf

                                                           height = bodyHeight
                              If rootHeight > bodyHeight : height = rootHeight : EndIf

                              width + 22

                              ResizeGadget(#WebSnapShot, 0, 0, width, height)

                              hr = m_pWebBrowser\QueryInterface(?IID_IViewObject2, @pViewObject)

                              If hr = #S_OK

                                If pViewObject

                                  Define.i hdcMain

                                     hdcMain = GetDC_(0)
                                  If hdcMain

                                    Define.i hdcMem

                                       hdcMem  = CreateCompatibleDC_(hdcMain)
                                    If hdcMem

                                      Define.i hBitmap

                                         hBitmap = CreateCompatibleBitmap_(hdcMain, width, height)
                                      If hBitmap

                                        Define.i oldImage

                                        oldImage = SelectObject_(hdcMem, hBitmap)

                                        rcBounds\top = 0
                                        rcBounds\left = 0
                                        rcBounds\right = width
                                        rcBounds\bottom = height

                                        pViewObject\Draw(#DVASPECT_CONTENT, -1, 0, 0, hdcMain, hdcMem, rcBounds, 0, 0, 0)

                                        Define.i Image

                                           Image = CreateImage(#PB_Any, width, height)
                                        If Image

                                          Define.i img_hDC

                                               img_hDC = StartDrawing(ImageOutput(Image))
                                            If img_hDC

                                              BitBlt_(img_hDC, 0, 0, width, height, hdcMem, 0, 0, #SRCCOPY)
                                            StopDrawing()

                                               iGoodSave.i = SaveImage(Image,sFilename,#PB_ImagePlugin_PNG,10,24)
                                            If(iGoodSave <> 0) : bolFlag = #True : EndIf

                                          EndIf ; img_hDC

                                          FreeImage(Image)

                                        EndIf ; Image

                                        SelectObject_(hdcMem, oldImage)

                                      EndIf ; hBitmap

                                      DeleteDC_(hdcMem)

                                    EndIf ; hdcMem

                                    ReleaseDC_(0, hdcMain)

                                  EndIf ; hdcMain

                                  pViewObject\Release()

                                EndIf ; pViewObject
                              EndIf; HR

                              pDocument3\Release()

                            EndIf ; pDocument3

                          EndIf ; HR

                        EndIf ; HR

                      EndIf ; HR

                      pElement2\Release()

                    EndIf ; pElement2

                  EndIf ; HR

                  pElement\Release()

                EndIf ; pElement

              EndIf ; HR

              pDocument\Release()

            EndIf ; pDocument

          EndIf ; HR

          pDispatch\Release()

        EndIf ; pDispatch

      EndIf ; HR

      CloseWindow(#WinSnapShot)

  EndIf

  ProcedureReturn bolFlag

EndProcedure

Procedure Snapshot()
;-------------------

  Define sURL.s
  Define sFileOut.s
  Define sNow.s

      sURL = "www.qinbiying.co.uk"
  sFileOut = ReplaceString(sURL, ".", "_")

      sNow = FormatDate("_%yyyy%mm%dd_%hh%ii%ss", Date())

                                        sFileOut = "C:\" + sFileOut + sNow + ".png"
  If CaptureWebPage(sURL, sFileOut)

  Else
          MessageRequester("Web Page", "Capture Failed")
  EndIf

EndProcedure

Snapshot()

End


DataSection

  IID_IHTMLDocument2:
  ;332C4425-26CB-11D0-B483-00C04FD90119
  Data.i $332C4425
  Data.w $26CB, $11D0
  Data.b $B4, $83, $00, $C0, $4F, $D9, $01, $19

  IID_IHTMLDocument3:
  ;3050F485-98B5-11CF-BB82-00AA00BDCE0B
  Data.i $3050F485
  Data.w $98B5, $11CF
  Data.b $BB, $82, $00, $AA, $00, $BD, $CE, $0B

  IID_IHTMLElement2:
  ;3050f434-98b5-11cf-bb82-00aa00bdce0b
  Data.i $3050F434
  Data.w $98B5, $11CF
  Data.b $BB, $82, $00, $AA, $00, $BD, $CE, $0B

  IID_IViewObject2:
  ;00000127-0000-0000-c000-000000000046
  Data.i $00000127
  Data.w $0000, $0000
  Data.b $C0, $00, $00, $00, $00, $00, $00, $46


EndDataSection

[ts-soft]

Works fine. but you should save on temp- or homedir, you can't save at C:\ on most newer systems.

[IdeasVacuum]

Works fine. but you should save on temp- or homedir, you can't save at C:\ on most newer systems.

....yes of course - it is a working snippet for the forum, no more, no less.

[IdeasVacuum]

A couple of gotchas. The CaptureWebPage Procedure can return a false positive. Change to:
Edit:

Code: Select all

   iGoodSave.i = SaveImage(Image,sFileName,#PB_ImagePlugin_PNG)
         If(iGoodSave <> 0) : bolFlag = #True : EndIf

You may also want to control the bit depth of the image, given that there may be a lot of text.

It's possible for the page length to exceed the maximum number of pixels for an image: [ERROR] Image 'Height' is > 32000 pixels.

You will need to test for this and decide whether to scale the image down or chop it's height.

Edit: Another tip: I'm finding that Save png sometimes fails but save bmp succeeds for the same page.

[IdeasVacuum]

I have been using this method to capture an image of a webpage for a while now. If my app is the being used to browse the web, it works most times.

It works a lot of the time when a seperate browser is used, but there is a major flaw because the page has to be re-opened in a WebGadget to be captured. It means that a secured webpage can't be captured, nor can any page that is "a page within a page" - that's where a set of pages will have an identical URL, so this method captures page 1 when you are looking at page 9 in your browser.

So, I'm looking for a different solution. The browser (be it IE, FireFox, Chrome or whatever) is, at the end of the day, a window. What I want to do is capture the entire client area currently on display, auto scrolling the browser window when the page is longer/narrower than the height/width of the window. I have not figured out how to do this yet........

[tj1010]

I think with embedded webkit libs you can actually do this 100% accurate through a single call...

[IdeasVacuum]

...yeah, I see Google are using it for Chrome, which presumably means Mozilla might do the same for FireFox. According to the Wiki Apple, Adobe and Goggle are working on the lib. It is the Safari Engine - I haven't used Safari but I hear it's fairly good. The lib has a good license too, so it would seem to be the ideal replacement for the current PB WebGadget().

Strangely, does not support Visual Studio versions later than 2005.

Would be interesting to hear the thoughts of other devs.

[tj1010]

Someone was doing a webkit lib for PB here that was documented and stable, but I think it turned into another PB abandonware. It had access to the call.

I know some vendors can do all platforms but hide their methods by putting server request queues in their over-priced software. I'm able to do mobile snapshots using web gadgets and zoom calc routines under xcode and ADK emulators. Gecko, Webkit, and Trident all have the single calls that generate bitmaps from DOM and cached bitmaps.

IMO: I'm actually surprised more people here haven't dealt with webkit. If you scrape or automate, most websites use javascript and/or flash to alter DOM, making curl, urllib, and raw http practically useless...

[IdeasVacuum]

This post suggests that webkit is already used by PB for Linux

http://www.purebasic.fr/english/viewtop ... 15&t=50942

Seems strange it is not used across all the platforms supported?