[Solved] ReceiveHTTPFile fails for me

[Thunder93]

ReceiveHTTPFile() suppose to download a file hosted on a HTTP site. For me, it is usually downloading and stopping BUT with a PB successful result, yet the file is incomplete aka corrupted.

[collectordave]

Trying to download OMDB info.

Replace spaces in title with %20

Code: Select all

  Title = ReplaceString(GetGadgetText(strTitle)," ","%20")
  
  InitNetwork()

  If Val(GetGadgetText(strYear)) > 1900
    *buf = ReceiveHTTPMemory("http://www.omdbapi.com/?t=" + Title + "&y=" + GetGadgetText(strYear))
  Else
    *buf = ReceiveHTTPMemory("http://www.omdbapi.com/?t=" + Title)
  EndIf
  CSVString = PeekS(*buf,-1,#PB_Ascii)

Then decode the comma separated string to get the info. After a first pass you get

Title":"Dust Up
Year":"2012
Rated":"UNRATED
Released":"28 Sep 2012
Runtime":"94 min
Genre":"Action, Adventure, Comedy
Director":"Ward Roberts
Writer":"Ward Roberts
Actors":"Amber Benson, Jeremiah Birkett, Aaron Gaffey, Devin Barry
Plot":"A former vigilante who has embraced his peaceful nature comes to the aid of a young mother in trouble with a cannibalistic drug lord.
Language":"English
Country":"USA
Awards":"N/A
Poster":"http://ia.media-imdb.com/images/M/MV5BM ... _SX300.jpg
Metascore":"N/A
imdbRating":"5.1
imdbVotes":"231
imdbID":"tt1991034
Type":"movie

Just used this to build a movie\DVD database for home use will post when a little more complete.

OMDB do limit the amount of calls you make I think unless you get a commercial licence.

[Fangbeast]

Just used this to build a movie\DVD database for home use will post when a little more complete.

I said in 2013 that I had solved this already.

I'll fix the title but I suggest you post your own solution in "Tips and tricks" for people to find it easier as I never posted the final code due to lack of interest.

[collectordave]

Thanks will post later

[RamRat]

Thanks will post later

Hi, have you posted anything yet?? I sooooo want the code!

[collectordave]

Hi

Posted here http://www.purebasic.fr/english/viewtop ... ilit=movie but removed link. If you need code for the movie database I can renable the link. Needed space for photos at the time.

Kind regards

cd

[RamRat]

Yeah mate , would love the link for the source posted.

[collectordave]

Hi

Link reposted at the above topic.

You need to look for a procedure GetWebContent()

Hope this helps

cd

[RamRat]

Awesome, cheers mate

[RamRat]

Hi Fangbeast,

try this:

Code: Select all

EnableExplicit


Procedure.i HTTPGet2Mem(URL$)
  
  Protected.i ConnectionID, TimeOutCounter, Laenge, Pos, Received, HelpLength, Length, HeaderEndPos, HeaderFinished, Size
  Protected Server$, Header$, String$, Result$, Page$, Parameter$, Host$, *Buffer, *HelpBuffer
  
  
  If LCase(Left(URL$, 7)) <> "http://"
    URL$ = "http://" + URL$
  EndIf
  
  ;Host$ = GetURLPart(URL$, #PB_URL_Site)
  
  Host$ = Mid(URL$, 8, FindString(URL$, "/", 8) - 8)
  
  Page$ = "/" + GetURLPart(URL$, #PB_URL_Path)
  Parameter$ = GetURLPart(URL$, #PB_URL_Parameters)
  If Parameter$
    Page$ + "&" + Parameter$
  EndIf
  
  ;Debug Host$
  
  ConnectionID = OpenNetworkConnection(Host$, 80)
  
  If ConnectionID
    
    ;Build header
        
    Header$ = "GET " + Page$ + " HTTP/1.1" + #CRLF$
    Header$ + "Host: " + Host$ + #CRLF$
    ;Header$ + "Accept: text/html" + #CRLF$
    Header$ + "Content-Type: application/x-www-form-urlencoded" + #CRLF$    
    ;String$ + "Connection: close" + #CRLF$
    Header$ + #CRLF$
    ; header is finished
    
    ;Debug Header$
    

    SendNetworkString(ConnectionID, Header$)
    TimeOutCounter = 1000
    Repeat
      If NetworkClientEvent(ConnectionID) = #PB_NetworkEvent_Data
        Break
      EndIf
      Delay(10)
      TimeOutCounter -1
    Until TimeOutCounter = 0
    
    If TimeOutCounter <> 0
      #Size = 10000
      *HelpBuffer = AllocateMemory(#Size)
      
      If *HelpBuffer
        String$ = ""      
        Repeat
          HelpLength = ReceiveNetworkData(ConnectionID, *HelpBuffer, #Size)
          If HelpLength > 0
            *Buffer = ReAllocateMemory(*Buffer, Length + HelpLength)
            CopyMemory(*HelpBuffer, *Buffer + Length, HelpLength)
            Length + HelpLength
            
            ;Debug Length
            
            If Not HeaderFinished
              HeaderEndPos = FindString(PeekS(*Buffer), #CRLF$ + #CRLF$)
              If HeaderEndPos
                HeaderFinished = #True
                Pos = FindString(PeekS(*Buffer), "Content-Length:")
                If Pos
                  Size = Val(PeekS(*Buffer + Pos + 15)) + HeaderEndPos + 3
                  ;Debug Size
                EndIf
              EndIf
            EndIf
            
            ;String$ = PeekS(*HelpBuffer + HelpLength - 10, 10, #PB_UTF8)
            ;String$ + PeekS(*HelpBuffer, -1, #PB_UTF8)
            ;Debug String$
          EndIf
          If Size > 0
            If Length >= Size
              Received = #True
            EndIf
          Else
            If FindString(String$, "</html") > 0
              Received = #True
            EndIf
          EndIf
        Until Received
        FreeMemory(*HelpBuffer)
      EndIf
    EndIf
      
    CloseNetworkConnection(ConnectionID)
  EndIf
  
  ProcedureReturn *Buffer
  
EndProcedure




Define *Buffer, Size.i, HeaderPos.i

UseJPEGImageDecoder()
InitNetwork()

*Buffer = HTTPGet2Mem("http://ia.media-imdb.com/images/M/MV5BMjIxNjAzODQ0N15BMl5BanBnXkFtZTcwODY2MjMyNA@@._V1_SX300.jpg")
If *Buffer <> #Null
  Size = MemorySize(*Buffer)
  
  HeaderPos = FindString(PeekS(*Buffer, 1024), #CRLF$ + #CRLF$)
  
  OpenWindow(0, 0, 0, 800, 640, "Test")
  CatchImage(0, *Buffer + HeaderPos + 3)
  ImageGadget(0, 0, 0, 0, 0, ImageID(0))  
  FreeMemory(*Buffer)
  
  Repeat : Until WaitWindowEvent() = #PB_Event_CloseWindow
  
Else
  MessageRequester("Error", "Not downloaded")
EndIf

But it is only for picture addresses in the web
Since only than the picture data starts immediately after the http header.

Bernd

Hi, um I tried this and couldn't get it to work any thoughts on what I'm doing wrong?

Cheers!

[elwood]

Hi, um I tried this and couldn't get it to work any thoughts on what I'm doing wrong?

Tell more about your problem or nobody will be able to help. Any error? What is it doing?

[RamRat]

Hi, um I tried this and couldn't get it to work any thoughts on what I'm doing wrong?

Tell more about your problem or nobody will be able to help. Any error? What is it doing?

Ah, I was a bit vague about what I wanted.... Ok I ran the above code and up pops a box "Error Not Downloaded" - It wouldn't download the image and show it on screen but when using webgadget it works fine but I really don't want to use webgadget.

So anyone know a way to download the example image and show it in a window?

"https://images-na.ssl-images-amazon.com ... _SX300.jpg"

Its a messy kludge but can I have webgadget hidden and download to it then copy the image from it or some other hack?

Cheers!

[collectordave]

Code to download file posted above try this to get the image posted above

Code: Select all

UseJPEGImageDecoder()


Global Window_0

Global String_0, Image_0, Button_0



  Window_0 = OpenWindow(#PB_Any, 0, 0, 890, 390, "", #PB_Window_SystemMenu)
  String_0 = StringGadget(#PB_Any, 10, 20, 870, 20, "")
  Image_0 = ImageGadget(#PB_Any, 340, 80, 270, 260, 0)
  Button_0  = ButtonGadget(#PB_Any, 710, 220, 120, 60, "")
  
  InitNetwork()
  

  Repeat
    Event = WaitWindowEvent()
  Select event
    Case #PB_Event_CloseWindow
   End



    Case #PB_Event_Gadget
      Select EventGadget()
        Case Button_0
          ReceiveHTTPFile("https://images-na.ssl-images-amazon.com/images/M/MV5BMTUxMjc2MTcxNV5BMl5BanBnXkFtZTcwMzgzOTY0MQ@@._V1_SX300.jpg","C:\Temp\MyImage.jpg")
          LoadImage(0, "C:\Temp\MyImage.jpg")
          SetGadgetState(Image_0,ImageID(0))
          
      EndSelect
  EndSelect
ForEver

regards CD