ReceiveHTTPMemory

Just starting out? Need help? Post your questions and find answers here.
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

ReceiveHTTPMemory

Post by williamvanhoecke »

Hello,
The following lines is returning a *Buffer while it should return 0 (the URL exists but the map BUTTONS does not)
The buffer in memory contains

<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<p>Additionally, a 404 Not Found
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>


Am i doing somthing wrong here ?

Code: Select all

Define *Buffer = ReceiveHTTPMemory("Https://www.listrik.be/BUTTONS/my-test-image.png", #PB_HTTP_NoRedirect)
  If *Buffer 
    ProcedureReturn *Buffer
  Else  
    ProcedureReturn EncodeImage(LoadImage(#PB_Any,exe_path + "BUTTONS/schakelaar.png"), #PB_ImagePlugin_PNG)
  EndIf  
User avatar
Bisonte
Addict
Addict
Posts: 1305
Joined: Tue Oct 09, 2007 2:15 am

Re: ReceiveHTTPMemory

Post by Bisonte »

first : You see that there is no file with that name on this url. so it cannot be downloaded.
second: why you are using EncodeImage ? (if the download was correct) You have a png file, so CatchImage() to get the image and SaveImage() to save it...
so UsePNGImageEncoder and Decoder is needed...

Edit :

Code: Select all

UsePNGImageDecoder()
UseJPEGImageDecoder()


*Buffer = ReceiveHTTPMemory(URLEncoder("https://www.listrik.be/eendraadschema index_htm_files/81.jpg"), #PB_HTTP_NoRedirect)

If *Buffer
  Image = CatchImage(#PB_Any, *Buffer)
  If IsImage(Image)
    Debug "Image Ok"  
  EndIf
Else  
  Image = LoadImage(#PB_Any, exe_path.s + "BUTTONS/schakelaar.png")
EndIf  
PureBasic 6.21 (Windows x64) | Windows 11 Pro | AsRock B850 Steel Legend Wifi | R7 9800x3D | 64GB RAM | RTX 5080 | ThermaltakeView 270 TG ARGB | build by vannicom​​
English is not my native language... (I often use DeepL.)
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

Re: ReceiveHTTPMemory

Post by williamvanhoecke »

Hi, thanks for your reply

the calling routine is

Code: Select all

DrawImage(ImageID(CatchImage(#PB_Any,get_image())),0,0)
If the image is found then a it is in memory
If not found on the server the image is loaded locally from disk and the put in memory.
in both cases I return a pointer to the catchimage routine.

But that is not the problem, the If *Buffer always fires even if the file is not found.... normally the ]Buffer should be 0
User avatar
Bisonte
Addict
Addict
Posts: 1305
Joined: Tue Oct 09, 2007 2:15 am

Re: ReceiveHTTPMemory

Post by Bisonte »

williamvanhoecke wrote: Tue Mar 04, 2025 11:29 pm If *Buffer always fires even if the file is not found.... normally the ]Buffer should be 0
It is "guilty" because there is something in this *Buffer... Image or not. Now it is on the developer to do something with it.
Thats why I try to use "Catchimage". This is the test, if it is a 404 HTTP Error (or something else) or an Image.
PureBasic 6.21 (Windows x64) | Windows 11 Pro | AsRock B850 Steel Legend Wifi | R7 9800x3D | 64GB RAM | RTX 5080 | ThermaltakeView 270 TG ARGB | build by vannicom​​
English is not my native language... (I often use DeepL.)
Thorium
Addict
Addict
Posts: 1305
Joined: Sat Aug 15, 2009 6:59 pm

Re: ReceiveHTTPMemory

Post by Thorium »

williamvanhoecke wrote: Tue Mar 04, 2025 11:29 pm But that is not the problem, the If *Buffer always fires even if the file is not found.... normally the ]Buffer should be 0
It is correct that it is not 0. The server is sending back a HTML file to display the error message on a browser. That is typical behavior of web servers as they expect a browser to fetch the file and need a way to show the user what wend wrong.

You need to parse the HTML to determine if the file exists.
First check the size of the returned buffer. Then you can check the title tag. If the first characters are "<title>" you can assume it's not a png file but a html response from the server.
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

Re: ReceiveHTTPMemory

Post by williamvanhoecke »

Thanks thorium,
I already worked around like below.

Code: Select all

Procedure.i GetImagePointer(File.s)
  Define *Buffer = ReceiveHTTPMemory("https://www.listrik.be/ListrikTheme/" + File, #PB_HTTP_NoRedirect)
  If FindString(PeekS(*Buffer, MemorySize(*Buffer), #PB_UTF8), "404 Not Found") > 0
    Debug LoadImage(#PB_Any,exe_path + "BUTTONS/" + GetFilePart(File))
    ProcedureReturn EncodeImage(LoadImage(#PB_Any,exe_path + "BUTTONS/" + GetFilePart(File)), #PB_ImagePlugin_PNG)
  Else
    ProcedureReturn *Buffer
  EndIf
EndProcedure
Still, I think this is a bug, I would expect the 'ReceiveHTTPMemory()' function to do the check and return a valid pointer or 0
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

Re: ReceiveHTTPMemory

Post by williamvanhoecke »

Bisonte wrote: Wed Mar 05, 2025 7:45 am
williamvanhoecke wrote: Tue Mar 04, 2025 11:29 pm If *Buffer always fires even if the file is not found.... normally the ]Buffer should be 0
It is "guilty" because there is something in this *Buffer... Image or not. Now it is on the developer to do something with it.
Thats why I try to use "Catchimage". This is the test, if it is a 404 HTTP Error (or something else) or an Image.
Yes, I know now, I found out by testing and checking, but documentation says otherwise, is says to check if the returned memorypointer = 0 or not -> "Returns the new memory buffer address if the download was successful, zero otherwise."
User avatar
kenmo
Addict
Addict
Posts: 2032
Joined: Tue Dec 23, 2003 3:54 am

Re: ReceiveHTTPMemory

Post by kenmo »

ReceiveHTTPMemory() is giving you the exact data it received from the server... I think it's OK as-is.

You can go one step further and check the HTTP StatusCode, here is a helper function:

Code: Select all

Procedure.i ReceiveHTTPMemoryStrict(URL.s, Flags.i = 0, UserAgent.s = "")
  Protected *Buffer = #Null
  If (UserAgent = "")
    UserAgent = "Mozilla/5.0 Gecko/41.0 Firefox/41.0" ; default per ReceiveHTTPMemory() help
  EndIf
  NewMap Headers.s()
  Headers("User-Agent") = UserAgent
  Protected Req.i = HTTPRequest(#PB_HTTP_Get, URL, "", Flags, Headers())
  If (Req)
    Select (Val(HTTPInfo(Req, #PB_HTTP_StatusCode)))
      Case 200 To 299 ; Success
        *Buffer = HTTPMemory(Req)
    EndSelect
    FinishHTTP(Req)
  EndIf
  ProcedureReturn (*Buffer)
EndProcedure

URL.s = "Https://www.listrik.be/BUTTONS/my-test-image.png"
;URL.s = "https://www.listrik.be/eendraadschema index_htm_files/81.jpg"
;URL.s = "https://www.purebasic.com/img/pb-logo-white.png"

*Buffer = ReceiveHTTPMemoryStrict(URL, #PB_HTTP_NoRedirect)
If *Buffer
  ShowMemoryViewer(*Buffer, MemorySize(*Buffer))
  CallDebugger
  FreeMemory(*Buffer)
Else
  Debug "Failed"
EndIf
User avatar
NicTheQuick
Addict
Addict
Posts: 1502
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: ReceiveHTTPMemory

Post by NicTheQuick »

williamvanhoecke wrote: Tue Mar 04, 2025 11:29 pm the calling routine is

Code: Select all

DrawImage(ImageID(CatchImage(#PB_Any,get_image())),0,0)
Are you aware that this is a memory leak?
`CatchImage(#PB_Any, ...)` returns a freshly created object that is never stored in a variable. So you will never be able to use `FreeImage()` with it. So if that line is executed more than once your memory consumption will increase which each iteration.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
User avatar
Kiffi
Addict
Addict
Posts: 1484
Joined: Tue Mar 02, 2004 1:20 pm
Location: Amphibios 9

Re: ReceiveHTTPMemory

Post by Kiffi »

kenmo wrote: Fri Mar 07, 2025 1:18 amReceiveHTTPMemory() is giving you the exact data it received from the server... I think it's OK as-is.
+1
Hygge
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

Re: ReceiveHTTPMemory

Post by williamvanhoecke »

NicTheQuick wrote: Fri Mar 07, 2025 11:15 am
williamvanhoecke wrote: Tue Mar 04, 2025 11:29 pm the calling routine is

Code: Select all

DrawImage(ImageID(CatchImage(#PB_Any,get_image())),0,0)
Are you aware that this is a memory leak?
`CatchImage(#PB_Any, ...)` returns a freshly created object that is never stored in a variable. So you will never be able to use `FreeImage()` with it. So if that line is executed more than once your memory consumption will increase which each iteration.
Thanks for your concern, and yes I am aware of the memory consumption, but it is about images that are there as long as the program lives.
williamvanhoecke
User
User
Posts: 65
Joined: Wed Jun 07, 2017 10:13 pm

Re: ReceiveHTTPMemory

Post by williamvanhoecke »

kenmo wrote: Fri Mar 07, 2025 1:18 am ReceiveHTTPMemory() is giving you the exact data it received from the server... I think it's OK as-is.

You can go one step further and check the HTTP StatusCode, here is a helper function:

Code: Select all

Procedure.i ReceiveHTTPMemoryStrict(URL.s, Flags.i = 0, UserAgent.s = "")
  Protected *Buffer = #Null
  If (UserAgent = "")
    UserAgent = "Mozilla/5.0 Gecko/41.0 Firefox/41.0" ; default per ReceiveHTTPMemory() help
  EndIf
  NewMap Headers.s()
  Headers("User-Agent") = UserAgent
  Protected Req.i = HTTPRequest(#PB_HTTP_Get, URL, "", Flags, Headers())
  If (Req)
    Select (Val(HTTPInfo(Req, #PB_HTTP_StatusCode)))
      Case 200 To 299 ; Success
        *Buffer = HTTPMemory(Req)
    EndSelect
    FinishHTTP(Req)
  EndIf
  ProcedureReturn (*Buffer)
EndProcedure

URL.s = "Https://www.listrik.be/BUTTONS/my-test-image.png"
;URL.s = "https://www.listrik.be/eendraadschema index_htm_files/81.jpg"
;URL.s = "https://www.purebasic.com/img/pb-logo-white.png"

*Buffer = ReceiveHTTPMemoryStrict(URL, #PB_HTTP_NoRedirect)
If *Buffer
  ShowMemoryViewer(*Buffer, MemorySize(*Buffer))
  CallDebugger
  FreeMemory(*Buffer)
Else
  Debug "Failed"
EndIf
Ok... nice code
I would expect to do al this checking stuff with HTTPRequestMemory()
The ReceiveHTTPMemory() however is explicitly to receive a pointer to a file, not some other stuff the server decides to send, so i would expect the function itself to do the checking and return an valid pointer or 0.
I was just mislead by the documentation: Returns nonzero if the download was successful, zero otherwise

Simply solved it like this

Code: Select all

Procedure.i MyReceiveHTTPMemory(url.s)
  Define *Buffer = ReceiveHTTPMemory(url, #PB_HTTP_NoRedirect)
  If *Buffer = 0 Or FindString(PeekS(*Buffer, MemorySize(*Buffer), #PB_UTF8), "404 Not Found") > 0
	Retun 0
  else
	ProcedureReturn *Buffer
  EndIf
EndProcedure
Post Reply