DownloadToMemory (for all OS's)

[Heathen]

Code: Select all

Procedure.s DownloadToMemory(URL.s,maxbytes=0)
  Protected ServerID.l, Header.s, *Buffer = AllocateMemory(1000), String.s, Server.s, Path.s, i.l, DataLength.l,Bytes.l
 
  URL = RemoveString(URL,"http://",1)
  i = FindString(URL,"/",1)
  If i
    Server = Left(URL,i-1)
    Path = Right(URL,Len(URL)-i)
  Else
    Server = URL
  EndIf
 
  ServerID = OpenNetworkConnection(Server,80)
  If ServerID
    ;Header for the GET request
    Header = "GET /"+Path+" HTTP/1.1"+#CRLF$
    Header + "Host: "+Server+#CRLF$+#CRLF$
    SendNetworkData(ServerID,@Header,Len(Header)) ;Send the GET request
   
    Repeat ;Wait for it to start sending data
      Delay(2) ;No need to use 100% CPU while waiting
    Until NetworkClientEvent(ServerID) = #PB_NetworkEvent_Data
   
    Repeat ;Put all recieved data in a sring
      DataLength = ReceiveNetworkData(ServerID,*Buffer,1000)
      String + PeekS(*Buffer,DataLength)
      Bytes + DataLength
    Until DataLength = 0 or (maxbytes > 0 and bytes > maxbytes)
   
    FreeMemory(*Buffer) ;Don't need to use this memory anymore
   
    ;Cut of the header of the recieved data
    i = FindString(String,#CRLF$+#CRLF$,1)
    String = Mid(String,i+4,Len(String)-i-3)
    ProcedureReturn String
  EndIf
EndProcedure

[Joakim Christiansen]

There we have it, no need for panic!

[Heathen]

There we have it, no need for panic!

[ricardo]

And what about if i want to download a binary file (executable, mp3, sfw, etc) and keep it in memory (don't write it to disk)? How can do that?

[CadeX]

By simply... using the procedure and not writing it to a file? Read the script... It's not all just one procedure... there's stuff under it

[dracflamloc]

you could just use my procedure. just instead of using a file, use a buffer the size of the maxlen and writedata to the large buffer.

[Shannara]

Hmm, but the problem is, all the code posted so far only deal with strings and not binary data. Anybody have code for binary?

[dracflamloc]

If you look in my code you'll notice that it only deals with strings until the header information has been received. After that its just getting the raw data.

[Joakim Christiansen]

Is there a way to start it at a specific place?

(Nevermind, my uberness worked it out. Thanks for the tip, DarkDragon)

Care about sharing some code?

[thyphoon]

Hello

I try to mix some code to have a Download command who can use proxy and who are All Os Compatible
But i don't know how to erase header.
And i don't know why if i don't use Poxy After the header i have a value and at the end of the file a 0 ! Why ?

Code: Select all

Structure ProxyStructure
Adr.s
Port.l
login.s
password.s
EndStructure

Global Proxy.ProxyStructure
Proxy\Adr="xxxxxxxxxxxx" ;Url ou IP
Proxy\Port=xxxx ;Numero de port
Proxy\login="" ;Le login si il y en a un
Proxy\password="";Le mot de passe si il y en a un

InitNetwork()

Procedure.s MyDownload(Url.s,UseProxy=#False)
  Protected Server.s,Port.l,conc$,enc$,Header.s,Res.l,*Buffer = AllocateMemory(1024),String.s,DataLength.l
 
  ;Si j'ai un Proxy je dois me connecter au Proxy
  If UseProxy=#True
    Path.s=Url
    Server.s=Proxy\Adr
    Port.l=Proxy\Port
    conc$=Proxy\login+":"+Proxy\password
    OutputBuffer = AllocateMemory(Len(conc$)*4)
    Base64Encoder(@conc$,Len(conc$),OutputBuffer,Len(conc$)*4)
    enc$=PeekS(OutputBuffer)
 
  ;Si je n'ai pas de proxy on se connecte directement au serveur 
  ElseIf UseProxy=#False
    Port=80
    Path.s = Url
    Url.s = RemoveString(Url,"http://",1)
     i = FindString(Url,"/",1)
     If i
       Server = Left(Url,i-1)
     Else
       Server = Url
     EndIf
  EndIf
 
  ;Connection au Serveur
  ConnectionID = OpenNetworkConnection(Server, Port)
  If ConnectionID

    ;Creation de l'entête a envoyer
    Header ="GET "+Path+" HTTP/1.1"+#CRLF$
   
    If UseProxy=#True
      Header + "Proxy-Authorization: Basic "+enc$+#CRLF$
    Else
      Header + "Host: "+Server+#CRLF$+#CRLF$
    EndIf
 
    Header+#CRLF$
     Debug Header
    Res=SendNetworkData(ConnectionID,@Header,Len(Header))
 
    Delay(10)
    result = NetworkClientEvent(ConnectionID)
    Repeat ;On lit les données qui arrive
      DataLength = ReceiveNetworkData(ConnectionID,*Buffer,1024)
      String + PeekS(*Buffer,DataLength)
    Until DataLength<1024
    FreeMemory(*Buffer) ;POuff! Pu besoin on efface

    MessageRequester("Done!","Your Data" + Chr(13) + Chr(10) + Trim(String),0)
           
    ;C'est finit on se deconnecte
    CloseNetworkConnection(ConnectionID)
    ProcedureReturn String
  EndIf
EndProcedure

String.s=MyDownload("http://fr.yahoo.com",#True)
;  If CreateFile(0,"test.txt")
;    WriteData(0,@String,Len(String))
;    CloseFile(0)
;  EndIf
;  RunProgram("test.txt")

String.s=MyDownload("http://localhost/",#False)

can you help me ?

best regards

Thyphoon

[CadeX]

Is there a way to start it at a specific place?

(Nevermind, my uberness worked it out. Thanks for the tip, DarkDragon)

Care about sharing some code?

It honestly isn't hard, i don't have the code now seeing as it has been so long. Google is your friend.