download a file

Crystal Noir · Post by **Crystal Noir** » Mon Oct 30, 2006 1:11 am

Hi all

I use Pb 3.94 with mandriva 2007 (v4 doesn't work with me).

I search how to download a file via http. I have found some code in there but they don't work

I have some errors with constant like #RF (not found).

So how can I download a file with pb via http (I don't want to use an external program like wget).

Thx a lot.

[/code]

ricardo · Post by **ricardo** » Mon Oct 30, 2006 1:24 am

One easy option is

Code: Select all

FileToDownload$ = "http://www.site.com/myfile.exe"
PathTosaveFile$ = "c:\myfile.exe"
URLDownloadToFile_(0,FileToDownload$,PathTosaveFile$,0,0)

walker · Post by **walker** » Mon Oct 30, 2006 1:41 am

please see which forum the post is in.... WINAPI doesn't help much on Linux

My suggestion: use RunProgramm() and call wget or curl to download a file,,,,

Crystal Noir · Post by **Crystal Noir** » Mon Oct 30, 2006 11:36 am

yes but if the user don't have wget our curl....

WishMaster · Post by **WishMaster** » Mon Oct 30, 2006 12:25 pm

wget should exist everywhere

DarkDragon · Post by **DarkDragon** » Mon Oct 30, 2006 1:05 pm

Code: Select all

; HTTP Commands - DarkDragon
; ***************************
; Commands:
; ---------
; HTTPRequest(URL.s, *EndSize.LONG, PacketSize, *Callback) - Result: Pointer to a buffer containing the webdata
; URL.s       - [in]  Nullterminated string with the link to the file
; *EndSize    - [out] Pointer to a long variable which receives the size of the resulting buffer
; PacketSize  - [in]  Long variable containing the size of the packets(steps) to download
; *Callback   - [in]  Pointer to a callback method receiving the following parameters: CurrentDownloadedSize.l, MaximumLength.l

; HTTPRequest_Password(URL.s, *EndSize.LONG, PacketSize, Username.s, Password.s, *Callback) - Result: Pointer to a buffer containing the webdata
; URL.s       - [in]  Nullterminated string with the link to the file
; *EndSize    - [out] Pointer to a long variable which receives the size of the resulting buffer
; PacketSize  - [in]  Long variable containing the size of the packets(steps) to download
; Username.s  - [in]  Nullterminated string containing the username
; Password.s  - [in]  Nullterminated string containing the password(NOT encoded)
; *Callback   - [in]  Pointer to a callback method receiving the following parameters: CurrentDownloadedSize.l, MaximumLength.l

InitNetwork()

#TIMEOUT = 1500

CompilerIf #PB_Compiler_OS = #PB_OS_Linux
  #CR$ = Chr(13)
  #LF  = 10
  #LF$ = Chr(10)
  #CRLF$ = #CR$+#LF$
CompilerEndIf

Procedure.s ReceiveLine(ConnectionID)
  Text.s = ""
  While char.b <> #LF
    char = 0
    ReceiveNetworkData(ConnectionID, @char, 1)
    If char <> 0
      Text.s + Chr(char)
    EndIf
  Wend
  
  ProcedureReturn RemoveString(RemoveString(Text, #CR$), #LF$)
EndProcedure

Procedure HTTPRequest(URL.s, *EndSize.LONG, PacketSize, *Callback)
  Protected Size.l, Data_.s, s.l, Method.s, File.s, Content.s, Text.s, Length.l, Line.s
  Protected CurSize.l, oSize.l, t.l, ConnectionID.l
  
  If Left(URL, 7) = "http://" : URL = Right(URL, Len(URL)-7) : EndIf
  s = FindString(URL, "/", 1)
  Host.s = "" : File.s = ""
  If s <> 0 : Host.s = Left(URL, s-1) : File.s = Right(URL, Len(URL)-s) : Else : Host = URL : EndIf
  
  s = FindString(File, "?", 1)
  If s <> 0
    Method.s = "POST"
    Content.s = Right(File, Len(File)-s)
    File = Left(File, s-1)
  Else
    Method.s = "GET"
    Content.s = ""
  EndIf
  
  ConnectionID = OpenNetworkConnection(Host, 80)
  If ConnectionID
    Data_.s = Method+" /"+File+" HTTP/1.0"+#CRLF$
    Data_.s + "Host: "+Host+#CRLF$
    Data_.s + "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7"+#CRLF$
    If Content <> ""
      Data_.s + "Connection: close"+#CRLF$
      Data_.s + "Content-Type: application/x-www-form-urlencoded"+#CRLF$
      Data_.s + "Content-Length: "+Str(Len(Content))+#CRLF$
      Data_.s + #CRLF$ + Content.s + #CRLF$+#CRLF$
    Else
      Data_.s + "Connection: close"+#CRLF$+#CRLF$
    EndIf
    
    SendNetworkString(ConnectionID, Data_)
    While NetworkClientEvent(ConnectionID) <> 2 : Delay(10) : Wend
    
    Length = 0
    Line.s = ""
    Repeat
      Line = ReceiveLine(ConnectionID)
      Select LCase(StringField(Line, 1, ":"))
        Case "content-length"
          
          Length = Val(Trim(StringField(Line, 2, ":")))
      EndSelect
    Until Len(Trim(Line)) <= 4
    
    Size = 0
    If Length <> 0
      
      *Result = AllocateMemory(Length)
      
      While Size < Length
        If CurSize > 0
          Size + CurSize
          If *Callback
            CallFunctionFast(*Callback, Size, Length)
          EndIf
        EndIf
        If Size > (Length-PacketSize)
          PacketSize = Length-Size
        EndIf
        If PacketSize > 0
        CurSize = ReceiveNetworkData(ConnectionID, *Result+Size, PacketSize)
        EndIf
      Wend
      
    Else
      
      *Buffer = AllocateMemory(PacketSize)
      *Result = AllocateMemory(1)
      t = ElapsedMilliseconds()
      While ElapsedMilliseconds()-t <= #TIMEOUT
        If NetworkClientEvent(ConnectionID) = 2
          CurSize = ReceiveNetworkData(ConnectionID, *Buffer, PacketSize)
          If CurSize > 0
            oSize = Size
            Size + CurSize
            *Result = ReAllocateMemory(*Result, Size)
            CopyMemory(*Buffer, *Result+oSize, CurSize)
            If *Callback
              CallFunctionFast(*Callback, Size, 0)
            EndIf
          EndIf
          t = ElapsedMilliseconds()
        EndIf
      Wend
      Length = Size
      
    EndIf
    
    *EndSize\l = Length
    If *Callback
      CallFunctionFast(*Callback, Size, Length)
    EndIf
    
    CloseNetworkConnection(ConnectionID)
    
    ProcedureReturn *Result
  EndIf
EndProcedure

Procedure HTTPRequest_Password(URL.s, *EndSize.LONG, PacketSize, Username.s, Password.s, *Callback)
  Protected Size.l, Data_.s, s.l, Method.s, File.s, Content.s, Text.s, Length.l, Line.s
  Protected CurSize.l, oSize.l, t.l, ConnectionID.l
  
  If Left(URL, 7) = "http://" : URL = Right(URL, Len(URL)-7) : EndIf
  s = FindString(URL, "/", 1)
  Host.s = "" : File.s = ""
  If s <> 0 : Host.s = Left(URL, s-1) : File.s = Right(URL, Len(URL)-s) : Else : Host = URL : EndIf
  
  s = FindString(File, "?", 1)
  If s <> 0
    Method.s = "POST"
    Content.s = Right(File, Len(File)-s)
    File = Left(File, s-1)
  Else
    Method.s = "GET"
    Content.s = ""
  EndIf
  
  InputBuffer.s = Username+":"+Password
  OutputBuffer.s = Space(256)
  Base64Encoder(@InputBuffer, Len(InputBuffer), @OutputBuffer, 255)
  
  ConnectionID = OpenNetworkConnection(Host, 80)
  If ConnectionID
    Data_.s = Method+" /"+File+" HTTP/1.0" + #CRLF$
    Data_.s + "Host: "+Host + #CRLF$
    Data_.s + "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7" + #CRLF$
    Data_.s + "Authorization: Basic "+ OutputBuffer + #CRLF$
    If Content <> ""
      Data_.s + "Connection: close" + #CRLF$
      Data_.s + "Content-Type: application/x-www-form-urlencoded"+#CRLF$
      Data_.s + "Content-Length: "+Str(Len(Content))+#CRLF$
      Data_.s + #CRLF$ + Content.s + #CRLF$+#CRLF$
    Else
      Data_.s + "Connection: close" + #CRLF$ + #CRLF$
    EndIf
    
    SendNetworkString(ConnectionID, Data_)
    While NetworkClientEvent(ConnectionID) <> 2 : Delay(10) : Wend
    
    Length = 0
    Line.s = ""
    Repeat
      Line = ReceiveLine(ConnectionID)
      Select LCase(StringField(Line, 1, ":"))
        Case "content-length"
          
          Length = Val(Trim(StringField(Line, 2, ":")))
      EndSelect
    Until Len(Trim(Line)) <= 4
    
    Size = 0
    If Length <> 0
      
      *Result = AllocateMemory(Length)
      
      While Size < Length
        If CurSize > 0
          Size + CurSize
          If *Callback
            CallFunctionFast(*Callback, Size, Length)
          EndIf
        EndIf
        If Size > (Length-PacketSize)
          PacketSize = Length-Size
        EndIf
        If PacketSize > 0
        CurSize = ReceiveNetworkData(ConnectionID, *Result+Size, PacketSize)
        EndIf
      Wend
      
    Else
      
      *Buffer = AllocateMemory(PacketSize)
      *Result = AllocateMemory(1)
      t = ElapsedMilliseconds()
      While ElapsedMilliseconds()-t <= #TIMEOUT
        If NetworkClientEvent(ConnectionID) = 2
          CurSize = ReceiveNetworkData(ConnectionID, *Buffer, PacketSize)
          If CurSize > 0
            oSize = Size
            Size + CurSize
            *Result = ReAllocateMemory(*Result, Size)
            CopyMemory(*Buffer, *Result+oSize, CurSize)
            If *Callback
              CallFunctionFast(*Callback, Size, 0)
            EndIf
          EndIf
          t = ElapsedMilliseconds()
        EndIf
      Wend
      Length = Size
      
    EndIf
    
    *EndSize\l = Length
    If *Callback
      CallFunctionFast(*Callback, Size, Length)
    EndIf
    
    CloseNetworkConnection(ConnectionID)
    
    ProcedureReturn *Result
  EndIf
EndProcedure

; Example:

; *Buffer = HTTPRequest("http://www.purebasic.com/", @Size, 16, 0)
; If *Buffer
;   Debug Size
;   Debug PeekS(*Buffer, Size)
;   FreeMemory(*Buffer)
; Else
;   Debug "Website not available!"
; EndIf

Should work, well if it says some constants are missing: just add them. I added the constants my code uses at the top(hopefully it works, don't know if Chr() is a macro or a function)

thamarok · Post by **thamarok** » Mon Oct 30, 2006 3:10 pm

A little information for you, Crystal Noir:
wget is a standard in all Linux platforms. On Debian, if you use apt-get, it will actually download the files via wget. On Red Hat it will also mostly use wget (don't know the name of the installer)..

ricardo · Post by **ricardo** » Mon Oct 30, 2006 5:51 pm

walker wrote: please see which forum the post is in.... WINAPI doesn't help much on Linux

Sorry

Crystal Noir · Post by **Crystal Noir** » Tue Oct 31, 2006 10:11 am

thx but doesn't work because of #LF what's this constant ?

ts-soft · Post by **ts-soft** » Tue Oct 31, 2006 10:37 am

Crystal Noir wrote:thx but doesn't work because of #LF what's this constant ?

Code: Select all

#LF$ = Chr(10)

DarkDragon · Post by **DarkDragon** » Tue Oct 31, 2006 10:50 am

ts-soft wrote:
Crystal Noir wrote:thx but doesn't work because of #LF what's this constant ?
Code: Select all
#LF$ = Chr(10)

#LF = 10
#LF$ = Chr(#LF)

It's called Line-Feed and on linux it's used for the newline character. On windows you need CRLF, so it's 2 symbols and on Mac it's just CR. I posted the code which should work. Try it.

thamarok · Post by **thamarok** » Tue Oct 31, 2006 11:37 am

DarkDragon wrote:
ts-soft wrote:
Crystal Noir wrote:thx but doesn't work because of #LF what's this constant ?
Code: Select all
#LF$ = Chr(10)
#LF = 10
#LF$ = Chr(#LF)

It's called Line-Feed and on linux it's used for the newline character. On windows you need CRLF, so it's 2 symbols and on Mac it's just CR. I posted the code which should work. Try it.

By the way, can someone explain me why is this?

On Linux, Line Feed
On Window$, Carriage Return + Line Feed
On Macintosh, Carriage Return

As you see, Window$ is the only one combining CR and LF. Was it designed so to be compatible with Mac and Linux documents? I don't get it.

bembulak · Post by **bembulak** » Tue Oct 31, 2006 11:42 am

I guess it was more like doing something completely different, negotiating standards or already existing things and so on. Same thing as M$ always does.

ts-soft · Post by **ts-soft** » Tue Oct 31, 2006 12:07 pm

The CRLF in windows comes from typewriter and teleprinter

Dare · Post by **Dare** » Tue Oct 31, 2006 12:10 pm

Think typewriters:

CR = Carriage Return (go back to start of line).
LF = New line.

Think 1950-ish - typist grabs the handle thing and pulls it left (CR), so the roller with the paper is now back at the start of the line. Pulling a little harder turns the paper roller, the paper moves up.

In the early days ascii codes decimal 13 and decimal 10 were assigned as control characters that would tell writers (line or barrel printers or whatever) to - tah dah : wait for it - go to the start of a line (CR) and go to a new line (LF).

You could do neat ascii art by telling the printer to CR without LF to get special effects.

So the standalone LF and the CR are the deviations from the original standard - which means doodly-squat nowadays.