HTTPS not working?

[stmdbe2019]

In a secured network i am only allowed to use HTTPS and the code needs to download PDF from the cloud.
PDF download from HTTP works. But not working when using HTTPS.

how to fix it?

Code: Select all

XIncludeFile "HTTP_GET.pbi"
If InitNetwork() = 0
  Debug "No TCP provider..."
Else
  ; WORKS
  ;Status.l = HTTP_GET("http://unec.edu.az/application/uploads/2014/12/pdf-sample.pdf", "C:\exe\print_tray1.pdf")
  ;Status.l = HTTP_GET("http://unec.edu.az/application/uploads/2014/12/pdf-sample.pdf", "C:\exe\print_tray5.pdf")

  ; !!! NOT WORKING !!! NOT WORKING !!!! NOT WORKING !!! OMG panic
  Status.l = HTTP_GET("https://www.antennahouse.com/XSLsample/pdf/sample-link_1.pdf","C:\exe\print_tray1.pdf") 
  MessageRequester("OK", Str(Status)) 
EndIf 

Code: Select all

; HTTP_GET.pbi
; by Rasmus Schultz
; Version 1.0c

; Based on a technique by Marius Eckardt (Thalius)

#HTTP_BUFFER_SIZE = 4096  ; receive buffer size
#HTTP_IDLE_DELAY = 20     ; delay during inactivity
#HTTP_LAG_DELAY = 50      ; delay during little activity
#HTTP_TIMEOUT = 10000     ; timeout in milliseconds

#HTTP_OK = 0

#HTTP_ERROR_CONNECT   = -1     ; Unable to connect to the specified server
#HTTP_ERROR_MEMORY    = -2     ; Unable to allocate memory for response buffer
#HTTP_ERROR_TIMEOUT   = -3     ; #HTTP_TIMEOUT exceeded
#HTTP_ERROR_FILE      = -4     ; Local file could not be created
#HTTP_ERROR_PROTOCOL  = -5     ; Unknown HTTP protocol (version 1.0 or 1.1 required)

Enumeration ; Parser states
  #HTTP_STATE_EXPECT_HEADER
  #HTTP_STATE_HEADER
  #HTTP_STATE_DATA
EndEnumeration

Macro HTTP_Debug(Str)
  Debug Str ; comment out this line to disable debugging messages
EndMacro

Procedure HTTP_GET(URL.s, LocalFilename.s)
  
  ; function returns 0 on successful download (HTTP status 200)
  ; negative number in case of tehnical errors (see #HTTP_ERROR codes above)
  ; or positive number in case of HTTP status code other than "200"
  
  Protected Host.s      ; Server's hostname
  Protected Path.s      ; Remote path
  Protected Port.l = 80 ; Port number
  
  Protected Pos.l       ; Used for various string operations
  
  Protected Con.l       ; Connection ID
  
  Protected Request.s   ; HTTP request headers
  
  Protected CRLF.s = Chr(13) + Chr(10)
  
  ; Parse URL:
  
  If FindString(URL, "http://", 1) = 1 : URL = Right(URL, Len(URL)-7) : EndIf
  
  Pos = FindString(URL, "/", 1)
  If Pos = 0
    Host = URL
    Path = "/"
  Else
    Host = Left(URL, Pos-1)
    Path = Right(URL, Len(URL)-Pos+1)
  EndIf
  
  Pos = FindString(Host, ":", 1)
  If Pos > 0
    Port = Val(Right(Host, Len(Host)-Pos))
    Host = Left(Host, Pos-1)
  EndIf
  
  HTTP_Debug("Host: " + Chr(34) + Host + Chr(34))
  HTTP_Debug("Path: " + Chr(34) + Path + Chr(34))
  HTTP_Debug("Port: " + Str(Port))

  ; Allocate response buffer:
  
  Protected *Buffer
  *Buffer = AllocateMemory(#HTTP_BUFFER_SIZE)
  If Not *Buffer : ProcedureReturn #HTTP_ERROR_MEMORY : EndIf
  
  ; Open connection:
  
  Con = OpenNetworkConnection(Host, Port)
  If Con = 0 : ProcedureReturn #HTTP_ERROR_CONNECT : EndIf
  
  ; Send HTTP request:
  
  Request = "GET " + Path + " HTTP/1.0" + CRLF
  Request + "Host: " + Host + CRLF
  Request + "Connection: Close" + CRLF + CRLF
  
  SendNetworkString(Con, Request)
  
  ; Create output file:
  
  Protected FileID.l = 0
  
  ; Process response:
  
  Protected Exit.l = #False   ; Exit flag
  
  Protected Bytes.l           ; Number of bytes received
  
  Protected Time.l = ElapsedMilliseconds() ; Time of last data reception
  
  Protected Status.l = #HTTP_OK ; Status flag
  
  Protected State.l = #HTTP_STATE_EXPECT_HEADER
  
  Protected String.s          ; Parser input
  Protected Index.l           ; Parser position
  Protected Char.s            ; Parser char
  
  Protected Header.s          ; Current header
  
  Protected HTTP_Protocol.s   ; HTTP protocol version
  Protected HTTP_Status.l = 0 ; HTTP status code
  
  Protected Redirected.b = #False
  
  Repeat
    
    If NetworkClientEvent(Con) = #PB_NetworkEvent_Data
      
      Repeat
        
        Bytes = ReceiveNetworkData(Con, *Buffer, #HTTP_BUFFER_SIZE)
        
        If Bytes = 0
          
          Exit = #True
          
        Else
          
          If Bytes < #HTTP_BUFFER_SIZE : Delay(#HTTP_LAG_DELAY) : EndIf
          
          HTTP_Debug("Received: " + Str(Bytes) + " bytes")
          
          If State = #HTTP_STATE_DATA
            
            WriteData(FileID, *Buffer, Bytes)
            
          Else
            
            String = PeekS(*Buffer, Bytes, #PB_Ascii)
            Index = 0
            
            Repeat
              
              Index + 1
              Char = Mid(String, Index, 1)
              
              Select State
                
                Case #HTTP_STATE_EXPECT_HEADER
                  If Char = Chr(10)
                    State = #HTTP_STATE_DATA
                    HTTP_Debug("Creating file: " + LocalFilename)
                    FileID = CreateFile(#PB_Any, LocalFilename)
                    If FileID = 0
                      Exit = #True
                      Status = #HTTP_ERROR_FILE
                    ElseIf Index < Bytes
                      WriteData(FileID, *Buffer+Index, Bytes-Index)
                    EndIf
                  ElseIf Char = Chr(13)
                    ; (ignore)
                  Else
                    Header = Char
                    State = #HTTP_STATE_HEADER
                  EndIf
                
                Case #HTTP_STATE_HEADER
                  If Char = Chr(10)
                    If HTTP_Status = 0
                      HTTP_Protocol = StringField(StringField(Header, 1, " "), 2, "/")
                      HTTP_Status = Val(StringField(Header, 2, " "))
                      If ((HTTP_Protocol <> "1.0") And (HTTP_Protocol <> "1.1")) Or (StringField(StringField(Header, 1, " "), 1, "/") <> "HTTP") Or (HTTP_Status = 0)
                        HTTP_Debug("HTTP Protocol error!")
                        Exit = #True
                        Status = #HTTP_ERROR_PROTOCOL
                      EndIf
                      HTTP_Debug("HTTP Protocol " + HTTP_Protocol + ", Status " + Str(HTTP_Status))
                      If (HTTP_Status >= 300) And (HTTP_Status < 400)
                        HTTP_Debug("Redirection...")
                        Redirected = #True
                      ElseIf HTTP_Status <> 200
                        Status = HTTP_Status
                        Exit = #True
                        HTTP_Debug("Status <> 200 - abort!")
                      EndIf
                    ElseIf Left(Header, 10) = "Location: "
                      Status = HTTP_GET(Right(Header, Len(Header)-10), LocalFilename)
                      Exit = #True
                    Else
                      HTTP_Debug(Header)
                    EndIf
                    State = #HTTP_STATE_EXPECT_HEADER
                  ElseIf Char = Chr(13)
                    ; (ignore)
                  Else
                    Header + Char
                  EndIf
                
              EndSelect
              
            Until (State = #HTTP_STATE_DATA) Or (Index = Bytes) Or (Exit = #True)
            
          EndIf
          
          Time = ElapsedMilliseconds()
          
        EndIf
        
      Until Exit = #True
      
    Else
      
      HTTP_Debug("Idle...")
      Delay(#HTTP_IDLE_DELAY)
      
      If ElapsedMilliseconds() - Time > #HTTP_TIMEOUT
        Exit = #True
        Status = #HTTP_ERROR_TIMEOUT
      EndIf
      
    EndIf
    
  Until Exit = #True
  
  ; Close and finish:
  
  CloseNetworkConnection(Con)
  FreeMemory(*Buffer)
  If FileID <> 0 : CloseFile(FileID) : EndIf
  
  ProcedureReturn Status
  
EndProcedure

Also, tried following but same it also fail for HTTPS

Code: Select all

; Download PDF from HTTPs
Procedure DownloadPDF(url$, file$)
  file$ = ""
  If InternetGetConnectedState_(@flags,#Null)  
    hResult = URLDownloadToFile_(#Null,url$,file$,#Null,#Null)  
    If hResult = #S_OK
      MessageRequester("OK", "OK")  
    Else
      MessageRequester("OK", "Not OK")       
    EndIf  
  EndIf  
EndProcedure

[wombats]

The built-in ReceiveHTTPFile command supports HTTPS. Is there a reason you're not using that?

[kenmo]

Any chance you are using Windows XP?

I went through a very similar situation, my PB program (running on XP) stopped working recently with HTTPS urls. I was using ReceiveHTTPFile().

Then I noticed that Internet Explorer failed to load the exact same URLs that PB failed to download!
So I assumed it was a network settings problem... tried dozens of fixes, nothing worked... PB and IE could not load HTTPS. (PB just calls Windows API, probably same functions as IE.)

BUT Firefox and Chrome worked fine... so it must just be a software problem...

I tried curl.exe commandline tool for downloading urls, it seemed to help but still was not perfect (I forget what cases were wrong).

Then I switched to wget.exe, similar tool but less features, and it works on all the HTTPS sites I had problems with.

I even wrote a macro which transparently replaces ReceiveHTTPFile() with a procedure that runs wget.exe.



SHORT ANSWER:
Download wget and try:

Code: Select all

wget.exe -O "file" -U "user-agent" "https://URL"

[infratec]

You have to use a libcurl pbi and not your currently used one.

[stmdbe2019]

Thanks following works.

Code: Select all

If InitNetwork() = 0
  Debug "No TCP provider..."
Else
  Filename$ = "C:\exe\download_pdf_from_https.pdf"
  If ReceiveHTTPFile("https://www.antennahouse.com/XSLsample/pdf/sample-link_1.pdf", Filename$)
    Debug "Success"
  Else
    Debug "Failed"
  EndIf
EndIf

[stmdbe2019]

HTTPS still failing in some Enterprise network but on the same network/PC when i use Google chrome/Chromium/Firefox/Opera the same HTTPS PDF link then the file is downloading. But its not working with default ReceiveHTTPFile.


Can someone with more knowledge debug and fix it (oviously a BUG)

Code: Select all

Procedure DownloadHTTPSPDF(url$, file$)
  If InitNetwork() = 0
    Debug "No TCP provider..."
  Else
    If ReceiveHTTPFile(url$, file$)
      Debug "Success"
    Else
      Debug "Failed"
    EndIf
  EndIf  
EndProcedure


DownloadHTTPSPDF("https://www.antennahouse.com/XSLsample/pdf/sample-link_1.pdf", "C:\exe\print_tray1.pdf")

[infratec]

Nothing to fix, it works here.

PB 5.61 x86 on Windows 10 x64.
Ups, I updated to 5.62. So 5.62 x86 is working.

[Fred]

May be you need to setup a proxy if it fails in enterprise network

[kpeters58]

Works perfectly fine within my company's corp. network (after I adjusted the destination path, of course)...

[infratec]

Well, sometimes you have to use a proxy server inside companies.
So check your browser setup if there is a proxy active.
If yes:

http://www.purebasic.fr/english/viewtop ... 13&t=67019