PureBasic Forums - English

Posted: **Wed Jun 12, 2024 2:44 am**

Hi @infratec, thanks for your amazing work and dedication.

I can't find any license notice in the main source file nor in this thread, so I was wondering whether you're willing to add a license to your libcurl.pbi project — the MIT License would be perfect, and in line with the cURL license, which inspired on the MIT, ISC, 3-clause BSD, and 4-clause BSD licenses (which are all mostly compatible, just different legalese involved):

https://github.com/curl/curl/blob/master/COPYING

But really, any of the above mentioned licenses would be fine, including the cURL license itself.

The lack of an explicit license is a big obstacle for me currently, preventing me from using it in my own projects. Actually, using the bindings without a proper FOSS license is illegal. Many people think that as long as the original developer is OK with it everything is fine, but that's not the case. In Europe (particularly, in Germany) there are agencies actively engaged in finding license violators (which include those who didn't properly credit a license, as legally required) and fine them. The following case mentioned by Wikipedia shows how the German VGSE (Association for the Protection of Intellectual Property in the Internet) actively hunts license violators on the Internet to fine them:

https://en.wikipedia.org/wiki/Creative_ ... .28VGSE.29

So, licenses being legally binding agreement contracts in every aspect (i.e. Copyright being the natural condition of unlicensed material, in case of a license it's assumed the end user has agreed to its legal terms), the ways of doing things in the past where generic approval by the original creator (expressed in forums discussions, or via his/her implicit consent to witnessing it being used by third parties) was considered enough, is no longer feasible today.

I also believe that by giving it a license you'd ensure continuity to the binding project, e.g. in case at some point in the future you can't/won't keep updating it.

Unfortunately, many good PureBasic projects seem to have already gone lost, with their forum links having gone dead in the course of time. Today it's possible to ensure preservation of PB projects by republishing them on platforms like GitHub, GitLab, etc., but this can only be done for projects which have a proper FOSS license, since all the major hosting platforms now enforce Copyright laws.

Again, thanks for the excellent job with libcurl PureBasic! And keep up the good work...

Posted: **Wed Jun 12, 2024 8:45 am**

Have you really in each of your files a copyright text

If I follow you, I have to copy the MIT text in each of the files.

How can you violate a license when there is none?

Maybe Fred should add in the impressum that each code provided by a user is MIT licensed.

I will add a feature request for this.

Posted: **Wed Jun 12, 2024 1:43 pm**

infratec wrote: Wed Jun 12, 2024 8:45 am Have you really in each of your files a copyright text

If I follow you, I have to copy the MIT text in each of the files.

You don't have to do that, it's enough to have a LICENSE (or COPYING) file in the project, somewhere. It's good practice to at least add the Copyright statement in each source file though (e.g. "XXX (c) Copyright by Name, 2024, MIT License"), just to ensure that it's clear which files fall under the project license — usually, in a project there are also other files, many of which don't necessarily fall under the main license, e.g. third party scripts, or documentation, and so on.

If you really want to make your project globally findable by it's license, you could also stick to SPDX and add the required metadata, so that your project gets indexed according to its license by dedicated robots that mine for packages:

https://spdx.dev/
https://en.wikipedia.org/wiki/Software_ ... a_Exchange

infratec wrote: Wed Jun 12, 2024 8:45 am How can you violate a license when there is none?

If there's no license attached then it's assumed it's protected by Copyright, so it's a case of Copyright infringement — which, by the way, in most countries is considered a crime, not just an infraction. Right now the state of you bindings, not having any license attached, is to be considered as copyrighted material.

infratec wrote: Wed Jun 12, 2024 8:45 am Maybe Fred should add in the impressum that each code provided by a user is MIT licensed.

But then, not every project might be MIT Licensed, it really depends on the code desires, or sometimes one is forced to use another license because of some third party components which enforce more restrictive licenses (e.g. GPL family, etc.).

For Fred to enforce a standard license on code submitted by third parties would be tricky. Although many adopt the strategy of documenting that every submitted code falls under a given license, from a legal point of view that's not legally binding. The proper way to do that (as seen on many GitHub/GitLab repositories) is to have each and every contributor sign online a CLA (Contributor License Agreement) before submission, as a proof that the author really intended to release his/her code under that license:

https://en.wikipedia.org/wiki/Contribut ... _Agreement

It's much easier if the author himself assigns the license, directly, than having third party middle-persons arranging for that.

Software licenses, in general, are a rather complex topic, especially with international laws being at place, etc., and unlike big corporations (which have lots of specialized lawyers on the payroll) for indy developers the easiest approach is to stick to well tested licenses in the standard way.

Posted: **Mon Jun 24, 2024 7:57 am**

updated the libcurls dlls to version 8.8.0

Posted: **Wed Jul 10, 2024 4:10 pm**

I tried to download an email with one "larger" attachment (12MB) using the example in pop3.pbi. The download only works up to approx. 2MB and then hangs. After closing the application the debugger reports on Win7:

file: libcurl.pbi
procedure: ProcedureC.l LibCurl_WriteFunction()
error: invalid memory access

On Mac function Pop3_GetMessageByNumber() hangs as well without error message.

No problems with small files on windows and Mac.

Is it possible to simply increase the memory for large messages before downloading them?

Posted: **Wed Jul 10, 2024 8:12 pm**

I updated the pop3.pbi file.

You can now use an optional filename for the mail.
Please try this out. The mail is then saved into this file.
If it was succesfull, the filename is returned.

Posted: **Thu Jul 11, 2024 3:11 pm**

Hello, thanks for the information. I've now tried the code on MAC OS (curl 7.54.0 ).

The larger message (about 13 MB with attachment) is now saved in the file, however the end of the file is somehow cut off. For example, in the case of an attachment, the end of the boundary is missing.

The behavior is similar for messages without attachments. Sometimes a piece of HTML code is missing in the file.
For smaller messages - with small attachments - writing to the string (*userdata\String) is the better solution.

The behavior is strange. The same pointers etc. are used both for writing to the string and to the file. The function LibCurl_WriteFunction() is called by curl for each line of the transmitted message... Maybe there is a timing problem somewhere ? Why can large messages be written to a file and not to the string?

I will now test it under Windows...

Posted: **Thu Jul 11, 2024 3:33 pm**

The problem is

String$ + NewPart$

PB needs a linear address range for the String$
If there is no more space left at the original place it seems to crash.

Writing to a file is never a problem.
I don't think that it is a timing problem, since curl is waiting for the return of the write function.

Posted: **Fri Jul 12, 2024 9:02 pm**

I have now found another solution. The function LibCurl_WriteFunction() is called from curl for each line of the message. So I added the individual lines to a list and copied the elements into a buffer later...

Here are the relevant procedures:

you have to add the global in your code:

Code: Select all

Global NewList Pop3_Message.s()

I changed the function LibCurl_WriteFunction() as follows:

Code: Select all

ProcedureC.l LibCurl_WriteFunction(*ptr, Size, NMemB, *userdata.libcurl_userdata_structure)
  Protected SizeProper.i = Size & 255
  Protected Result.l      
  If *userdata
    If IsFile(*userdata\File)
      Result = WriteData(*userdata\File, *ptr, SizeProper * NMemB)
    Else
      ; save each line in list, NOT in string
      AddElement(Pop3_Message())
      Pop3_Message() =  PeekS(*ptr, SizeProper * NMemB, #PB_UTF8 | #PB_ByteLength)          
      Result = SizeProper * NMemB
    EndIf
  EndIf
  ProcedureReturn Result
EndProcedure

Here the procedure Pop3_GetMessageByNumber():

Code: Select all

Procedure.s Pop3_GetMessageByNumber(Url$, User$, Password$, MessageNumber.i, FileName$="")
  
  Protected curl.i, res.i, i.i, Message$, userdata.libcurl_userdata_structure
  Protected *Pop3_Buffer,*Pop3_Pointer
  
  curl = curl_easy_init()
  If curl
    ; Set username And password
    curl_easy_setopt_str(curl, #CURLOPT_USERNAME, User$)
    curl_easy_setopt_str(curl, #CURLOPT_PASSWORD, Password$)
    curl_easy_setopt_str(curl, #CURLOPT_URL, "pop3s://" + Url$ + "/" + Str(MessageNumber))

    ;curl_easy_setopt(curl, #CURLOPT_CONNECTTIMEOUT, 5)  ;
   ; curl_easy_setopt(curl, #CURLOPT_TIMEOUT, 15)          ; Timeout
    curl_easy_setopt(curl, #CURLOPT_SSL_VERIFYPEER, 1)
    curl_easy_setopt(curl, #CURLOPT_SSL_VERIFYHOST, 1)  
    curl_easy_setopt(curl, #CURLOPT_VERBOSE, 1)
    
    If FileName$ <> ""
      userdata\File = CreateFile(#PB_Any, FileName$)
    EndIf
    
    curl_easy_setopt(curl, #CURLOPT_WRITEFUNCTION, @LibCurl_WriteFunction())
    curl_easy_setopt(curl, #CURLOPT_WRITEDATA, @userdata)
    
    ; Perform the retr
    res = curl_easy_perform(curl)
    If res = #CURLE_OK
      If FileName$ <> ""                          ; read from file
        If IsFile(userdata\File) 
          CloseFile(userdata\File)      ; file is still open, we have to close first
          If ReadFile(0, FileName$)      
            Message$ = ReadString(0, #PB_File_IgnoreEOL)  
            CloseFile(0)  
            DeleteFile(FileName$)
          EndIf
        EndIf
  
      Else

        ; allocate max memory (Test)
        *Pop3_Buffer  = AllocateMemory(100000000)
        *Pop3_Pointer = *Pop3_Buffer
           
        If *Pop3_Buffer                                  ; copy list to memory
          If ListSize(Pop3_Message()) > 0
            CopyMemoryString("", @*Pop3_Pointer) 
            ; copy each line to the end of memory
            ForEach Pop3_Message() : CopyMemoryString(Pop3_Message()) : Next
            ; read message
            Message$ = PeekS(*Pop3_Buffer)
            ; cleanup
            FreeMemory(*Pop3_Buffer)  
            ClearList(Pop3_Message()) 
          EndIf
        EndIf
      EndIf  
    Else
      Debug "curl_easy_perform() failed: " + curl_easy_strerror(res)
    EndIf
    
    ; Always cleanup
    curl_easy_cleanup(curl)
  EndIf
  ProcedureReturn Message$
EndProcedure

Both variants work well so far with larger attachments and no longer crash like the original code.
Sure, you may still have to determine how much memory you really need, but for the first solution it works quite well...

I keep testing...

Posted: **Sat Jul 13, 2024 1:23 am**

ok, of course it also works without a list...

Here the code, but not yet fully developed:

Code: Select all

      ProcedureC.l LibCurl_WriteFunction(*ptr, Size, NMemB, *userdata.libcurl_userdata_structure)
          Protected SizeProper.i = Size & 255
          Protected Result.l
          Protected temp$ 
          
          If *userdata
            If IsFile(*userdata\File)
              Result = WriteData(*userdata\File, *ptr, SizeProper * NMemB)
            Else 
              temp$=PeekS(*ptr, SizeProper * NMemB, #PB_UTF8 | #PB_ByteLength)
              CopyMemoryString(temp$)          
;              *userdata\String + PeekS(*ptr, SizeProper * NMemB, #PB_UTF8|#PB_ByteLength)              
              Result = SizeProper * NMemB
            EndIf
          EndIf
          
          ProcedureReturn Result     
        EndProcedure

and changes for Procedure.s Pop3_GetMessageByNumber()

Code: Select all

Procedure.s Pop3_GetMessageByNumber(Url$, User$, Password$, MessageNumber.i, FileName$="")
  
  Protected curl.i, res.i, i.i, Message$, userdata.libcurl_userdata_structure
  Protected *Pop3_Buffer, *Pop3_Pointer
  
  curl = curl_easy_init()
  If curl
    ; Set username And password
    curl_easy_setopt_str(curl, #CURLOPT_USERNAME, User$)
    curl_easy_setopt_str(curl, #CURLOPT_PASSWORD, Password$)
    curl_easy_setopt_str(curl, #CURLOPT_URL, "pop3s://" + Url$ + "/" + Str(MessageNumber))

    ;curl_easy_setopt(curl, #CURLOPT_CONNECTTIMEOUT, 5)  ;
   ; curl_easy_setopt(curl, #CURLOPT_TIMEOUT, 15)        ; curl_easy_perform() failed: Timeout was reached
    curl_easy_setopt(curl, #CURLOPT_SSL_VERIFYPEER, 1)
    curl_easy_setopt(curl, #CURLOPT_SSL_VERIFYHOST, 1)  
    curl_easy_setopt(curl, #CURLOPT_VERBOSE, 1)
    
    If FileName$ <> ""
      userdata\File = CreateFile(#PB_Any, FileName$)
    Else
      ; allocate max memory (Test)
      *Pop3_Buffer  = AllocateMemory(100000000)
      *Pop3_Pointer = *Pop3_Buffer        
      If *Pop3_Buffer
        CopyMemoryString("", @*Pop3_Pointer) 
      EndIf  
    EndIf
    
    curl_easy_setopt(curl, #CURLOPT_WRITEFUNCTION, @LibCurl_WriteFunction())
    curl_easy_setopt(curl, #CURLOPT_WRITEDATA, @userdata)
    
    ; Perform the retr
    res = curl_easy_perform(curl)
    
    If res = #CURLE_OK
      If FileName$ <> ""      
        If IsFile(userdata\File) 
          CloseFile(userdata\File)        ; file is still open, we hav to close first
          If ReadFile(0, FileName$)      
            Message$ = ReadString(0, #PB_File_IgnoreEOL)  
            CloseFile(0)  
            DeleteFile(FileName$)
          EndIf
        EndIf
  
      Else

       If *Pop3_Buffer                      
          ; read message
          Message$ = PeekS(*Pop3_Buffer)
          ; cleanup
          FreeMemory(*Pop3_Buffer)  
          ClearList(Pop3_Message()) 
        EndIf
      EndIf  
    Else
      Debug "curl_easy_perform() failed: " + curl_easy_strerror(res)
    EndIf
    
    ; Always cleanup
    curl_easy_cleanup(curl)
  EndIf
  ProcedureReturn Message$
EndProcedure

what do you think ?

Posted: **Sat Jul 13, 2024 11:04 am**

I changend now the Write function inside libcurl.pbi like the one from HTTPRequestI.pbi

The problem should be fixed.

Please download (in a different directory) again and test it.
I also modified all examples which used the original Write procedure.

This way it is faster and don't use always a string function inside of the callback.

Posted: **Wed Jul 17, 2024 9:58 am**

I have now tested and it works just like in my example for Win7/Mac. The problem with the attachments seems to be fixed.
At least I tested up to 23 MB ...

But what I notice is that libcurl is extremely slow with larger files.
Downloading the attachments takes approx. 8x as long as with Thunderbird or Outlook.

Posted: **Wed Jul 17, 2024 10:11 am**

What I also noticed is that under Windows 7/10 no SSL connection (port 995) is possible to receive the messages.

I tested on Win 7, Win 10 but it doesn't work with pop3s. Unsecured connections are working.
The error message is: curl_easy_perform() failed: SSL peer certificate or SSH remote key was not OK

Also using different options e.g "curl_easy_setopt(curl, #CURLOPT_SSLVERSION, #CURL_SSLVERSION_MAX_TLSv1_2) doesn't help.
No problem under MAC, as libCurl is already integrated there...

There is no certificate error.
Other programs like Outlook, Thunderbird or even the PB example with LibSSL work fine ...

Any ideas ?
I read on the web (many people have this problem) that it could possibly be due to the version?

Posted: **Fri Jul 19, 2024 1:33 pm**

Is there any source where I can download old dlls for Windows 7 (x64) ?

As often described here in the forum and in the web, there are different versions of libcurl.dll with different errors. Therefore I need a source that still offers old versions (x64) for download...

I only need one functionality from libcurl. And this is to download of a message via Pop3s (SSL).
This works without any problems on MAC, as curl is included with MAC.

But for Windows 7 and Linux there are often problems when using Pop3s/SSL.
Some people report that version 7.1.0 still works, others report that version 7.5.3 works. I would like to test this out...

Thx in advance for any information.

Posted: **Fri Jul 19, 2024 10:04 pm**

On windows 7 ? There is no TLS 1.2+ so it cant work with actual servers ... Or am I wrong ?

PureBasic Forums - English

libcurl.pbi

License?

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

libcurl.pbi - problems with large attachments

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi

Re: libcurl.pbi