Regarding to this thread http://www.purebasic.fr/english/viewtopic.php?t=30671 I want to make suggestions how to improve this library.
I also appreciate, if others can give us their comments on what they'd like to see.
Basically, I would suggest to introduce the system with the IDs, but also add one/some "quick commands" (don't know, how to call them).
First, I talk about the "quick commands". Second, I talk about the actual feature request.
-------------------------------------------------------------------------------------
A "quick command" could be ReceiveHTTPFile(). It's task is just to send a quick request that does not require any HTTP knowledge of the programmer.
- Advantage: Quickly download a HTTP-file without any HTTP knowledge
Disadvantage: Low accuracy (see 'bugs' in linked topic above) and no preferences can be specified
- ReceiveHTTPHeader()
Already existing, should send a "HEAD"-Request to gain general information about a HTTP file - GetHTTPFileAttribute(URL$, Flag)
Should aquire some detailed information about a HTTP-File. Can be controled via flags. These flags could be for example #PB_HTTP_FileSize (get file size), #PB_HTTP_GetDate (get last modification date) or #PB_HTTP_PartialDownloadAvailable (should detect, wheter this file may be downloaded partially (useful to continue an aborted download, e. g. for download managers)).
All these information can be aquired by using "HEAD" instead of "GET". HEAD advises the server just to send the HTTP respond without the file itself. Useful if you want the information quickly.
This is also useful to know the file size before downloading and you can also get the information, if you can download a file partially. if yes, the server should send the "Accept-Ranges" attribute.
-------------------------------------------------------------------------------------
For the HTTP experts and for those, who want more control there should be the actual HTTP-library that bases on the ID system.
For each HTTP transaction you create your own HTTP job through special commands.
The "new"/reconstructed HTTP library should offer (at least) the following possibilities:
- download a HTTP document/file
- to disk
- to memory
- get specific and detailed information also before the transaction starts
- file size
- partial download possible
- last modification date
- ...
- programer is able to interfer:
He can manipulate the HTTP request, e. g. the User-Agent attribute, HTTP version, etc. - Cookie handling
- POST-data handling (for data and file upload)
- Inform the user by passing through the HTTP status codes
- Authentification handling (don't swap this with HTTPS!)
- Callback handling (to make the programer able to calculate downloadspeed, remaining time, etc.)
- Proxy handling and HTTPS support
(Attention: This is only an imaginary idea. I don't know how/if this can be realized. Furthemore, this last point has a low priority IMHO).
As I said, I would vote for an ID system like you assign IDs to files, images, sprites, etc.
The advantage is that the HTTP management is simple, but very powerful!
Due to different IDs, you can differ between many HTTP transactions. You can also download more files simultaniously (e. g. by using threads).
Remember that if you don't need this extended stuff, you can still use the "quick commands" as I described above.
I don't know, how PB handles this system with the IDs internally; I would do this with a (global/shared) LinkedList.
Nevertheless, let's have a look, how an extended HTTP transaction in PureBasic could look like:
Code: Select all
InitNetwork()
; A very simple download
AllocateHTTP(0, "http://www.and51.de/index.html")
Debug DownloadHTTP(0, "C:\index.htm")
FreeHTTP(0)
; A more complex download
Define *greetins=AllocateMemory(100)
PokeS(*greetings, "Hello from PureBasic!")
AllocateHTTP(1, "http://www.and51.de/index.html?language=de")
Debug ExamineHTTP(1)
Define SIZE=GetHTTPAttribute(1, #PB_HTTP_FileSize)
Debug FormatDate("%hhh$iim%sss %dd-%mm-%yyyy", GetHTTPAttribute(1, #PB_HTTP_ModificationDate))
; Okay, we got our information and we can "prepare" the download
; Let's say, we're dependent of HTTP/1.0, we only want the first 50 bytes and we want to send some POST data
SetHTTPRequest(1, #PB_HTTP_UserAgent, "myPureBasic Program/1.0")
SetHTTPRequest(1, #PB_HTTP_Range, "50")
SetHTTPRequest(1, #PB_HTTP_CacheControl, "only-if-chaced")
SetHTTPRequest(1, #PB_HTTP_Referer, "http://www.purebasic.com/")
SetHTTPRequest(1, #PB_HTTP_HTTPVersion, "1.0")
SetHTTPRequestPost(1, #PB_HTTP_PostData, *greetings)
; No more ideas so let's accomplish the job!
Debug DownloadHTTP(1, "C:\temp.txt")
; Second download, but the complete file this time
SetHTTPRequest(1, #PB_HTTP_Range, "")
Define *buffer=AllocateMemory(SIZE)
Debug ReceiveHTTP(1, *buffer)
FreeHTTP(1)Well, this could be easily done by a "quick command", but it should show you the elementary idea. Similiar to the file library, for example, you create a HTTP job and free it afterwards. During the meantime, you can do many things with it.
The more complex example
First, we need a buffer for the POST data, we want to send later on.
Second, we prepare the HTTP transaction. Therefore we use the AllocateHTTP() command that expects an ID (could also be #PB_Any) and the URI (Unique Resource Identifier) (in fact, URI = URL).
The URI can also contain the port (if port <> 80) and it can contain GET-Variables).
Internally, AllocateHTTP() could add an element to the global HTTP-List which is accessed by the other HTTP-functions. This would be my way of realizing it; as I said, I don't know how PB manages IDs internally. If the system of PB is better/different you can use PB's system.
In order to acquire information about a URI before the actual download begins, there are 2 possibilities:
Either each GetHTTPAttribute() creates its own connection to the server getting the specified information or each GetHTTPAttribute() call accesses the global LinkedList which is being filled with infotmation by a previous call of ExamineHTTP().
In fact, I prefer the second choice, because it saves time as there is only 1 connection necessary.
Of course, the returning value ExamineHTTP() should be checked, to detect, wether the information could be retrieved successfully. I abdicated this to simplify the exmaple.
Once, the information have been gained, you can read them. First, we save the URI size to a variable for later use.
Then you might want to know the modification date, so we debug it after passing FormatDate().
At this point, you could also detect, if the download is resumeable (#PB_HTTP_PartialDownloadAvailable). Remember this constant? I used it above, when I talked about the "quick commands".
Before the actual download, we specifiy the parameters. Our own User-Agent (yabadadoo!) and the Referer. Note: Some sites require the referer, so this is also a reson to allow the programer to interfere.
When we save the file to disk, we only want the first 50 byte of the URI.
Just because this is an example code, we also want cached-only files and because we're so funny, we set the HTTP-Version to 1.0.
Another important thing is the ability to append POST data. Remember that when you append POST data, you must add two HTTP-attributes:
1) Content-Length (tell the server how large your data bombardement is
)2) Content-Encoding (tell the server what type of data you're sending (default: "multipart/formdata" or "Application/www-url-encoded")
Exactly this is the task of the HTTP library: It adds these two information on his own (whereas the programer still can change the MIME-type later on).
This is an example, so we download the URI regarding the given parameters. The Debug in front of DownloadHTTP() should remind us of the returning value. Remember, it would be useful to pass information to the programer.
In this case, the returning value could be 1 or 0 (self explaing what this stands for).
Attention: Please, differ bewteen DownloadHTTP() and ReceiveHTTP()! The first command saves the result to disk, the second pokes the URI to the memory.
We didn't free the HTTP job, so we're able to repeat it as often as we want. In this case we want to whole file into the memory.
All we need to do is to slightly change the parameters. Not only download the first 50 bytes, but the whole file, so we need to delete the Range-Attribute. We did save the URI size to a variable before, so we're able to allocate a dynamic memory block which is filled by ReceiveHTTP().
Remember that this is an example!
I don't know if it's best to feed the procedure with a user-created buffer or if the procedure should create it's own buffer. This is one of many points we have to discuss (we = PB Team & PB community).
So don't be shy and post your comment here!
Decide on your own:
- the procedure creates the buffer
If the procedure creates the buffer, it must return the address. The concrete advantage is that this is more comfortable for the programer. - the user provides a buffer
If the user provides a buffer, the procedure is able to pass through the very important HTTP status codes! The programer is able to detect which code has been returned! He can easily find out if the download was successfull (code 200/OK), the partial download was successfull (code 206/Partial Content) or if the server processed the request but doesn't send data back (code 204/No Content). Of course other codes can be returned as well, for example a sever error (code 500/Internal Server Error), errors occured in the request (code 400/Bad Request) or the very famous code 404/File not Found.
-------------------------------------------------------------------------------------
Closing words:
- This bloody thread took me >1 hour, so read it!!!!
- Feel free to post your own comment!
Please, make suggestions, changes or ask if something is unclear! - I'm looking forward to read your feedback @PB-Team and @PB-Community.
Happy coding, AND51
But yea I just meant download to a memory buffer.
