PureBasic Forums - English

Posted: **Wed Aug 26, 2015 12:38 pm**

There are numerous network protocols, but only a few essential ones to really get into. First and foremost is HTTP. Then the email protocols like POP3 and IMAP. The file transfer protocol of FTP comes in handy, and some games make use of UDP.

PureBasic does HTTP up to a point, but going beyond that point is neither documented for PureBasic or exemplified in sample code. Browsers have to do quite a bit when it comes to handling HTML documents, much of which is exemplified in WebBrowser.pb. This is only about 45 lines long, but I can't find anything to go to when it comes to trying to go further with it.

I've been looking at the nature of handling cookies, something that WebBrowser.pb does not do. I've learned that it is the browser's responsibiluty to make cookies happen. They have to send a request to the server for a request header, and then include that in ecery request they send the server going forward. It's pretty well described at https://en.wikipedia.org/wiki/HTTP_cookie.
There you also learn rhat RFC 6265 covers the accepted requirements of working with cookies.

Cookies perform an essential role when working with a browser and dealing with the sites you visit on the internet. Without enabling cookies on your browser, you are just an onlooker. You can't get anything done. So it's important to have the means of doing that. What I don't yet see is hiw to do it with PureBasic. Either is not there, not documented, or has not been coded into example code. If not there, it should be added, If not documented and.or coded, it whould ve addressed.

WebBrowser.pb shows how ro add and link three buttons to events. But that is only three, and there are many events possible with respect to the keyboard and mouse. What are the other ones named? Can't use it if you can't name it.

Posted: **Wed Aug 26, 2015 2:04 pm**

Unfortunately, I'm behind when it comes to different languages. Even human ones. I've said in the past that if I didn't have to know at least one language, I qould never have learned to use english. I didn't even learn the parts of speech until i tried to learn spanish, which I failed three times.

So when I find example code on the internet, I share the link and let others figure it out. If they want to post their results in changing it to PB, that's great. Helps everybody, not just me. But it isn't a requirement. I post it to help others as best I can, which is to show that it is possible, and how others have done it. Others could do as much of course, but sometimes the ball is in my court, so I do what I can.

Here is a link on actually doing the cooking thing as a client:
http://stackoverflow.com/questions/4579 ... h-a-cookie

Here is a detailed-oriented document:
http://curl.haxx.se/rfc/cookie_spec.html

Example code and other stuff:
https://github.com/request/request

Discussion of different and conflicting cookie standards:
http://hc.apache.org/httpclient-3.x/cookies.html

Choosing a cookie standard:
http://hc.apache.org/httpcomponents-cli ... emgmt.html

Doing the Client thing in XML using Java:
http://www.avajava.com/tutorials/lesson ... rvlet.html

Doing it with C# on Windows:
https://msdn.microsoft.com/en-us/librar ... s.95).aspx

Basic Cookie-Client Handling:
https://www.npmjs.com/package/cookie-client

Example HTTP Calls:
https://www.npmjs.com/package/client-http

Dealling with requests manually (how to test what you are doing in code):http-client-0.1.0.0: An HTTP client engine

An HTTP-Client Cookies Engine:
https://hackage.haskell.org/package/htt ... okies.html

The cookiejar library for Python:
https://docs.python.org/2/library/cookielib.html

PHP can handle it:
http://book.cakephp.org/3.0/en/core-lib ... lient.html

What's apparent is that there are solutions offered, but then you have to deal with several operating systems and different libraries and/or languages. Is there a one size fits all? That's not evident at this point. other input is welcomed.

Whu not just use CGI, PHP, or Python if available? I think the answer is, let's try to get all our eggs in one basket. If one language covers it all, that might be the language to go with. If you want PureBasic to be that language, then it might have to stretch a bit. After all, it already covers several platforms, which is more than can be said for a lot of other choices.

What I haven't uncovered yet is a treasure trove of programs written with PureBasic. Most important in my mind is not compiled code, but enough source code to show what it can do and how it has been done by others. One statement does not a program make. Many statements together is often required. It's example source code where you can really dig in and learn, by beginning with something that has proven out by the trial and error results of others. In one of the links I posted above, the writer wrote some code, then said "this should work". You mean he didn't bother ti test it first? What if he's wrong? Even a simple typo can spoil a line of code.

Posted: **Tue Sep 01, 2015 11:23 pm**

I've posted network code, both server and client. I know a number of people have as well.

Freak has posted Internet Information Server code that I have used to write dll's that are called natively by Microsoft's Internet Information Web Server.

I've posted code examples that allow the use of SSL from the fine folks that wrote CryptLib.

There's plenty of code there. Download it and try it out, When you have questions, just ask.

Rich

Posted: **Thu Sep 03, 2015 2:22 pm**

Thanks for your reply. Problem is, I'm not sure where to go to find any of the code you are referring to. Am I just not making the right connection? I recognize the Fred (or Freak) reference as being to the 2nd coder for PureBasic, this by way of the Support page at PureBasic.com. But I'm not able to relate your name to any tie-ins. Not that this is particularly important, because you are all way ahead of me, and I am grateful for any assist that I can get.

I'm adapting WebBrowser.pb to go after a large number of web sites that I can reach by other browser means. I'm trying to determine which ones work with it, and which don't. Some of the failures give you a reason, and I am trying to determine what is needed overall. It's slow going, but I don't know a better way to get this matter out for attention, likely by others.

I've done a fair amount of programming, but mostly database stuff. I had to learn to write terminal sessions to get to data stores on mainframes to get my data. Then I had to analyze the data collected over time for patterns. This was network data, about service quality and disruption of service. I learned something key that had not been evident before, that some network failures are slow in developing, so you begin with sporatic failures that increase in frequency and duration. Others are virtually instantaneous. You can program to look for the first, but you will always be caught flatfooted by the second,

I had a director once that thought you could use statistics to project ahead what would fail next, and he wanted a process created to eliminate flaws before they happened by replacing things that would fail -- eventually. Given enough time. Thar's not what statistics are good for. The best you can do with statistics is look for probable points of failure first, and make sure you have enough spares on hand to treat the likely causes. There is no way to predict what will fail next, or where the failure will occur. He didn't understand that. He didn't last as director very long, but for other reasons. Just a strange thinking guy. He only got the job because he had two Master's Degrees, and someone thought it would be a waste not to place him somewhere.

Posted: **Sat Sep 05, 2015 10:02 pm**

I wonder if you might not be confusing protocols available over a network, with the network itself? When you write network code using PureBasic functions such as CreateNetworkServer(), OpenNetworkConnection(), SendNetworkData() and ReceiveNetworkData(), you are open to ANY of the protocols available on a network. Don't be confused by standards such as HTTP, NTP, FTP or others. They are just a definition of data formats sent over a network.

I think you might be best served by taking small steps, then working your way up. Here are a basic server and client. Run these, see what happens. Then save with name appended with an 'a', or '01', or whatever. Change the code, send more data, send less, send binary. Whatever you want.

When you get more comfortable, we can add code to check on the existence of data on the stack, i.e. is there data available to be read? Later, we can add better timing checks, debugging and error responses.

Rich

Code: Select all

; test_server.pb

EnableExplicit

Procedure ProcessRequest(connectNumber.i)

    Protected result.l
    Protected *memoryLocation
    Protected memLength.i     = 32767

    PrintN("Client " + Str(connectNumber) + " connected")

    *memoryLocation = AllocateMemory(memLength)

; loop until we have received data, or our timeout has been exceeded

    Repeat
        result = ReceiveNetworkData(connectNumber.i, *memoryLocation, memLength.i)
        Select result
        Case -1
            PrintN("Client " + Str(connectNumber) + " received -1")
            Break
        Case 0
            Delay(100); small time delay here
        Default
            PrintN("Client " + Str(connectNumber) + " received " + PeekS(*memoryLocation, result))
        EndSelect
    ForEver

    FreeMemory(*memoryLocation)

EndProcedure

Define NSEvent.i
Define thisClient.i
Define keyPressed.s
Define serverNumber.i
Define portNumber.i = 8901

InitNetwork()

serverNumber = CreateNetworkServer(#PB_Any, portNumber)

OpenConsole("Server socket tester")

PrintN("Started Network Server on port " + Str(portNumber))
PrintN("Press <esc> to exit")

Repeat
    NSEvent = NetworkServerEvent()  ; if we receive data, it will be indicated here

    Select NSEvent
    Case #PB_NetworkEvent_Connect   ; a socket has connected
    Case #PB_NetworkEvent_Data      ; raw data has been received
        thisClient = EventClient()  ; get the event client identifier
        ProcessRequest(thisClient)  ; procedure to process incoming data
    Case #PB_NetworkEvent_Disconnect; a socket disconnected
    Default                         ; no server event occurred
        Delay(100)                  ; sleep so we don't hammer the processor
    EndSelect

    keyPressed = Inkey()
Until keyPressed = #ESC$

CloseNetworkServer(serverNumber)

CloseConsole()

End
; IDE Options = PureBasic 5.31 (Windows - x64)
; ExecutableFormat = Console
; Executable = test_server.exe
; DisableDebugger
; HideErrorLog
; CurrentDirectory = D:\dev\PureBasic\temp\
; CompileSourceDirectory
; EnableCompileCount = 9
; EnableBuildCount = 6
; EnableExeConstant

Code: Select all

; test_client.pb

EnableExplicit

Define i.i
Define result.i
Define lenSendText.i
Define stringText.s
Define portNumber.i = 8901
Define connectNumber.i

OpenConsole()

InitNetwork()

PrintN("Started Network Client connecting to port " + Str(portNumber))
PrintN("")

connectNumber = OpenNetworkConnection("localhost", portNumber)
If connectNumber = 0
    PrintN("Unable to connect to localhost, port " + Str(portNumber))
    End
EndIf

For i=1 To 9
    lenSendText = i
    stringText  = LSet(Str(i), lenSendText, Str(i))

    PrintN(stringText)
    
    result = SendNetworkData(connectNumber, @stringText, lenSendText)
    If result > 0
        Delay(100); small delay here
    Else
        PrintN("Error sending data to the netwok server")
        Break        
    EndIf
Next

CloseNetworkConnection(connectNumber)

PrintN("Network sending complete, Press <enter> to exit")

Input()

CloseConsole()

End
; IDE Options = PureBasic 5.31 (Windows - x64)
; ExecutableFormat = Console
; CursorPosition = 41
; Executable = test_client.exe
; DisableDebugger
; HideErrorLog
; CurrentDirectory = \
; CompileSourceDirectory
; EnableCompileCount = 7
; EnableBuildCount = 4
; EnableExeConstant

Posted: **Sun Sep 06, 2015 7:19 pm**

I will do that. I know starting out means stepping up, but when dealing with the real world, it's pretty much about HTML when visiting various sites. That means doing a lot more than just making a connection. That server expects to be approached with certain requests, and it needs some supporting protocols and capabilities on the client side. If they aren't in place, you are at a dead end.

PureBasic example code does a lot, and builds on a lot, but where are the missing pieces to put it all together? If you need libraries, where are they? If additional API calls are then needed, what are those? How do you include the library and make the calls? What needs to get passed, in what form, and in what order? What do the results tell you?

Go to an existing browser library and use it. What library, where to find it, and what about the documentation needed to work with it? How about example code to begin with? There is a gap in my understanding, and related searches on the internet are not narrowing that gap. Hey, playing games like tic-tac-toe or moving sprites around a dark screen may be your thing, but I want a lot more than that for the time and effort I put into it. I can't even learn if the Blinked-based browsers can be programmed from the client side. I would suspect thar they likely have javascript as a tool, maybe CGI, and who knows what all else, but it's a void, an abyss, when it comes to any mention of the subject, much less getting specific details.

If not found on the internet, how about books on the sybject? O don't know of any, and none of them would be centered around using PureBasic as your means of getting there anyway. Maybe if I want to be a serious programmer, I should go devote myself to C++ or C#, study .Net and other stuff, get into Perl, Python, Ruby, and just drop PureBasic altogether. Is that what you want to tell me?

Hey, I know PureBasic is capable of it. The guys that put it together must be some of the brightest people around. They started with C or C++, adapted and refined it, and made it Basic-like as to syntax. It isn't a blind mystery to them. But I don't see a ladder here for getting beyond about the 5th floor, and HTTP with HTML or XML is up in the clouds. I knew it would be a big challenge going in as far as how high I had to climb, but again: I don't see a ladder.

I've run example code, meaning start the server with a Compile and Run, then did the same for the client. So something gets passed between them. Fine. If you are building your own little network, maybe have two PCs linked with a cross-over cable, this would be the way to go. But in the real world. it's immediately more involved.

"Separating them out" is just conceding defeat. It's like sheet music with no instruments, or the reverse. It's like PureBasic or any other top-layer compiler with no computers (or computers before they had layers of foundation code added to them). Code requiring compilers or interpreters would then only be theoretical in nature. When physicists were certain that nuclear fission and fusion were distinct possibilities, it still took two years and a vast effort by an army of scientists, engineers, and technicians, all at great cost, to produce two working prototypes, which did work when tested. Until then, there was no certainty that the concept was true.

If I am correct, then PureBasic and do HTML and XML without question. But other than a few examples thar skim the surface, where is there evidence to support this claim? My broad assumptions need prototypes as a way of proof, and to serve as a means for me (and others) to move forward. I'm sort of like a theorist, who needs the true scientists, engineers, and technicians to make it really happen, At best I am a high level coder, who adapts existing code to new uses.

Posted: **Mon Sep 07, 2015 8:15 pm**

I believe, once you have networking down, the rest is easy! That's why I emphasize it.

Yes, you're correct, there is a lot more to it, but that doesn't mean you have to immediately jump in and start writing Javascript! Find a simple page, right click on it, then view the source. For instance, one of the domains I own is rca2.com, because I am a 'Junior.' Go to http://rca2.com/, view its source, that's all you need to create a web page!

Go to another page: http://rca2.com/gmt_time.htm and view its source. This page has some Javascript in it.

Then, well, it depends on how much you want to get into it. Go to http://www.w3schools.com/, this will give you some pretty good basics.

Posted: **Wed Sep 09, 2015 10:38 am**

Thank you, thank you, thank you! But need to sleep now. Just trying to make sure my email is read first.

Posted: **Thu Sep 10, 2015 10:16 pm**

There is nothing, and I mean NOTHING that I have run into that I have not been able to accomplish with PureBasic. Now, that being said, there are some libraries I use, Cryptlib for one, that provide me with extra capabilities that I need. That is in regard to SSL to a secure server. I expect that I will replace this with the new 5.4 version of PureBasic when it is released. There are just some things you don't want to do yourself!

Since you don't know me, I will take a moment to explain my background. I have been writing software for more than 30 years. I have working in the retail industry, but now work in Public Safety. Specifically, I write and support software used by Police, Fire and EMS. I am certified by the FBI and multiple states to access the National Crime Information Center. I am certified by the FBI as a provider of Uniform Crime Reporting software, and software for the National Incident Based Reporting infrastructure. I am also certified as a provider of FireHouse Fire Reporting software.

Now, what all does that mean? Well, only that I have to learn a lot of different 'protocols.' But these really aren't protocols, just like the latest XML based Excel reports I am writing are not protocols. They are just data standards. Everyone of these are transported over TCP. TCP is accessed over sockets, just like Telnet is. You most likely know what telnet does, it gives you terminal access to a server. But did you now you can launch a Windows command prompt, and type 'telnet severname,' and get access just like if you launched terminal software? You could also type 'telnet google.com 80,' and connect to Google's server? Nothing would come in, since http requires you to send a 'GET' statement first, but you will connect!

For me, the best way to learn something, is to jump in and do it. I subscribe to the credo 'For everything, there is a first time.' Learning sockets is different from learning TCP or UDP. TCP is error correcting, where UDP is not. Both have their place, and I use both, but primarily I use TCP. There is much more overhead with TCP, but like I said, it's error correcting. There are instances when UDP is useful, but don't worry about it now. What I do when I wanted to learn about a new data format, is to look at the communication going back and forth. One of the easiest way to do this is to go here, and download socket sniff:

http://www.nirsoft.net/utils/socket_sniffer.html

Then, run it to monitor a bowser. Connect to a website, and watch the data co back and forth. You will see the standardization of command data going back and forth. When you learn sockets, and understand the data commands going back and forth, you can write software in PureBasic that can replace a telnet client, or a browser client. Once you get to that point, there is basically nothing you cannot do.

Rich

Posted: **Thu Sep 10, 2015 10:30 pm**

I am still not getting updates from this site, so if you need something, you can email me: my first name @ my PB username.com.

But I do try to browse the site at least once every few days.

Rich

Posted: **Fri Sep 11, 2015 9:13 pm**

Great background, Rich. I began in the Navy back in 1965, first in electronics, invited to switch to computers extra 2 years of service required). learned the hardware (and machine language programming), and gradually moved up. My big thing was collecting and analyzing strings of data.

In the Navy, I moved up to management. Could not program our dedicated systems, so did not get back into programming until the HP-25C RPN and TI SR-52 programmable calculators, then the Commodore PET 2000, the TI-99/4, and the TRS-80 were available. I only used machine language with the PET, which used a 6502 processor. The others were in one dialect of Basic or another, I had learned and used Jovial, Fortran IV, and a bit of COBOL years earlier, but was only proficient in Jovial (Jule's Own Version of an integrated Algorithmic Language). It was developed for the Air Force, but acquired by the Navy for its Fleet Operational Control Centers (NAVCOSSACT, FOCCLANT, and FOCCPAC). You look up Jovial now, the wiki says that Jovial stands for Jules Own Version of the International Algorithmic Language. Hey, I was there, in both NAVCOSSACT and FOCCPAC. I used it almost daily. I'm right, the wiki is wrong.

The CDC 1604A and C-160A computer were the last mainframe computers I programmed, except for two refresher courses I took on Fortran IV and COBOL, where I submitted punched cards that included my code and JCL (Job Control Language) for someone else to run. I just got my cards and a printout back, then I had to figure out what all the numbered errors meant, make new cards uo, and do it again.

I did program terminal-type applications using QuickBasic and PowerBasic to pull data off of several mainframes over a network, which I organized into large databases and ran queries against, looking for abnormal behavior in accumulated network data. Found a lot of problems, and pass this information on to Field Services to go out and correct. It was a real challenge, because terminal behavior is not consistent, meaning the mainframes were not keeping the flow going properly. No problem for human operators who can adjust to what they see on the screen, but plays havoc when a PC is sitting in for that operator. I had to learn ways to tell if something was amiss, then how to get the session back to the place where the PC was, at by telling the mainframe to go in or go back, until I got to the right screen to go on from. It took time, but I finally got it all solved.

I still see that type of problem on the internet at some web sites. Sessions that hang, forcing a reload. The web page supports a button to be pressed, but sometimes just blows right past it. Connections get lost for no apparent reason. Ending up on pages I'm not authorized to visit. Suddenly lose rights to be on the site at all (probably a fumbling of the cookies). Have the web page abruptly close on me, with no clue as to how to get it back (happens on this site when writing a long post sometimes). Or the browser may abruptly close due to a software glitch (too many tabs open at once perhaps). Or the browser suddenly generates an error report home without explaining why. I can hold down the Page Down button to try anf get to the end of a long page, and suddenly see it right back at the top or middle of the same page. I see that a lot at one site I frequent. Humans ignore or don't notice these things. I guess I am just a bit different.

Here is something else I see here a lot. Just got it, so this is a copy-and-paste: "The submitted form was invalid. Try submitting again" Yep. Page down and resubmit. Works the second time around.

Posted: **Wed Sep 16, 2015 3:10 am**

In 1965, I was 3! LOL!

Honestly, I don't see nearly the problems you are describing. Which browser do you use? Try disabling all add ins. I have FiOS, and never have an issue with web sites closing unexpectedly.

Posted: **Wed Sep 16, 2015 10:32 am**

RichAlgeni wrote:Go to http://rca2.com/, view its source, that's all you need to create a web page!

Go to another page: http://rca2.com/gmt_time.htm and view its source. This page has some Javascript in it.

Then, well, it depends on how much you want to get into it. Go to http://www.w3schools.com/, this will give you some pretty good basics.

I've been busy. I worked out enough syntax issues with bash, grep, awk, and ${} string parse features to finally get my shell script do what I want in PBfind, and took time to study javascript a bit, getting the example programs all copied off and put in 48 *.htm documents, then pulled them up in a browser and used gedit to toy with them. I found a discrepancy in an explanation of what document.write() is suppose to do, which is recognise \n as a new line. It doesn't. In fact, it wipes the page contents. Unexpectedly when called again, at least in one case. You have to use <p> </p> to get a new paragraph, or <br> to get a new line. You can write plain text in the <head> </head> and <body> </body> sections without needing document.write() at all. There are some issues about knowing where you are in a page as well, if you intend to refresh any part of it.

The tutorial ended after it covered Print, Objects, Nunbers, Strings, and Arrays. Nothing about URLs, other than to say up front that javascript does not permit file uploads or downloads. That's quite a restriction.

I did some more searching on the forums here for anything related to Linux HTTP connect URL, anf it's a shallow pool. Quite a few responses seemed to have been triggered by just the inclusion of the word Linux. Or maybe the other terms were in there, but further down. I found a source file on my machine that looked promising as it had two lines in the the file which got a match when I searched them with abbreviated terms. One haf a line in it that read NetworkOpen with some arguments after it. A later line had NetworkOpenURL_( followed by arguments.

I could not get the program to compile, came back to the forums, and found out by searching that NetworkOpen is a DLL call, meaning it is specific to Windows, not to the MacOS or Linux. I did unearth a post that had something that was claimed to do the whole thing without needing the DLL, so supposedly it would be cross-platform in nature. It compiles, it asks what host/path/file to download, and it runs. Only no file downloads. The program finishes, but the window does not close. I haven't had time to study it in detail yet.

I can't get my hands around the essentials of opening a HTTP session to connect to a client on port 80, to get the default page, to display it, to honor HTML/javascript/cookie requests, to wait awhile, and if no activity on my part, to time out and close the link. But indications are that javascript is not going to do it for me. Yet HTML has embedded links that get used, only it takes a mouse click to activate them. In settings, you can either get there through the current tab, or open a new tab. I mean I see it happening, but I don't have it down in code where I can understand it and work with it. I don't even have Telnet reduced to code to cast a little light on the matter.

It was suggested I look at a network sniffer program. I'm familiar with the concept of packets, that you have data blocks of a given size that have routing. sequencing, total packet count, protocol, and checksums appended in additional bytes that form a head and tail for each one. A sniffer looks at the heads and tails, gives you a breakdown of the info there, and the data inside is largely ignored. By most people, that is. Hackers may be able to pick userids and passwords out of some of them, and more than that if they really get into it.

What is it that I need to get a handle on to do what I want, which is raise a connection to a URL, meet the HTTP host requirements in a typical browser way, and then disconnect after a time?

Posted: **Fri Sep 18, 2015 10:22 pm**

You may have been going too deep in looking at network protocols. That's why I suggested 'socket sniff,' as it will just show you the data sent back and forth, and not the underlying TCP protocol.

It sounds like you are wanting to create a 'pseudo-browser?' Maybe not with all the bells and whistles of Chrome or Firefox, but one that will allow you to see traffic?

The code I will share is from Windows, it's what I have standardized on, but there is code similar for Linux. Not sure about Mac.

You can probably open a socket without too much trouble, right? Here is some code to check a receive buffer for incoming data, but the first thing I like to do with a client socket it to maximize the internal buffer size. So I call this procedure, for a bufferSize of 65535:

Code: Select all

; **********************************************************************************
; this procedure set the TCP receive buffer size, and keepalive
; **********************************************************************************

Procedure.i SetTCPBufferSize(connectNumber.i, bufferSize.i)

    Protected result.l       = 0; winsock utlizes a 32 bit integer only!
    Protected keepAlive.i    = 1
    Protected socketHandle.i = 0

    socketHandle = ConnectionID(connectNumber)

    result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_RCVBUF,    @bufferSize, SizeOf(bufferSize))
    result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_KEEPALIVE, @keepAlive,  SizeOf(keepAlive))

    ProcedureReturn result

EndProcedure

You've probably noticed the statement socketHandle = ConnectionID(connectNumber), this is needed to call the Windows API statement result = ioctlsocket_(socketHandle, #FIONREAD, @length). The constant #FIONREAD is a command to check the readbuffer. See below.

Next, you will create and send your 'GET' or 'POST' statement, ending with two successive carriage return/line feed pairs.

Then, you need to check for data returned. Ideally, you don't want this to block, you don't want it to consume too many resources, and you don't want it to wait forever, right? So call this procedure:

Code: Select all

; **********************************************************************************
; this procedure reads binary data from a network connection in one read attempt
; **********************************************************************************

Procedure.i NetworkRead(connectNumber.i, *memoryLoc, memLength.i, socketTimeout.i, maxReadTrys.i)

    Protected socketHandle.i = 0
    Protected attemptCount.i = 0
    Protected amountRead.i   = 0
    Protected errNumber.i    = 0
    Protected length.i       = 0
    Protected result.l       = 0; winsock utlizes a 32 bit integer only!

    socketHandle = ConnectionID(connectNumber)

; loop until we have received data, or our timeout has been exceeded

    Repeat
        result = ioctlsocket_(socketHandle, #FIONREAD, @length)

; if we encountered a socket error, we need to exit the loop

        If result <> 0
            errNumber  = WSAGetLastError_()
            amountRead = -1
            Break
        EndIf

; if nothing has been received, increment the attempt counter and sleep, else get out of the loop

        If length = 0
            attemptCount = attemptCount + 1
            If attemptCount > maxReadTrys
                Break
            Else
                Delay(socketTimeout)
            EndIf

; else get the data in the network receive buffer

        Else
            amountRead = ReceiveNetworkData(connectNumber, *memoryLoc, memLength)

            Break
        EndIf
    ForEver

; now return the amount we have received, if anything

    ProcedureReturn amountRead

EndProcedure

You'll pass in the socket (connectNumber), a pointer to a memory address with a size of memLength + 1, a timeout length in thousandth's of a second, and a maximum number of retries. I usually set the last two variables to a combination of 2 to 5 seconds, with a timeout of 100, and retries of 20 to 50.

The API statement of result = ioctlsocket_(socketHandle, #FIONREAD, @length) is used to determine the amount of data in the input buffer. If result returned is not 0, there is a socket error, and you should exit the procedure. Pass in a pointer to the integer variable length, when the API procedure returns, and length is greater than 0, read the socket as you would normally. Otherwise, call Delay, then increment the attemptCount variable. If the attemptCount variable exceeds the maxReadTrys, exit the procedure with nothing read.

For the purposes of this example, errNumber is not really used. Regardless, if result is not 0, a socket error occurred.

Now, the NetworkRead procedure can be called as often as you want, until you have all the data you need, or the socket terminates, either normally, or with an error. When a client socket terminates, ALWAYS call CloseNetworkConnection(connectNumber).

That should get you started, but here is my procedure for sending data:

Code: Select all

; **********************************************************************************
; this procedure writes data to a socket, can be used with TCP or UDP!
; **********************************************************************************

Procedure.i NetworkWrite(connectNumber.i, *memoryLoc, lenSendText.i, *netSource, debugThis.i)

    Protected totalSent.i = 0
    Protected errNumber.i = 0
    Protected lenToSend.i
    Protected result.i
    Protected length.i

; try to send the data, loop until all done, or error encountered

    Repeat
        If lenSendText > maxSendSize; make sure our packet size is not too large
            lenToSend  = maxSendSize
        Else
            lenToSend  = lenSendText
        EndIf

        result = SendNetworkData(connectNumber, *memoryLoc + totalSent, lenToSend)

; if we've received an error, set the condition to get out of the loop

        If result < 0
            totalSent   = result
            lenSendText = 0
            errNumber   = WSAGetLastError_()
            Break
        Else
            totalSent   = totalSent   + result
            lenSendText = lenSendText - result
            If lenSendText = 0
                Break
            EndIf
        EndIf
    Forever

    ProcedureReturn totalSent

EndProcedure

It's pretty straightforward. Sometimes, we have an tremendous amount of data to send, so we break it into packets of 32k or 64k, then send each packet.

There's more we can do, but digest this first!

Rich

Posted: **Sat Sep 19, 2015 2:47 am**

Thanks! Examples really make a difference. I'm doing a copy and paste. compiling each, and see how far it gets as I go. I encountered an error in the first code out, that the compiler did not recognize #SOL_SOCKET.

You put it well. A pseudo -browser is indeed what I had in mind. In my career with computers, it was always preferable to automate as many tasks as possible, to not only reduce the tedium of many jobs, but give users the time to get more things done. I resisted the move to a GUI environment like Windows because it effectively made a tighter integration of man-to-machine,
meaning more demand for user involvement and less applied automation in terms of pre-programmed tasking.

Seems contradictory in nature, but we actually get less back in terms of offloading onto the computer, while the complexity of doing what we do this has gone up. More software to master, more cryptic the interface, the more hotkeys involved, and so on. For some proof of this, look at the change that came when Microsoft introduced the "ribbon" in place of the drop menu. It added no real benefit, it required retraining, and it was all about making a dashing impression to increase sales. In fact, the Ubuntu counterpart in fighting this trend and satisfying a new user demand is named "Dash", though it is not as opulent or glittery.

A true browser is again about man-with-machine involvement. i want to reduce the "man" aspect, and have the machine do more on a cyclic bases. You could run pseudo-browser and get needful things done while you could be on a browser pursuing other interests.

Something that came and went was session recording that resulted in a script file of user actions in response to web page presentations. You click on something or type something, it became part of that script. But there were many things that had to be deduced from a review of the script, such as what did a left-mouse-button click at x,y coordinates really signify? What was at that coordinate? Them there was a paste done over at this coordinate, but what was in the clipboard at that time? Where did it come from? Oh, I know what it probably was. The current date was injected at that point, How do I get the script to acquire and apply the current date? How do I reduce a script recording to a minimal list of steps to just get the job done, while eliminating all the non-relevant human interactions that are occuring as well? How do I make determinations in code, particularly script code, that reflect judgment calls based on what is displayed on the screen?

It was simply simpler and more direct to program the actions and responses in a programming language where the user and programmer detailed what needed to be done, the how, and working out the why of it all in a top-down process. But without the programmer's involvement, it was left for the user to do it all through the MMI (Man-Machine Interface), which is often the GUI (Graphical User Interface). And that begins at what we commonly refer to as the "Desktop". So where do you go from there? Do I have an icon that I can click on? If no icon, how do I find the program? What's its name? Where is it stored? How does it work? Then what?

After the first hundred repetitions, the answers are mapped out in your head, and now it is a repeat chore. Anybody could do it (if they knew what you knew). Some of it is so routine that it becomes both tedious and boring (you learn the subtle differences in those terms). How do you get the computer to do more on its own?

Well, the choice is simple: Either program it or get it programmed for you. Either do that, or continue what you have been doing, hire and train someone to do it for you, or quit doing it altogether. Hiring someone else is dangerous, because they threaten the need for keeping you in the picture at all. Beside, that's costly. So is hiring a programmer. That takes you down to just three choices:

(1) Keep on going as you are
(2) Program it yourself (but with added assistance via forums)
(3) Drop it altogether

I'm trying to merge the first two. If I fail, I will be reduced to (3).

PureBasic Forums - English

More PB When Dealing With Network Protocols

More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols

Re: More PB When Dealing With Network Protocols