TCP Help Please

wooly · Post by **wooly** » Mon Aug 04, 2008 9:40 pm

Hi, I am starting to programme again after a long absence (Thank-you retirement

)

I have programmed a PIC micro controller to read several sensors around the house (temperature, humidity, voltage, current etc). The PIC than massages this date, does a little bit of magic and then spits out a 60 character string from its serial port every 60 seconds to an EM202 serial to Ethernet converter.

I can connect to the converter using putty or other telnet clients and "see" all the 60 character strings being sent once a minute. So at least that part is working.

Where the problem lies is in getting Purebasic to receive the data and save it to a file. The programme I have written captures part of the 60 character string, some times 3 or 4 characters usually around 30 and on very rare occasions all 60, there seems to be no pattern. There is no corrupt data, just missing data. The used is below (don't laugh too loud)

Code: Select all

If InitNetwork() = 0 
   MessageRequester("Error", "Unable to initialize network") 
EndIf 



ConnectionID = OpenNetworkConnection("192.168.1.100", 1001)
While 1=1
	*Buffer = AllocateMemory(60)

	result=ReceiveNetworkData(ConnectionID, *Buffer, 60) ;Result = # of bytes read 
	CloseNetworkConnection(ConnectionID)

	Debug "Number of Characters"
	Debug result


  	OpenFile(0, "D:\Test.txt")    ; opens file to save data in
    	FileSeek(0, Lof(0))           ; Get end of file position
    	WriteData(0,*Buffer ,60)      ; Append data		
    	FreeMemory(*Buffer)	      ; Release memory
    	CloseFile(0)		      ; Close file
        Delay(60000)		      ; wait 60 seconds
 Wend

So, is the code anyway near what I need?
Do I have to keep opening and closing the network connection or can I keep it open and just listen for new data, if so clues as to ho to do tis would be welcome!!

Thanks

Wooly

Derek · Post by **Derek** » Mon Aug 04, 2008 9:57 pm

I don't think the delay() is helping here, have you tried shortening it quite a bit and see what happens.

Also, you could try using NetworkClientEvent() to check when data is available and then read it.

When you recieve less than 60 bytes of data is it from the end of the transmitted data, from the start or just random?

wooly · Post by **wooly** » Tue Aug 05, 2008 3:27 am

Thanks for the response Derek, when I receive data it is always from the beginning of the string i.e the string is truncated. I tried the NetworkClientEvent() as suggested, and if I have implemented it correctly the debug always shows no data available

Code: Select all

If InitNetwork() = 0 
   MessageRequester("Error", "Unable to initialize network") 
EndIf 



While 1=1         ;Setup Endless loop for testing

ConnectionID = OpenNetworkConnection("192.168.1.100", 1001)  ;Open a connection To the EM202
IsThereAnything = NetworkClientEvent(ConnectionID)           ;Check if there is Data to receive

Debug "Anything There"
Debug IsThereAnything

If IsThereAnything >0                                   ;If there is a string go get it

 
   *Buffer = AllocateMemory(60) 

   result=ReceiveNetworkData(ConnectionID, *Buffer, 60)     ;Result = # of bytes Read 
    

   Debug "Number of Characters" 
   Debug result 


     OpenFile(0, "D:\NewTest.txt")                          ; opens file to save data in 
       FileSeek(0, Lof(0))                                  ; Get end of file position 
       WriteData(0,*Buffer ,60)                             ; Append data       
       FreeMemory(*Buffer)                                  ; Release memory 
       CloseFile(0)                                         ; Close file 
                                                 
 EndIf
 CloseNetworkConnection(ConnectionID)
 Delay(1000)                                                ; wait 1 second
 Wend

I changed the delay in increments down to 1 second with no noticeable change. I also took it to 70 seconds, but still getting truncated strings.

Wooly

netmaestro · Post by **netmaestro** » Tue Aug 05, 2008 3:45 am

Look at the NetworkServer.pb example program in the help file under "Network" in the general libraries section, it should help you a lot. Also read carefully the full text of the help for ReceiveNetworkData() for an understanding of how it works.

Post by **idle** » Tue Aug 05, 2008 8:37 am

Sent you a pm with a snippet, I'm over tired so maybe errors

wooly · Post by **wooly** » Tue Aug 05, 2008 8:14 pm

Thanks for the pointers idle. I spotted your 2 deliberate errors, no doubt designed to ensure I had a vague idea of what was going on

Anyway, there has been a marked improvement in received data, I can guarantee every second string I receive is correct and complete. In reality this means I can continue with the project. In practice it means this is going to bug me until it is resolved!
Below i a sample of the received data, those stings between {} are correct ones.

Code: Select all

{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{01234
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefgh
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789ab
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdef
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{01234
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{012345
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{012345}
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijkl
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefg
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghi
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{012345678
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcde
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefgh
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{01234
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789abcdefghijklmnopqr
{0123456789abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKL}
{0123456789a

As you can see the others are truncated randomly, also making random appearances are groups of three correct strings

I have played with buffer lengths and delays but all to no avail.

Thanks again for the info,
Off to read more books

Wooly

TerryHough · Post by **TerryHough** » Tue Aug 05, 2008 9:29 pm

Since the most recent code you are using wasn't posted, it is hard to find your problem.

Now way to test this, but this would be my starting point.

Code: Select all

If InitNetwork()
  ConnectionID = OpenNetworkConnection("192.168.1.100", 1001)  ;Open a connection to the EM202
  If ConnectionID
    *Buffer = AllocateMemory(60)
    OpenFile(0, "D:\NewTest.txt")                        ; Opens file to save data in
    FileSeek(0, Lof(0))                                  ; Get end of file position
    Repeat
      Event = NetworkClientEvent(ConnectionID)           ; Check if there is data to receive
      Select Event
      Case #PB_NetworkEvent_Data
        result = ReceiveNetworkData(ConnectionID, *Buffer, 60)     ;Result = # of bytes Read
        WriteData(0,*Buffer ,result)                     ; Append data
      Case #PB_NetworkEvent_File                         ; not applicable in your case
      Default
        Delay(1000)                                      ; wait 1 second
        ; Need a way to break out here... if no data received in ??? minutes 
        ; Break                                           ; Break out of the repeat loop      
      EndSelect 
    ForEver 
    FreeMemory(*Buffer)                                  ; Release memory
    CloseFile(0)                                         ; Close file
    CloseNetworkConnection(ConnectionID)
  Else
    MessageRequester("Read EM202", "Unable to create a network connection.",#MB_ICONERROR)
  EndIf
Else
  MessageRequester("Read EM202", "Unable to initialize TCB network access.",#MB_ICONERROR)
EndIf
End

wooly · Post by **wooly** » Tue Aug 05, 2008 10:28 pm

Hi Terry, I did not post the last code I used as "Idle" sent it to me in a PM. I am not sure whether he wanted it released or not. If he has no probs I'll post it.

Thanks for your input, so far it is running OK and seems to be handling complete strings properly. I'll play around some more this evening.

Wooly

Post by **idle** » Tue Aug 05, 2008 10:56 pm

I was tired, so I expect there'd be errors, I wouldn't see them, I suppose if I hit compile I may have spotted them. So no wasn't on purpose.

just a case of being very dyslexic.

wooly · Post by **wooly** » Thu Aug 07, 2008 2:44 pm

Thanks to all for your help, with code samples from both Idle and Terry I have managed to capture the data I need. More importantly though, I now have a much better idea of coding in PureBasic

Wooly

TerryHough · Post by **TerryHough** » Thu Aug 07, 2008 3:43 pm

Glad I could help.

I would be interested in seeing your final code if you don't mind. You can PM it to me if your prefer.

Terry

EdzUp[SD] · Post by **EdzUp[SD]** » Thu Aug 07, 2008 10:20 pm

One thing to take into consideration (one thing that got me too) is when using loops (repeat..until, while..wend) its always good to have a Delay( 1 ) somewhere then the application will not use 100% cpu usage this releases time for networking etc.

Thalius · Post by **Thalius** » Fri Aug 08, 2008 8:16 am

... unless your processing something.

Code: Select all

while ReceiveNetworkData() ...

Thalius

wooly · Post by **wooly** » Fri Aug 08, 2008 8:17 pm

TerryHough wrote:Glad I could help.

I would be interested in seeing your final code if you don't mind. You can PM it to me if your prefer.

Terry

Terry attached is the code to date, nothing earth shatteringly different. I open and close the data file every time new data is entered so that it resides in the file should the system crash.

Code: Select all

If InitNetwork() 
  ConnectionID = OpenNetworkConnection("192.168.1.100", 1001)     ; Open a connection to the EM202 
  If ConnectionID 
    *Buffer = AllocateMemory(8192)                                ; Use overly large array to capture old data if programme is restarted
      Repeat 
      Event = NetworkClientEvent(ConnectionID)                    ; Check if there is data to receive 0=NO 2=YES
      Debug "EVENT"
      Debug event
        If Event = 2
          OpenFile(0, "D:\NewTest3.txt")                          ; Opens file to save data in 
          FileSeek(0, Lof(0))                                     ; Position to EOF
          result = ReceiveNetworkData(ConnectionID, *Buffer, 8192); Result = # of bytes Read 
          WriteData(0,*Buffer ,result)                            ; Append data 
          CloseFile(0)
        Else                                                      ; Future data manipulation will take place here
          No_Data=No_data+1
          If No_Data=1                                           
            Time$ = FormatDate("%hh:%ii:%ss", Date())             ; Reset the time on first no data loop
          EndIf
          If No_Data =60
            MessageRequester("Data Loss at: " + Time$, "No Data Received for 5 minutes")
          EndIf   
          Delay(5000)                                           
        EndIf 
      ForEver 
    FreeMemory(*Buffer)                                           ; Release memory 
    CloseFile(0)                                                  ; Close file 
    CloseNetworkConnection(ConnectionID) 
  Else 
    MessageRequester("Read EM202", "Unable to create a network connection.",#MB_ICONERROR) 
  EndIf 
Else 
  MessageRequester("Read EM202", "Unable to initialize TCP network access.",#MB_ICONERROR) 
EndIf 
End

Wooly

TerryHough · Post by **TerryHough** » Sat Aug 09, 2008 3:18 am

wooly wrote: I open and close the data file every time new data is entered so that it resides in the file should the system crash.

Glad you have it working for you. Just some suggestions.

I refer you to the FileBuffersSize(#File, 0)
command that causes immediate writing to disk (or its companion FlushFileBuffers).

Opening and closing the file is very time consuming. While you are under no time constraints in your application, in other more data intensive systems it could cause problems.

You could leave your Buffer set to the desired 60 characters. You loop will just process as many blocks of 60 characters as have been sent when it starts. Or if you set it larger, I suggest it be a multiple of 60 characters for your application.