Reading HTTP-Chat stream for ChatBot

gas01ine · Post by **gas01ine** » Sat Feb 25, 2006 2:27 am

hi! is there any possibility to read a stream from a browserbased chat (e.g. www.chat.at)? i am kinda new with pb but want to give it a try - but i have no idea how to start...

my chatterbot needs company

thanks in advance!

Thalius · Post by **Thalius** » Sat Feb 25, 2006 7:10 pm

hihi =)

Of course its possible, however you would have prolly to write your own HTTP_GET Routines ( or fiddle around from some examples ). To read the stream from such a HTTP Server you need a to esatblish a "KEEP-ALIVE" connection and parse each incoming line. And to write a http_submit routine for the response input ( this you prolly could find using search and or browsing thru the various code collections.

always a good start is the Codearchive on purearena.net:
http://www.purearea.net/pb/CodeArchiv/

Thalius

gas01ine · Post by **gas01ine** » Sat Feb 25, 2006 7:36 pm

thanks! i just hope i understood what you suggested :roll:

has anybody done this before and provide me maybe some source code?
in the meantime i see what i can figure out myself.

to get the stream I am thinkin about an HTTP GET request or something (URL or POST) am I right?

Thalius · Post by **Thalius** » Sat Feb 25, 2006 8:40 pm

yes you will need GET and POST tho GET as "Keep-Alive" for a chat.

Http operates on what is called a request-response paradigm. This means that a _client_ generates a request for information, and passes it to the server, which answers it. In the original implementation of HTTP, each request created a new socket connection to the server, sent the request, then read from that connection to get the response.

This approach had one big advantage - it was simple. Simple to describe, simple to understand, and simple to code. It also had one big disadvantage - it was slow. So, keep-alive connections were invented for HTTP.

Id suggest you check out how a HTTP Header is composed.
To make a keep alive connection theres a simple switch:

Connection: Keep-Alive

You may want to read a bit into here:
http://en.wikipedia.org/wiki/HTTP

Some Part about building a header stolen from Num3 s code =)

Code: Select all

    com$="HEAD "+url$+" HTTP/1.1"+Chr(13)+Chr(10)
    com$+"Accept: */*"+Chr(13)+Chr(10)
    com$+"Host: "+host$+Chr(13)+Chr(10)
    com$+"User-Agent: PureDownload 1.0"+Chr(13)+Chr(10)
    If proxy$<>""
      com$+"Proxy-Authorization: Basic "+penc$+Chr(13)+Chr(10)
    EndIf

And here you find some HTTP GET Routines:
viewtopic.php?t=11095&highlight=httpget

Thalius

gas01ine · Post by **gas01ine** » Sat Feb 25, 2006 10:37 pm

Thank you for the fast response. In the last hours I made myself very familiar with the HTTP protocol and tried something to pass a line to the chat-server while being logged in via FireFox. But it doesn't work - i get no response in the chat in firefox. But if i use the very same statement (http://195.58.165.211/chatin?SID=241561 ... 1&OUT=TEST!) with the http://web-sniffer.net/ i _do_ get a response in the chat, just as if i typed it in there. any ideas?

oh, and a further question: does anyone know what the SID and ID are composed from? they change every time I log in.

Thank you for all the replies I got so far!

Code: Select all

 InitNetwork()
port = 80
server$ = "195.58.165.211"
eol$ = Chr(13)+Chr(10)
chatmessage$ = "TEST!"

ConnectionID = OpenNetworkConnection(server$, port)
  If ConnectionID
    SendNetworkString(ConnectionID, "GET /chatin?SID=241561026&ID=1898464651&OUT="+chatmessage$+" HTTP/1.0"+eol$)
    SendNetworkString(ConnectionID, "Connection: Keep-Alive"+eol$)
    SendNetworkString(ConnectionID, eol$)
  EndIf
end

Thalius · Post by **Thalius** » Sun Feb 26, 2006 12:31 am

SID is a so called Session ID which is consited often by a random number combination. Typical infos as user password and logintime are stored in such a session on the serverside. The key you get in the Url is just a "identifier" for your session. I Would assume ID is something similar or simply a user ID which then again is linked via the session to the user in the backend DB ( there are tons of ways to do this ).

However seeing from your example: to get a result like this i do not think a KEEP-Alive connection will work here.

See one thing is the downstream ( using keep-alive ) from the server ( typically displayed in a frame or similar ). The other thing is the upstream which typically POST is used as method which is NOT Kept-Alive.

ps. looking at chat.at just confirmed this .. they use two frames chat_in , chat_out

so to READ a response you need to Keep-Alive the chat_out stream while using POST on chat_in. Of course you will have to submit their variables aswell and make sure ( figure out ) what keeps their Session running ( basically handle sessions correctly ).

Hope this helps you further.. =)

Thalius

gas01ine · Post by **gas01ine** » Sun Feb 26, 2006 9:25 am

yes, it helps thanks. i coded a big deal during the night and am also able to read from the server.

on closer inspection i found out that the upstream is a GET command. i am also able now to read the correct SID and ID for each login session directly from the server.

but i am still having troubles: when i connect with my browser to the chat NETSTAT says that it is on "ESABLISHED", obviously via a "keep-alive" header, while when connecting with my app is always on "CLOSE_WAIT", which would explain why I don't get anything back from the chat... any ideas?

if i just had known about the complexity of such an simple task in the first place i probably wooulnd't have started - lol :roll:

only if you're interested in some messy code:

Code: Select all

[size=9];
;
;       Connect to Chat.at and chat!
;
If InitNetwork() = 0
  MessageRequester("Error", "Can't initialize the network !", 0)
  End
EndIf

Global port
port = 80

Global server$
server$ = "195.58.165.211"

Global sid$

Global id$

Global eol
eol$ = Chr(13)+Chr(10)

Global postfix$
postfix$ = " HTTP/1.0"+eol$
bufferLength=12040

Global nickname$      
nickname$ = "PureBasic"

Global http$
http$ = " HTTP/1.1"

Global pw$
pw$ = ""      

Global channel$
channel$ = "Foyer"

Procedure.s URLGet(url$)
   get.s = "GET "+url$+http$ + Chr(13) + Chr(10)
      If http$ = " HTTP/1.1"
         get + "Host :  "+server$+":"+Str(port)+Chr(13)+Chr(10)
      EndIf
      get + "Accept : */*" + Chr(13) + Chr(10)
     ; get + "Connection: Stay-Alive"+ Chr(13) + Chr(10)
      get + Chr(13) + Chr(10)
ProcedureReturn get
EndProcedure 

Procedure.s ChatOut()
   get.s = "GET http://"+server$+"/chatout?NICKNAME="+nickname$+"&PW="+pw$+"&CC=&SC=&CHANNEL="+channel$+"&PROVIDER=CHATAT&KEY=258&LI=12&BL=2&"+id$+"&"+sid$+"PI=0&MODE=1048576"+http$ + Chr(13) + Chr(10)
    If http$ = " HTTP/1.1"
        get + "Host :  "+server$+":"+Str(port)+Chr(13)+Chr(10)
    EndIf 
      get + "Accept : */*" + Chr(13) + Chr(10)
     ; get + "Connection: Stay-Alive"+ Chr(13) + Chr(10)
      get + Chr(13) + Chr(10)
ProcedureReturn get
EndProcedure 

Procedure.s ChatSay(say$)
get.s = "GET http://"+server$+"/chatin?"+sid$+"&"+id$+"&OUT="+say$+" HTTP/1.0"+ Chr(13) + Chr(10)
get + "Accept : */*" + Chr(13) + Chr(10)
;get + "Connection: Stay-Alive"+ Chr(13) + Chr(10)
get + Chr(13) + Chr(10)
ProcedureReturn get
EndProcedure

Procedure findSid(html.s)
        Repeat
        k = k + 1
        SID$ = StringField(html, k, "&")

        Until FindString(SID$, "SID=",1)
        k = 0
        
        Repeat
        k = k + 1
        ID$ = StringField(html, k, "&")
        Until FindString(ID$, "ID=",1)
        k = 0
EndProcedure
      
command.s = URLget("/chatstart?PROVIDER=CHATAT&KEY=258&MODE=1048576&NICKNAME="+Nickname$+"&PW="+pw$+"&CHANNEL="+channel$+"&FRAME=startframe&OUT=ja")

ConnectionID = OpenNetworkConnection(server$, port)

      If ConnectionID <> 0
          *Buffer = AllocateMemory(bufferLength)
          SendNetworkData(ConnectionID, @command, Len(command))
          ok = 0
          Repeat
           res = NetworkClientEvent(ConnectionID)
            Select res
              Case 0
              Case 2
                ReceivedData.s = ""
                ReceivedDataLength = 0
                Repeat
                  Result = ReceiveNetworkData(ConnectionID,*Buffer, BufferLength)
                  ReceivedData + PeekS(*Buffer)
                  ReceivedDataLength + Result
                Until Result < BufferLength
                ok = 1
              Case 3
               EndSelect
          Until ok = 1
          MessageBox_(0, "Received " + Str(ReceivedDataLength) + " bytes" + Chr(13) + Chr(10) + Chr(13) + Chr(10) + ReceivedData, "pb", #MB_ICONINFORMATION)
          
          findSid(ReceivedData)
          MessageRequester("SID & ID",sid$+ Chr(13) + Chr(10) + id$)
          SendNetworkData(ConnectionID, @ChatSay("Hello!"), Len(ChatSay("Hello!")))
          
          
          Debug ChatSay("Hello!")
          
          
     If ConnectionID <> 0
               SendNetworkString(ConnectionID, ChatSay("hi"))
          
          
          FreeMemory(*Buffer)
          *Buffer = AllocateMemory(bufferLength)
          
          ok = 0
          Repeat
           res = NetworkClientEvent(ConnectionID)
            Select res
              Case 0
              Case 2
                ReceivedData.s = ""
                ReceivedDataLength = 0
                Repeat
                  Result = ReceiveNetworkData(ConnectionID,@chatout(), BufferLength)
                  ReceivedData + PeekS(*Buffer)
                  ReceivedDataLength + Result
                  If FindString(ReceivedData, "999",1) And FindString(ReceivedData, "Fehler",1)
                    Debug "Fehler 999"
                    End
                  EndIf
                  
                Until Result < BufferLength
                ok = 1
              Case 3
               EndSelect
          Until ok = 1
       
       MessageBox_(0, "Received " + Str(ReceivedDataLength) + " bytes" + Chr(13) + Chr(10) + Chr(13) + Chr(10) + ReceivedData, "ok :", #MB_ICONINFORMATION)
        
      EndIf 
  EndIf[/size]