atomic web server threads

Share your advanced PureBasic knowledge/code with the community.
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

There's a mutex for the server thread to protect posting messages to the windows queue it's low contention and may or may not be necessary.
Then there's a client thread mutex to protect fetching from the job queue and also for sending and receiving this mutex is low contention but necessary to protect against connection reuse and then you have a mutex in the TLS.pbi which globally locks the sends and receives.

The complexity is partially due to error handling, for instance ReceiveNetworkData states
Returns the number of bytes received. If 'Result' is equal to DataBufferLength then more data is available to be read. If an error occurred on the connection (link broken, connection close by the server etc...) 'Result' will be -1.

but a socket will return 0 or n bytes read or -1 on error but I don't know if ReceiveNetworkData returns 0 or treats that as a -1
If you get a 0 you would know that a connection is dropped and you could stop polling until your timeout, if you get a -1
99% of the time its a wouldblock which means please wait and try again my tcp buffer is full, there are 7 recoverable errors that should be checked to see if you should continue the operation and the 20 or so other errors are to abort.

ReceiveNetworkData should wrap wsagetlasterror and errno and filter the errors, then it would behave like it's documented but at the moment is doesn't and I still don't know if it returns 0 at all.

So to fix Receive and Send NetworkData for TCP it should test for the below and delay and try again to either return n bytes read or -1 to abort

Code: Select all

  #WSA_IO_INCOMPLETE = 996
  #WSA_IO_PENDING = 997
  #WSAEINTR = 10004
  #WSAEMFILE = 10024
  #WSAEWOULDBLOCK = 10035
  #WSAEINPROGRESS = 10036
  #WSAEALREADY = 10037
  ;for libretls 
  #_WANT_POLLIN  = -2
  #_WANT_POLLOUT = -3


or just add a function to check if the error is recoverable so we can set the delay and try again. though I think it'd be better to wrap it.
Quin
Addict
Addict
Posts: 1122
Joined: Thu Mar 31, 2022 7:03 pm
Location: Colorado, United States
Contact:

Re: atomic web server threads

Post by Quin »

idle wrote: Sat Aug 03, 2024 1:57 am but a socket will return 0 or n bytes read or -1 on error but I don't know if ReceiveNetworkData returns 0 or treats that as a -1
If you get a 0 you would know that a connection is dropped and you could stop polling until your timeout, if you get a -1
99% of the time its a wouldblock which means please wait and try again my tcp buffer is full, there are 7 recoverable errors that should be checked to see if you should continue the operation and the 20 or so other errors are to abort.

ReceiveNetworkData should wrap wsagetlasterror and errno and filter the errors, then it would behave like it's documented but at the moment is doesn't and I still don't know if it returns 0 at all.

So to fix Receive and Send NetworkData for TCP it should test for the below and delay and try again to either return n bytes read or -1 to abort

Code: Select all

  #WSA_IO_INCOMPLETE = 996
  #WSA_IO_PENDING = 997
  #WSAEINTR = 10004
  #WSAEMFILE = 10024
  #WSAEWOULDBLOCK = 10035
  #WSAEINPROGRESS = 10036
  #WSAEALREADY = 10037
  ;for libretls 
  #_WANT_POLLIN  = -2
  #_WANT_POLLOUT = -3


or just add a function to check if the error is recoverable so we can set the delay and try again. though I think it'd be better to wrap it.
Should probably be moved to feature requests and wishlists, but I completely agree with this. Trying to write networking applications in PB that use large packets is incredibly annoying for exactly this reason.
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

Quin wrote: Sat Aug 03, 2024 2:23 am Should probably be moved to feature requests and wishlists, but I completely agree with this. Trying to write networking applications in PB that use large packets is incredibly annoying for exactly this reason.
Yes its a little frustrating. Fred will be reading this so I think he can make up his own mind.
It's been a long standing issue and needs addressing in the implementation to match the documentation.
and it should also set a last error.
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

Changed license to Eclipse V2
This is still permissive you can link to to statically and use it You can still use it commercially, it just means any changes to the library need to be shared.
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

version 3.1.0b12 PB 6.12 reverse proxy working needs the supplied dlls
version 3.1.1b1 PB 6.20 reverse proxy broken but doesn't need supplied the dll.s

changed uri handlers so you can catch paths like

tiles/z/x/y.mvt from a handler set with tiles/

Code: Select all

Procedure tiles(*request.Atomic_Server_Request)  
   Protected pos,*data,url.s,result,StateOK    
   Protected *Atomic_Server.Atomic_Server = *request\Serverid  
   
   pos  = FindString(*request\RequestedFile,"/tiles/") 
   If pos 
     url = "https://api.protomaps.com/" + PeekS(@*request\RequestedFile+pos<<1,-1) + "?key=abcdef123"  ;your key
     Debug url     
     Repeat 
     result = HTTPRequestMemory(#PB_HTTP_Get,url,0,0) 
     If result
       HTTPTimeout(result,10000)
       StateOK = 200 
       If HTTPInfo(result,#PB_HTTP_StatusCode) = "200"   
       
        *Data = HTTPMemory(result)
        *request\status = 200 
        *request\ContentType = *Atomic_server\MimeTypes("mvt")  ;Set the contentType
        FinishHTTP(result) 
            
        ProcedureReturn *data 
      Else 
        Debug HTTPInfo(result,#PB_HTTP_ErrorMessage) 
        FinishHTTP(result) 
       Break
      EndIf  
        
    EndIf
    Delay(10)
    Until StateOK 
    
   EndIf 
   
EndProcedure   
  
server1 = Atomic_Server_Init(title,"./www/",#Server_IP,"atomicwebserver.com",443,#PB_Network_IPv4,1000,0,0)  
Atomic_Server_Init_TLS(server1,"./certs/","atomicwebserver.com","certificate.crt","private.key","ca_bundle.crt")   
Atomic_Server_Add_Handler(server1,"atomicwebserver.com/tiles",@tiles())

OpenConsole()
Atomic_Server_start(server1,1,1) 
Repeat 
Until Input() = "quit" 
Atomic_Server_Exit(server1)
CloseConsole() 
the handler in this case substitutes the call with your key to the tile server so it's not visible in the JS to the user all they see is key=0 and you fetch the tiles from the cloud and route through the server.

I don't know when the multiple cert loading will get added to PB6.21 maybe but till then if you need to host multiple TLS domains on the same machine you will need to use version 3.1.0b12 PB 6.12 Atomic_web_Server3.pbi
User avatar
skinkairewalker
Enthusiast
Enthusiast
Posts: 772
Joined: Fri Dec 04, 2015 9:26 pm

Re: atomic web server threads

Post by skinkairewalker »

hi how u doing idle ?
atomic web server have support to linux and mac too ?
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

skinkairewalker wrote: Mon Feb 24, 2025 4:01 am hi how u doing idle ?
atomic web server have support to linux and mac too ?
I haven't had the time to test the changes on Linux, so it could be broken but in theory it should also work for OSX if you use Aomic_web_server_3.1.pbi which is for PB 6.20, it uses the PB TLS implementation but it can't do terminating TLS reverse proxy. Fred missed that part out in the implementation. It's specifically so you can host multiple domains on the same machine or on some other machine on the LAN and reach them via the TLS proxy on port 443

All it requires is to process a list of certs when creating the server that's acting as the proxy with tls_config_add_keypair_mem load the main domain and then loop through adding the proxied domains
I can probably get it to load additional certs after the servers started but haven't tried yet.

how to use

Code: Select all

#Server_IP = "0.0.0.0" 
;this assumes you have an ssl cert 
;dirs   www/yourdomain.com/index.html
;         certs/yourdomainr.com/certs 
XIncludeFile "Atomic_Web_Server3.1.pbi"
server1 = Atomic_Server_Init(title,"./www/",#Server_IP,"yourdomain.com",443,#PB_Network_IPv4,1000,0,0)  
Atomic_Server_Init_TLS(server1,"./certs/","yourdomain.com","certificate.crt","private.key","ca_bundle.crt")   
OpenConsole()
Atomic_Server_start(server1,1,1) 
Repeat 
Until Input() = "quit" 
Atomic_Server_Exit(server1)
CloseConsole() 
I've managed to get it working with 6.20 redoing the TLS imports and loading the PB libs directly so will update it again in a day or two.
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

Added tlsStatic.pbi so you can use atomic_web_server3.pbi with PB 6.20 for the Reverse Proxy, The dll's are no longer needed for 6.20 but are still there so it can be used with pb 6.12

If you don't need reverse proxy you can use the atomic_web_server3.1.pbi with 6.20 which uses the native PB TLS without the need for the TLS includes or the dlls

an example tile map page
https://atomicwebserver.com/maps.html#9 ... 6/535.3198

uses tiles from https://maps.eox.at/
and also data from https://protomaps.com which you need a free key to use
https://protomaps.com/account
User avatar
skinkairewalker
Enthusiast
Enthusiast
Posts: 772
Joined: Fri Dec 04, 2015 9:26 pm

Re: atomic web server threads

Post by skinkairewalker »

Error to load spiderbasic page :

screenshot : https://prnt.sc/ouaCBnpl4dLu
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

skinkairewalker wrote: Wed Feb 26, 2025 6:35 pm Error to load spiderbasic page :

screenshot : https://prnt.sc/ouaCBnpl4dLu
Did you compile with thread safe?
User avatar
skinkairewalker
Enthusiast
Enthusiast
Posts: 772
Joined: Fri Dec 04, 2015 9:26 pm

Re: atomic web server threads

Post by skinkairewalker »

idle wrote: Wed Feb 26, 2025 8:11 pm
skinkairewalker wrote: Wed Feb 26, 2025 6:35 pm Error to load spiderbasic page :

screenshot : https://prnt.sc/ouaCBnpl4dLu
Did you compile with thread safe?
yep, Can I share the source I'm using?
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

skinkairewalker wrote: Thu Feb 27, 2025 1:44 am
idle wrote: Wed Feb 26, 2025 8:11 pm
skinkairewalker wrote: Wed Feb 26, 2025 6:35 pm Error to load spiderbasic page :

screenshot : https://prnt.sc/ouaCBnpl4dLu
Did you compile with thread safe?
yep, Can I share the source I'm using?
sure dropbox the link and I'll take a look
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

version 3.1.0b15 6.03-6.20 uses PB TLS libs 6.20 or 6.03-6.12 uses the TLS Dlls
version 3.1.1b3 6.20 only uses native TLS but no reverse proxy

redid network error continue and changed the send and receives routines
fixed CORS issue where it was failing to load resources
User avatar
Caronte3D
Addict
Addict
Posts: 1355
Joined: Fri Jan 22, 2016 5:33 pm
Location: Some Universe

Re: atomic web server threads

Post by Caronte3D »

Hi!
Is POST finished?
We can upload files? Can you provide a simple example?
Thanks! :wink:
User avatar
idle
Always Here
Always Here
Posts: 5834
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: atomic web server threads

Post by idle »

Caronte3D wrote: Sun Mar 30, 2025 9:37 pm Hi!
Is POST finished?
We can upload files? Can you provide a simple example?
Thanks! :wink:
Yes with post it's easy via form submission, pictures for instance you can use mimetype/base64EncodedImage
so you can store the data in a safe way and browsers decode it. l will make a demo.
Post Reply