Page 9 of 9
Re: atomic web server threads
Posted: Sat Aug 03, 2024 1:57 am
by idle
There's a mutex for the server thread to protect posting messages to the windows queue it's low contention and may or may not be necessary.
Then there's a client thread mutex to protect fetching from the job queue and also for sending and receiving this mutex is low contention but necessary to protect against connection reuse and then you have a mutex in the TLS.pbi which globally locks the sends and receives.
The complexity is partially due to error handling, for instance ReceiveNetworkData states
Returns the number of bytes received. If 'Result' is equal to DataBufferLength then more data is available to be read. If an error occurred on the connection (link broken, connection close by the server etc...) 'Result' will be -1.
but a socket will return 0 or n bytes read or -1 on error but I don't know if ReceiveNetworkData returns 0 or treats that as a -1
If you get a 0 you would know that a connection is dropped and you could stop polling until your timeout, if you get a -1
99% of the time its a wouldblock which means please wait and try again my tcp buffer is full, there are 7 recoverable errors that should be checked to see if you should continue the operation and the 20 or so other errors are to abort.
ReceiveNetworkData should wrap wsagetlasterror and errno and filter the errors, then it would behave like it's documented but at the moment is doesn't and I still don't know if it returns 0 at all.
So to fix Receive and Send NetworkData for TCP it should test for the below and delay and try again to either return n bytes read or -1 to abort
Code: Select all
#WSA_IO_INCOMPLETE = 996
#WSA_IO_PENDING = 997
#WSAEINTR = 10004
#WSAEMFILE = 10024
#WSAEWOULDBLOCK = 10035
#WSAEINPROGRESS = 10036
#WSAEALREADY = 10037
;for libretls
#_WANT_POLLIN = -2
#_WANT_POLLOUT = -3
or just add a function to check if the error is recoverable so we can set the delay and try again. though I think it'd be better to wrap it.
Re: atomic web server threads
Posted: Sat Aug 03, 2024 2:23 am
by Quin
idle wrote: Sat Aug 03, 2024 1:57 am
but a socket will return 0 or n bytes read or -1 on error but I don't know if ReceiveNetworkData returns 0 or treats that as a -1
If you get a 0 you would know that a connection is dropped and you could stop polling until your timeout, if you get a -1
99% of the time its a wouldblock which means please wait and try again my tcp buffer is full, there are 7 recoverable errors that should be checked to see if you should continue the operation and the 20 or so other errors are to abort.
ReceiveNetworkData should wrap wsagetlasterror and errno and filter the errors, then it would behave like it's documented but at the moment is doesn't and I still don't know if it returns 0 at all.
So to fix Receive and Send NetworkData for TCP it should test for the below and delay and try again to either return n bytes read or -1 to abort
Code: Select all
#WSA_IO_INCOMPLETE = 996
#WSA_IO_PENDING = 997
#WSAEINTR = 10004
#WSAEMFILE = 10024
#WSAEWOULDBLOCK = 10035
#WSAEINPROGRESS = 10036
#WSAEALREADY = 10037
;for libretls
#_WANT_POLLIN = -2
#_WANT_POLLOUT = -3
or just add a function to check if the error is recoverable so we can set the delay and try again. though I think it'd be better to wrap it.
Should probably be moved to feature requests and wishlists, but I completely agree with this. Trying to write networking applications in PB that use large packets is incredibly annoying for exactly this reason.
Re: atomic web server threads
Posted: Sat Aug 03, 2024 5:01 am
by idle
Quin wrote: Sat Aug 03, 2024 2:23 am
Should probably be moved to feature requests and wishlists, but I completely agree with this. Trying to write networking applications in PB that use large packets is incredibly annoying for exactly this reason.
Yes its a little frustrating. Fred will be reading this so I think he can make up his own mind.
It's been a long standing issue and needs addressing in the implementation to match the documentation.
and it should also set a last error.
Re: atomic web server threads
Posted: Mon Dec 16, 2024 3:07 am
by idle
Changed license to Eclipse V2
This is still permissive you can link to to statically and use it You can still use it commercially, it just means any changes to the library need to be shared.
Re: atomic web server threads
Posted: Sun Feb 23, 2025 11:14 pm
by idle
version 3.1.0b12 PB 6.12 reverse proxy working needs the supplied dlls
version 3.1.1b1 PB 6.20 reverse proxy broken but doesn't need supplied the dll.s
changed uri handlers so you can catch paths like
tiles/z/x/y.mvt from a handler set with tiles/
Code: Select all
Procedure tiles(*request.Atomic_Server_Request)
Protected pos,*data,url.s,result,StateOK
Protected *Atomic_Server.Atomic_Server = *request\Serverid
pos = FindString(*request\RequestedFile,"/tiles/")
If pos
url = "https://api.protomaps.com/" + PeekS(@*request\RequestedFile+pos<<1,-1) + "?key=abcdef123" ;your key
Debug url
Repeat
result = HTTPRequestMemory(#PB_HTTP_Get,url,0,0)
If result
HTTPTimeout(result,10000)
StateOK = 200
If HTTPInfo(result,#PB_HTTP_StatusCode) = "200"
*Data = HTTPMemory(result)
*request\status = 200
*request\ContentType = *Atomic_server\MimeTypes("mvt") ;Set the contentType
FinishHTTP(result)
ProcedureReturn *data
Else
Debug HTTPInfo(result,#PB_HTTP_ErrorMessage)
FinishHTTP(result)
Break
EndIf
EndIf
Delay(10)
Until StateOK
EndIf
EndProcedure
server1 = Atomic_Server_Init(title,"./www/",#Server_IP,"atomicwebserver.com",443,#PB_Network_IPv4,1000,0,0)
Atomic_Server_Init_TLS(server1,"./certs/","atomicwebserver.com","certificate.crt","private.key","ca_bundle.crt")
Atomic_Server_Add_Handler(server1,"atomicwebserver.com/tiles",@tiles())
OpenConsole()
Atomic_Server_start(server1,1,1)
Repeat
Until Input() = "quit"
Atomic_Server_Exit(server1)
CloseConsole()
the handler in this case substitutes the call with your key to the tile server so it's not visible in the JS to the user all they see is key=0 and you fetch the tiles from the cloud and route through the server.
I don't know when the multiple cert loading will get added to PB6.21 maybe but till then if you need to host multiple TLS domains on the same machine you will need to use version 3.1.0b12 PB 6.12 Atomic_web_Server3.pbi
Re: atomic web server threads
Posted: Mon Feb 24, 2025 4:01 am
by skinkairewalker
hi how u doing idle ?
atomic web server have support to linux and mac too ?
Re: atomic web server threads
Posted: Mon Feb 24, 2025 5:06 am
by idle
skinkairewalker wrote: Mon Feb 24, 2025 4:01 am
hi how u doing
idle ?
atomic web server have support to linux and mac too ?
I haven't had the time to test the changes on Linux, so it could be broken but in theory it should also work for OSX if you use Aomic_web_server_3.1.pbi which is for PB 6.20, it uses the PB TLS implementation but it can't do terminating TLS reverse proxy. Fred missed that part out in the implementation. It's specifically so you can host multiple domains on the same machine or on some other machine on the LAN and reach them via the TLS proxy on port 443
All it requires is to process a list of certs when creating the server that's acting as the proxy with tls_config_add_keypair_mem load the main domain and then loop through adding the proxied domains
I can probably get it to load additional certs after the servers started but haven't tried yet.
how to use
Code: Select all
#Server_IP = "0.0.0.0"
;this assumes you have an ssl cert
;dirs www/yourdomain.com/index.html
; certs/yourdomainr.com/certs
XIncludeFile "Atomic_Web_Server3.1.pbi"
server1 = Atomic_Server_Init(title,"./www/",#Server_IP,"yourdomain.com",443,#PB_Network_IPv4,1000,0,0)
Atomic_Server_Init_TLS(server1,"./certs/","yourdomain.com","certificate.crt","private.key","ca_bundle.crt")
OpenConsole()
Atomic_Server_start(server1,1,1)
Repeat
Until Input() = "quit"
Atomic_Server_Exit(server1)
CloseConsole()
I've managed to get it working with 6.20 redoing the TLS imports and loading the PB libs directly so will update it again in a day or two.
Re: atomic web server threads
Posted: Tue Feb 25, 2025 2:48 am
by idle
Added tlsStatic.pbi so you can use atomic_web_server3.pbi with PB 6.20 for the Reverse Proxy, The dll's are no longer needed for 6.20 but are still there so it can be used with pb 6.12
If you don't need reverse proxy you can use the atomic_web_server3.1.pbi with 6.20 which uses the native PB TLS without the need for the TLS includes or the dlls
an example tile map page
https://atomicwebserver.com/maps.html#9 ... 6/535.3198
uses tiles from
https://maps.eox.at/
and also data from
https://protomaps.com which you need a free key to use
https://protomaps.com/account
Re: atomic web server threads
Posted: Wed Feb 26, 2025 6:35 pm
by skinkairewalker
Error to load spiderbasic page :
screenshot :
https://prnt.sc/ouaCBnpl4dLu
Re: atomic web server threads
Posted: Wed Feb 26, 2025 8:11 pm
by idle
Did you compile with thread safe?
Re: atomic web server threads
Posted: Thu Feb 27, 2025 1:44 am
by skinkairewalker
idle wrote: Wed Feb 26, 2025 8:11 pm
Did you compile with thread safe?
yep, Can I share the source I'm using?
Re: atomic web server threads
Posted: Thu Feb 27, 2025 1:49 am
by idle
skinkairewalker wrote: Thu Feb 27, 2025 1:44 am
idle wrote: Wed Feb 26, 2025 8:11 pm
Did you compile with thread safe?
yep, Can I share the source I'm using?
sure dropbox the link and I'll take a look
Re: atomic web server threads
Posted: Fri Mar 28, 2025 2:26 am
by idle
version 3.1.0b15 6.03-6.20 uses PB TLS libs 6.20 or 6.03-6.12 uses the TLS Dlls
version 3.1.1b3 6.20 only uses native TLS but no reverse proxy
redid network error continue and changed the send and receives routines
fixed CORS issue where it was failing to load resources
Re: atomic web server threads
Posted: Sun Mar 30, 2025 9:37 pm
by Caronte3D
Hi!
Is POST finished?
We can upload files? Can you provide a simple example?
Thanks!
Re: atomic web server threads
Posted: Sun Mar 30, 2025 10:49 pm
by idle
Caronte3D wrote: Sun Mar 30, 2025 9:37 pm
Hi!
Is POST finished?
We can upload files? Can you provide a simple example?
Thanks!
Yes with post it's easy via form submission, pictures for instance you can use mimetype/base64EncodedImage
so you can store the data in a safe way and browsers decode it. l will make a demo.