lweb a multithreaded webserver with HTTPS support

Applications, Games, Tools, User libs and useful stuff coded in PureBasic
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

Hi all,

Did some work the last days on this web server. And i think it is in a more or less stable state to announce it here :)

The HTTPS Part is currently Linux only and the whole Server so only for Linux. If someone would do improvement there do it 8)

Source is available here: https://gitea.lihaso.com/PB_Includes/lweb

You could use it in a already exist source as a module or standalone webserver...

17.02.2022: Fixed some Problems with PB 6.0b4 and now a Large File could be processed without memory explosion (Read Full file to memory and then send :twisted: .) Now Files bigger than currently 5MiB will be Streamed/Uploaded en Blocks of 3MiB.

This could be changed for high Load Server to smaller (Per Connection is up to 3 Time Blocksize in Memory so for ex. with 300 Connection ca. 3.3Gib Memory is needed :lol: )

20.02.2022: Fixed missing header and wrong header lines.
Added Directory detection (ReadFile on a directory give a correct filehandle ... and so my detection routine was failing...)

Next big things for now are:
  1. Proxy like functions
  2. Virtualhost function (Thats the biggest work on todo list.)
And then a first stable lweb release who is installable.


Greetings...
Last edited by Ground0 on Sun Feb 20, 2022 2:01 pm, edited 1 time in total.
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

Re: lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

lweb did a big step forward since 2021 :)

* Now it could handle big files on send: Default is a big File: 512 KiB an Blocksize goes to 64 KiB
A Ryzen 1700 Core/Thread could handle this way ~ 1 GBit/s

* Redirect on wrong subdirectory ex. https://localhost:8446/testportal -> https://localhost:8446/testportal/index.html

It Still Works with 5.73 LTS and also with 6.0b4 ASM & C Compiler
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

That looks very good. thanks for sharing.

I've just been looking into libtls on windows but couldn't resolve the undefined symbols due to the messed up CRT. Maybe I need to redo it with a Mingw build instead.

While it's fine to use dynamic linking on linux you would probably be better off using static bindings. do you want me to do a test?
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

Re: lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

Hi idle

You could do that, but i've used dynamic linking to fast replace the libraries with secure Versions (CVE & co.) maybe i should do a better linking procedure and not static way (.22 .20 and co.)

You could use the whole source as you want (Currently only developed by me so it is GPLv2 /LPGL v2.1)

If you register you could also commit to this repo directly 8)

Greeting
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

you can still use the dynamic libs on linux but rather than using prototypes just use ImportC, I'm currently on windows but this should find the installed version. I will check it later.

Code: Select all

; /* $OpenBSD: tls.h,v 1.58 2020/01/22 06:44:02 beck Exp $ */
; /*
;  * Copyright (c) 2014 Joel Sing <jsing@openbsd.org>
;  *
;  * Permission To use, copy, modify, And distribute this software For any
;  * purpose With Or without fee is hereby granted, provided that the above
;  * copyright notice And this permission notice appear in all copies.
;  *
;  * THE SOFTWARE IS PROVIDED "AS IS" And THE AUTHOR DISCLAIMS ALL WARRANTIES
;  * With REGARD To THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;  * MERCHANTABILITY And FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE For
;  * ANY SPECIAL, DIRECT, INDIRECT, Or CONSEQUENTIAL DAMAGES Or ANY DAMAGES
;  * WHATSOEVER RESULTING FROM LOSS OF USE, Data Or PROFITS, WHETHER IN AN
;  * ACTION OF CONTRACT, NEGLIGENCE Or OTHER TORTIOUS ACTION, ARISING OUT OF
;  * Or IN CONNECTION With THE USE Or PERFORMANCE OF THIS SOFTWARE.
;  */

Macro const : string : EndMacro 

#TLS_API=	20200120

#TLS_PROTOCOL_TLSv1_0=	(1 << 1)
#TLS_PROTOCOL_TLSv1_1=	(1 << 2)
#TLS_PROTOCOL_TLSv1_2=	(1 << 3)
#TLS_PROTOCOL_TLSv1_3=	(1 << 4)

#TLS_PROTOCOL_TLSv1= #TLS_PROTOCOL_TLSv1_0 | #TLS_PROTOCOL_TLSv1_1 | #TLS_PROTOCOL_TLSv1_2 | #TLS_PROTOCOL_TLSv1_3

#TLS_PROTOCOLS_ALL= #TLS_PROTOCOL_TLSv1
#TLS_PROTOCOLS_DEFAULT= (#TLS_PROTOCOL_TLSv1_2|#TLS_PROTOCOL_TLSv1_3)

#TLS_WANT_POLLIN=	-2
#TLS_WANT_POLLOUT=	-3

;/* RFC 6960 Section 2.3 */
#TLS_OCSP_RESPONSE_SUCCESSFUL=		0
#TLS_OCSP_RESPONSE_MALFORMED=		1
#TLS_OCSP_RESPONSE_INTERNALERROR=		2
#TLS_OCSP_RESPONSE_TRYLATER=		3
#TLS_OCSP_RESPONSE_SIGREQUIRED=		4
#TLS_OCSP_RESPONSE_UNAUTHORIZED=		5

;/* RFC 6960 Section 2.2 */
#TLS_OCSP_CERT_GOOD=			0
#TLS_OCSP_CERT_REVOKED=			1
#TLS_OCSP_CERT_UNKNOWN=			2

;/* RFC 5280 Section 5.3.1 */
#TLS_CRL_REASON_UNSPECIFIED=		0
#TLS_CRL_REASON_KEY_COMPROMISE=		1
#TLS_CRL_REASON_CA_COMPROMISE=		2
#TLS_CRL_REASON_AFFILIATION_CHANGED=	3
#TLS_CRL_REASON_SUPERSEDED=		4
#TLS_CRL_REASON_CESSATION_OF_OPERATION=	5
#TLS_CRL_REASON_CERTIFICATE_HOLD=		6
#TLS_CRL_REASON_REMOVE_FROM_CRL=		8
#TLS_CRL_REASON_PRIVILEGE_WITHDRAWN=	9
#TLS_CRL_REASON_AA_COMPROMISE=		10

#TLS_MAX_SESSION_ID_LENGTH=		32
#TLS_TICKET_KEY_SIZE=			48

PrototypeC tls_read_cb(*_ctx,*_buf,_buflen,*_cb_arg);
PrototypeC tls_write_cb(*_ctx,*_buf,_buflen,*_cb_arg);

ImportC "-ltls"
  tls_init()                                                                                                   ;
  tls_config_error(*config)                                                                            ;
  tls_error(*ctx)                                                                                      ;
  tls_config_new()                                                                                             ;
  tls_config_free(*config)                                                                             ;
  tls_default_ca_cert_file()                                                                                   ;
  tls_config_add_keypair_file(*config,*_cert_file.const,*_key_file.const)                              ;
  tls_config_add_keypair_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i)        ;
  tls_config_add_keypair_ocsp_file(*config,*_cert_file.const,*_key_file.const,*_ocsp_staple_file.const);
  tls_config_add_keypair_ocsp_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i,*_staple.const,_staple_len.i);
  tls_config_set_alpn(*config,*_alpn.const)                                                                                           ;
  tls_config_set_ca_file(*config,*_ca_file.const)                                                                                     ;
  tls_config_set_ca_path(*config,*_ca_path.const)                                                                                     ;
  tls_config_set_ca_mem(*config,*_ca.const,_len.i)                                                                               ;
  tls_config_set_cert_file(*config,*_cert_file.const)                                                                                 ;
  tls_config_set_cert_mem(*config,*_cert.const,_len.i)                                                                           ;
  tls_config_set_ciphers(*config,*_ciphers.const)                                                                                     ;
  tls_config_set_crl_file(*config,*_crl_file.const)                                                                                   ;
  tls_config_set_crl_mem(*config,*_crl.const,_len.i)                                                                             ;
  tls_config_set_dheparams(*config,*_params.const)                                                                                    ;
  tls_config_set_ecdhecurve(*config,*_curve.const)                                                                                    ;
  tls_config_set_ecdhecurves(*config,*_curves.const)                                                                                  ;
  tls_config_set_key_file(*config,*_key_file.const)                                                                                   ;
  tls_config_set_key_mem(*config,*_key.const,_len.i)                                                                             ;
  tls_config_set_keypair_file(*config,*_cert_file.const,*_key_file.const)                                                             ;
  tls_config_set_keypair_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i)                                       ;
  tls_config_set_keypair_ocsp_file(*config,*_cert_file.const,*_key_file.const,*_staple_file.const)                                    ;
  tls_config_set_keypair_ocsp_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i,*_staple.const,staple_len.i) ;
  tls_config_set_ocsp_staple_mem(*config,*_staple.const,_len.i)                                                                  ;
  tls_config_set_ocsp_staple_file(*config,*_staple_file.const)                                                                        ;
  tls_config_set_protocols(*config,_protocols.l)                                                                               ;
  tls_config_set_session_fd(*config,_session_fd.l)                                                                                  ;
  tls_config_set_verify_depth(*config,_verify_depth.l)                                                                              ;
  tls_config_prefer_ciphers_client(*config)                                                                                           ;
  tls_config_prefer_ciphers_server(*config)                                                                                           ;
  tls_config_insecure_noverifycert(*config)                                                                                           ;
  tls_config_insecure_noverifyname(*config)                                                                                           ;
  tls_config_insecure_noverifytime(*config)                                                                                           ;
  tls_config_verify(*config)                                                                                                          ;
  tls_config_ocsp_require_stapling(*config)                                                                                           ;
  tls_config_verify_client(*config)                                                                                                   ;
  tls_config_verify_client_optional(*config)                                                                                          ;
  tls_config_clear_keys(*config)                                                                                                      ;
  tls_config_parse_protocols(*_protocols,*_protostr.const)                                                                           ;
  tls_config_set_session_id(*config,*_session_id.const,_len.i)                                                                   ;
  tls_config_set_session_lifetime(*config,_lifetime.l)                                                                              ;
  tls_config_add_ticket_key(*config,_keyrev.i,*_key,_keylen.i)                                                   ;
  tls_client()                                                                                                                                ;
  tls_server()                                                                                                                                ;
  tls_configure(*ctx,*config)                                                                                                 ;
  tls_reset(*ctx)                                                                                                                     ;
  tls_free(*ctx)                                                                                                                      ;
  tls_accept_fds(*ctx,*_cctx,_fd_read.l,_fd_write.l)                                                                       ;
  tls_accept_socket(*ctx,*_cctx,_socket.l)                                                                                   ;
  tls_accept_cbs(*ctx,*_cctx,*_read_cb.tls_read_cb,*_write_cb.tls_write_cb,*_cb_arg)                                      ;
  tls_connect(*ctx,*_host.const,*_port.const)                                                                                         ;
  tls_connect_fds(*ctx,_fd_read.l,_fd_write.l,*_servername.const)                                                                 ;
  tls_connect_servername(*ctx,*_host.const,*_port.const,*_servername.const)                                                           ;
  tls_connect_socket(*ctx,_s.l,*_servername.const)                                                                                  ;
  tls_connect_cbs(*ctx,*_read_cb.tls_read_cb,*_write_cb.tls_write_cb,*_cb_arg,*_servername.const)                                ;
  tls_handshake(*ctx)                                                                                                                 ;
  tls_read(*ctx,*_buf,_buflen.i)                                                                                            ;
  tls_write(*ctx,*_buf.const,_buflen.i)                                                                                          ;
  tls_close(*ctx)                                                                                                                     ;
  tls_peer_cert_provided(*ctx)                                                                                                        ;
  tls_peer_cert_contains_name(*ctx,*_name.const)                                                                                      ;
  tls_peer_cert_hash(*ctx)                                                                                                            ;
  tls_peer_cert_issuer(*ctx)                                                                                                          ;
  tls_peer_cert_subject(*ctx)                                                                                                         ;
  tls_peer_cert_notbefore(*ctx)                                                                                                       ;
  tls_peer_cert_notafter(*ctx)                                                                                                        ;
  tls_peer_cert_chain_pem(*ctx,*_len)                                                                                          ;
  tls_conn_alpn_selected(*ctx)                                                                                                        ;
  tls_conn_cipher(*ctx)                                                                                                               ;
  tls_conn_cipher_strength(*ctx)                                                                                                      ;
  tls_conn_servername(*ctx)                                                                                                           ;
  tls_conn_session_resumed(*ctx)                                                                                                      ;
  tls_conn_version(*ctx)                                                                                                              ;
  tls_load_file(*_file.const,*_len,*_password)                                                                                    ;
  tls_unload_file(*_buf,len.i)                                                                                                   ;
  tls_ocsp_process_response(*ctx,*_response.const,_size.i)                                                                       ;
  tls_peer_ocsp_cert_status(*ctx)                                                                                                     ;
  tls_peer_ocsp_crl_reason(*ctx)                                                                                                      ;
  tls_peer_ocsp_next_update(*ctx)                                                                                                     ;
  tls_peer_ocsp_response_status(*ctx)                                                                                                 ;
  tls_peer_ocsp_result(*ctx)                                                                                                          ;
  tls_peer_ocsp_revocation_time(*ctx)                                                                                                 ;
  tls_peer_ocsp_this_update(*ctx)                                                                                                     ;
  tls_peer_ocsp_url(*ctx)                                                                                                             ;
EndImport

If tls_init() = 0 
  Debug "ok"
EndIf   


User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

I've got the server working on windows, it wasn't hard but I still need to sort out the config a bit more to make a cross platform example. It's a nice bit of code, well done!

I've made certs for localhost though you still need to allow it to pass through but it's working, I just need to sanity check the import code and do some tests on linux and get the Pi version going which might be compiler issues?
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: lweb a multithreaded webserver with HTTPS support

Post by Kwai chang caine »

Is it normal the "lweb\lhs_lib\" is empty ? :|
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

It was a link I downloaded it separately but I haven't posted what I've done yet still i still need to test changes on Linux and pi
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

Re: lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

Kwai chang caine wrote: Sun Mar 06, 2022 6:06 pm Is it normal the "lweb\lhs_lib\" is empty ? :|
Yes it is Normal.

You could Clone it directly with:

Code: Select all

git clone https://gitea.lihaso.com/PB_Includes/lweb.git --recurse-submodules
Or if you already cloned it:

Code: Select all

git submodule init
git submodule update
It is now linked with https and not anymore with ssh.
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

here's a version for windows x64 with Libretls

https://www.dropbox.com/s/vfkob35073v6fwv/lweb.zip?dl=0
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

Re: lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

A Lot of new work is done :-)

https://gitea.lihaso.com/PB_Includes/lweb

Who wants to help is invited ... account creation is free :)
  • Configuration is now xml based (Simple but it is XML 8) )
  • Multi Server Host on different Ports
  • Some library changes lhs_lib
  • Dynamic Library loading for advanced PureBasic Power in the Web 8)
A lot of more stuff is incomming :!:
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

How come you didn't integrate the changes i made so it worked on both Linux and windows.
Ground0
User
User
Posts: 40
Joined: Fri Jun 09, 2006 10:20 pm
Location: Stilli, Schweiz
Contact:

Re: lweb a multithreaded webserver with HTTPS support

Post by Ground0 »

Hi idle

I didn't setup a Windows Dev machine until now... so my last changes to the code is tested on Windows ... but i will add the socket and tls windows change as soon as possible.

Greeting
Sys 1 HW: Ryzen 7 3700X, 32GB RAM, RX 5700, NVME 512GiB And SATA 2TiB OS: openSUSE Tumbleweed x64
Sys 2 HW: 7 1700X, 16GB RAM, R7 360, SSD 480GiB & 1TiB OS: "
Sys 3 NB: HP envy X360, 5 2500U ,16GB RAM, NVME 256Gib OS: " zus. Win. 10 x64
User avatar
skinkairewalker
Enthusiast
Enthusiast
Posts: 627
Joined: Fri Dec 04, 2015 9:26 pm

Re: lweb a multithreaded webserver with HTTPS support

Post by skinkairewalker »

it would be epic if it had Websocket servers, much like Jetty does.
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: lweb a multithreaded webserver with HTTPS support

Post by idle »

There is no issue using an import lib on windows to load a dll and on Linux it's encouraged to link with -l so it will find either .a or .so if you want to force it to dynamic you add the .so
It's about flexibility you want dynamic I want static and if you use the imports you get both. Static or dynamic so which is better?
Post Reply