Page 1 of 1

lweb a multithreaded webserver with HTTPS support

Posted: Fri May 21, 2021 10:57 am
by Ground0
Hi all,

Did some work the last days on this web server. And i think it is in a more or less stable state to announce it here :)

The HTTPS Part is currently Linux only and the whole Server so only for Linux. If someone would do improvement there do it 8)

Source is available here: https://gitea.lihaso.com/PB_Includes/lweb

You could use it in a already exist source as a module or standalone webserver...

17.02.2022: Fixed some Problems with PB 6.0b4 and now a Large File could be processed without memory explosion (Read Full file to memory and then send :twisted: .) Now Files bigger than currently 5MiB will be Streamed/Uploaded en Blocks of 3MiB.

This could be changed for high Load Server to smaller (Per Connection is up to 3 Time Blocksize in Memory so for ex. with 300 Connection ca. 3.3Gib Memory is needed :lol: )

20.02.2022: Fixed missing header and wrong header lines.
Added Directory detection (ReadFile on a directory give a correct filehandle ... and so my detection routine was failing...)

Next big things for now are:
  1. Proxy like functions
  2. Virtualhost function (Thats the biggest work on todo list.)
And then a first stable lweb release who is installable.


Greetings...

Re: lweb a multithreaded webserver with HTTPS support

Posted: Fri Feb 18, 2022 4:16 pm
by Ground0
lweb did a big step forward since 2021 :)

* Now it could handle big files on send: Default is a big File: 512 KiB an Blocksize goes to 64 KiB
A Ryzen 1700 Core/Thread could handle this way ~ 1 GBit/s

* Redirect on wrong subdirectory ex. https://localhost:8446/testportal -> https://localhost:8446/testportal/index.html

It Still Works with 5.73 LTS and also with 6.0b4 ASM & C Compiler

Re: lweb a multithreaded webserver with HTTPS support

Posted: Fri Feb 25, 2022 6:24 am
by idle
That looks very good. thanks for sharing.

I've just been looking into libtls on windows but couldn't resolve the undefined symbols due to the messed up CRT. Maybe I need to redo it with a Mingw build instead.

While it's fine to use dynamic linking on linux you would probably be better off using static bindings. do you want me to do a test?

Re: lweb a multithreaded webserver with HTTPS support

Posted: Tue Mar 01, 2022 10:23 pm
by Ground0
Hi idle

You could do that, but i've used dynamic linking to fast replace the libraries with secure Versions (CVE & co.) maybe i should do a better linking procedure and not static way (.22 .20 and co.)

You could use the whole source as you want (Currently only developed by me so it is GPLv2 /LPGL v2.1)

If you register you could also commit to this repo directly 8)

Greeting

Re: lweb a multithreaded webserver with HTTPS support

Posted: Wed Mar 02, 2022 1:09 am
by idle
you can still use the dynamic libs on linux but rather than using prototypes just use ImportC, I'm currently on windows but this should find the installed version. I will check it later.

Code: Select all

; /* $OpenBSD: tls.h,v 1.58 2020/01/22 06:44:02 beck Exp $ */
; /*
;  * Copyright (c) 2014 Joel Sing <jsing@openbsd.org>
;  *
;  * Permission To use, copy, modify, And distribute this software For any
;  * purpose With Or without fee is hereby granted, provided that the above
;  * copyright notice And this permission notice appear in all copies.
;  *
;  * THE SOFTWARE IS PROVIDED "AS IS" And THE AUTHOR DISCLAIMS ALL WARRANTIES
;  * With REGARD To THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
;  * MERCHANTABILITY And FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE For
;  * ANY SPECIAL, DIRECT, INDIRECT, Or CONSEQUENTIAL DAMAGES Or ANY DAMAGES
;  * WHATSOEVER RESULTING FROM LOSS OF USE, Data Or PROFITS, WHETHER IN AN
;  * ACTION OF CONTRACT, NEGLIGENCE Or OTHER TORTIOUS ACTION, ARISING OUT OF
;  * Or IN CONNECTION With THE USE Or PERFORMANCE OF THIS SOFTWARE.
;  */

Macro const : string : EndMacro 

#TLS_API=	20200120

#TLS_PROTOCOL_TLSv1_0=	(1 << 1)
#TLS_PROTOCOL_TLSv1_1=	(1 << 2)
#TLS_PROTOCOL_TLSv1_2=	(1 << 3)
#TLS_PROTOCOL_TLSv1_3=	(1 << 4)

#TLS_PROTOCOL_TLSv1= #TLS_PROTOCOL_TLSv1_0 | #TLS_PROTOCOL_TLSv1_1 | #TLS_PROTOCOL_TLSv1_2 | #TLS_PROTOCOL_TLSv1_3

#TLS_PROTOCOLS_ALL= #TLS_PROTOCOL_TLSv1
#TLS_PROTOCOLS_DEFAULT= (#TLS_PROTOCOL_TLSv1_2|#TLS_PROTOCOL_TLSv1_3)

#TLS_WANT_POLLIN=	-2
#TLS_WANT_POLLOUT=	-3

;/* RFC 6960 Section 2.3 */
#TLS_OCSP_RESPONSE_SUCCESSFUL=		0
#TLS_OCSP_RESPONSE_MALFORMED=		1
#TLS_OCSP_RESPONSE_INTERNALERROR=		2
#TLS_OCSP_RESPONSE_TRYLATER=		3
#TLS_OCSP_RESPONSE_SIGREQUIRED=		4
#TLS_OCSP_RESPONSE_UNAUTHORIZED=		5

;/* RFC 6960 Section 2.2 */
#TLS_OCSP_CERT_GOOD=			0
#TLS_OCSP_CERT_REVOKED=			1
#TLS_OCSP_CERT_UNKNOWN=			2

;/* RFC 5280 Section 5.3.1 */
#TLS_CRL_REASON_UNSPECIFIED=		0
#TLS_CRL_REASON_KEY_COMPROMISE=		1
#TLS_CRL_REASON_CA_COMPROMISE=		2
#TLS_CRL_REASON_AFFILIATION_CHANGED=	3
#TLS_CRL_REASON_SUPERSEDED=		4
#TLS_CRL_REASON_CESSATION_OF_OPERATION=	5
#TLS_CRL_REASON_CERTIFICATE_HOLD=		6
#TLS_CRL_REASON_REMOVE_FROM_CRL=		8
#TLS_CRL_REASON_PRIVILEGE_WITHDRAWN=	9
#TLS_CRL_REASON_AA_COMPROMISE=		10

#TLS_MAX_SESSION_ID_LENGTH=		32
#TLS_TICKET_KEY_SIZE=			48

PrototypeC tls_read_cb(*_ctx,*_buf,_buflen,*_cb_arg);
PrototypeC tls_write_cb(*_ctx,*_buf,_buflen,*_cb_arg);

ImportC "-ltls"
  tls_init()                                                                                                   ;
  tls_config_error(*config)                                                                            ;
  tls_error(*ctx)                                                                                      ;
  tls_config_new()                                                                                             ;
  tls_config_free(*config)                                                                             ;
  tls_default_ca_cert_file()                                                                                   ;
  tls_config_add_keypair_file(*config,*_cert_file.const,*_key_file.const)                              ;
  tls_config_add_keypair_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i)        ;
  tls_config_add_keypair_ocsp_file(*config,*_cert_file.const,*_key_file.const,*_ocsp_staple_file.const);
  tls_config_add_keypair_ocsp_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i,*_staple.const,_staple_len.i);
  tls_config_set_alpn(*config,*_alpn.const)                                                                                           ;
  tls_config_set_ca_file(*config,*_ca_file.const)                                                                                     ;
  tls_config_set_ca_path(*config,*_ca_path.const)                                                                                     ;
  tls_config_set_ca_mem(*config,*_ca.const,_len.i)                                                                               ;
  tls_config_set_cert_file(*config,*_cert_file.const)                                                                                 ;
  tls_config_set_cert_mem(*config,*_cert.const,_len.i)                                                                           ;
  tls_config_set_ciphers(*config,*_ciphers.const)                                                                                     ;
  tls_config_set_crl_file(*config,*_crl_file.const)                                                                                   ;
  tls_config_set_crl_mem(*config,*_crl.const,_len.i)                                                                             ;
  tls_config_set_dheparams(*config,*_params.const)                                                                                    ;
  tls_config_set_ecdhecurve(*config,*_curve.const)                                                                                    ;
  tls_config_set_ecdhecurves(*config,*_curves.const)                                                                                  ;
  tls_config_set_key_file(*config,*_key_file.const)                                                                                   ;
  tls_config_set_key_mem(*config,*_key.const,_len.i)                                                                             ;
  tls_config_set_keypair_file(*config,*_cert_file.const,*_key_file.const)                                                             ;
  tls_config_set_keypair_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i)                                       ;
  tls_config_set_keypair_ocsp_file(*config,*_cert_file.const,*_key_file.const,*_staple_file.const)                                    ;
  tls_config_set_keypair_ocsp_mem(*config,*_cert.const,_cert_len.i,*_key.const,_key_len.i,*_staple.const,staple_len.i) ;
  tls_config_set_ocsp_staple_mem(*config,*_staple.const,_len.i)                                                                  ;
  tls_config_set_ocsp_staple_file(*config,*_staple_file.const)                                                                        ;
  tls_config_set_protocols(*config,_protocols.l)                                                                               ;
  tls_config_set_session_fd(*config,_session_fd.l)                                                                                  ;
  tls_config_set_verify_depth(*config,_verify_depth.l)                                                                              ;
  tls_config_prefer_ciphers_client(*config)                                                                                           ;
  tls_config_prefer_ciphers_server(*config)                                                                                           ;
  tls_config_insecure_noverifycert(*config)                                                                                           ;
  tls_config_insecure_noverifyname(*config)                                                                                           ;
  tls_config_insecure_noverifytime(*config)                                                                                           ;
  tls_config_verify(*config)                                                                                                          ;
  tls_config_ocsp_require_stapling(*config)                                                                                           ;
  tls_config_verify_client(*config)                                                                                                   ;
  tls_config_verify_client_optional(*config)                                                                                          ;
  tls_config_clear_keys(*config)                                                                                                      ;
  tls_config_parse_protocols(*_protocols,*_protostr.const)                                                                           ;
  tls_config_set_session_id(*config,*_session_id.const,_len.i)                                                                   ;
  tls_config_set_session_lifetime(*config,_lifetime.l)                                                                              ;
  tls_config_add_ticket_key(*config,_keyrev.i,*_key,_keylen.i)                                                   ;
  tls_client()                                                                                                                                ;
  tls_server()                                                                                                                                ;
  tls_configure(*ctx,*config)                                                                                                 ;
  tls_reset(*ctx)                                                                                                                     ;
  tls_free(*ctx)                                                                                                                      ;
  tls_accept_fds(*ctx,*_cctx,_fd_read.l,_fd_write.l)                                                                       ;
  tls_accept_socket(*ctx,*_cctx,_socket.l)                                                                                   ;
  tls_accept_cbs(*ctx,*_cctx,*_read_cb.tls_read_cb,*_write_cb.tls_write_cb,*_cb_arg)                                      ;
  tls_connect(*ctx,*_host.const,*_port.const)                                                                                         ;
  tls_connect_fds(*ctx,_fd_read.l,_fd_write.l,*_servername.const)                                                                 ;
  tls_connect_servername(*ctx,*_host.const,*_port.const,*_servername.const)                                                           ;
  tls_connect_socket(*ctx,_s.l,*_servername.const)                                                                                  ;
  tls_connect_cbs(*ctx,*_read_cb.tls_read_cb,*_write_cb.tls_write_cb,*_cb_arg,*_servername.const)                                ;
  tls_handshake(*ctx)                                                                                                                 ;
  tls_read(*ctx,*_buf,_buflen.i)                                                                                            ;
  tls_write(*ctx,*_buf.const,_buflen.i)                                                                                          ;
  tls_close(*ctx)                                                                                                                     ;
  tls_peer_cert_provided(*ctx)                                                                                                        ;
  tls_peer_cert_contains_name(*ctx,*_name.const)                                                                                      ;
  tls_peer_cert_hash(*ctx)                                                                                                            ;
  tls_peer_cert_issuer(*ctx)                                                                                                          ;
  tls_peer_cert_subject(*ctx)                                                                                                         ;
  tls_peer_cert_notbefore(*ctx)                                                                                                       ;
  tls_peer_cert_notafter(*ctx)                                                                                                        ;
  tls_peer_cert_chain_pem(*ctx,*_len)                                                                                          ;
  tls_conn_alpn_selected(*ctx)                                                                                                        ;
  tls_conn_cipher(*ctx)                                                                                                               ;
  tls_conn_cipher_strength(*ctx)                                                                                                      ;
  tls_conn_servername(*ctx)                                                                                                           ;
  tls_conn_session_resumed(*ctx)                                                                                                      ;
  tls_conn_version(*ctx)                                                                                                              ;
  tls_load_file(*_file.const,*_len,*_password)                                                                                    ;
  tls_unload_file(*_buf,len.i)                                                                                                   ;
  tls_ocsp_process_response(*ctx,*_response.const,_size.i)                                                                       ;
  tls_peer_ocsp_cert_status(*ctx)                                                                                                     ;
  tls_peer_ocsp_crl_reason(*ctx)                                                                                                      ;
  tls_peer_ocsp_next_update(*ctx)                                                                                                     ;
  tls_peer_ocsp_response_status(*ctx)                                                                                                 ;
  tls_peer_ocsp_result(*ctx)                                                                                                          ;
  tls_peer_ocsp_revocation_time(*ctx)                                                                                                 ;
  tls_peer_ocsp_this_update(*ctx)                                                                                                     ;
  tls_peer_ocsp_url(*ctx)                                                                                                             ;
EndImport

If tls_init() = 0 
  Debug "ok"
EndIf   



Re: lweb a multithreaded webserver with HTTPS support

Posted: Sun Mar 06, 2022 1:00 am
by idle
I've got the server working on windows, it wasn't hard but I still need to sort out the config a bit more to make a cross platform example. It's a nice bit of code, well done!

I've made certs for localhost though you still need to allow it to pass through but it's working, I just need to sanity check the import code and do some tests on linux and get the Pi version going which might be compiler issues?

Re: lweb a multithreaded webserver with HTTPS support

Posted: Sun Mar 06, 2022 6:06 pm
by Kwai chang caine
Is it normal the "lweb\lhs_lib\" is empty ? :|

Re: lweb a multithreaded webserver with HTTPS support

Posted: Sun Mar 06, 2022 8:21 pm
by idle
It was a link I downloaded it separately but I haven't posted what I've done yet still i still need to test changes on Linux and pi

Re: lweb a multithreaded webserver with HTTPS support

Posted: Mon Mar 07, 2022 6:58 am
by Ground0
Kwai chang caine wrote: Sun Mar 06, 2022 6:06 pm Is it normal the "lweb\lhs_lib\" is empty ? :|
Yes it is Normal.

You could Clone it directly with:

Code: Select all

git clone https://gitea.lihaso.com/PB_Includes/lweb.git --recurse-submodules
Or if you already cloned it:

Code: Select all

git submodule init
git submodule update
It is now linked with https and not anymore with ssh.

Re: lweb a multithreaded webserver with HTTPS support

Posted: Thu Mar 31, 2022 3:09 am
by idle
here's a version for windows x64 with Libretls

https://www.dropbox.com/s/vfkob35073v6fwv/lweb.zip?dl=0

Re: lweb a multithreaded webserver with HTTPS support

Posted: Wed Mar 29, 2023 7:16 pm
by Ground0
A Lot of new work is done :-)

https://gitea.lihaso.com/PB_Includes/lweb

Who wants to help is invited ... account creation is free :)
  • Configuration is now xml based (Simple but it is XML 8) )
  • Multi Server Host on different Ports
  • Some library changes lhs_lib
  • Dynamic Library loading for advanced PureBasic Power in the Web 8)
A lot of more stuff is incomming :!:

Re: lweb a multithreaded webserver with HTTPS support

Posted: Wed Mar 29, 2023 7:59 pm
by idle
How come you didn't integrate the changes i made so it worked on both Linux and windows.

Re: lweb a multithreaded webserver with HTTPS support

Posted: Thu Mar 30, 2023 6:56 am
by Ground0
Hi idle

I didn't setup a Windows Dev machine until now... so my last changes to the code is tested on Windows ... but i will add the socket and tls windows change as soon as possible.

Greeting

Re: lweb a multithreaded webserver with HTTPS support

Posted: Thu Mar 30, 2023 5:11 pm
by skinkairewalker
it would be epic if it had Websocket servers, much like Jetty does.

Re: lweb a multithreaded webserver with HTTPS support

Posted: Fri Mar 31, 2023 8:12 pm
by idle
There is no issue using an import lib on windows to load a dll and on Linux it's encouraged to link with -l so it will find either .a or .so if you want to force it to dynamic you add the .so
It's about flexibility you want dynamic I want static and if you use the imports you get both. Static or dynamic so which is better?