PB_Civet_Server full stack web solution for PB

Applications, Games, Tools, User libs and useful stuff coded in PureBasic
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

PB_Civet_Server full stack web solution for PB

Post by idle »

PB_Civet_Server, a full stack web solution for PB.
An embedded in memory http / https web server with a PB tag preprocessor
to unleash the pure power of PB on the web, built on civetweb a MIT fork of mongoose server before it went GPL in 2013.

Current build is v1.0 built on Civetweb 1.10, windows x86 build at the moment

The package contains all you need to get started.
See civet_server_in_memSSL.pb which is based on civetserver.pbi wrapper to civetweb.pbi import

Civet_server_in_memSSL.pb demonstrates an embedded in memory ssl server, served to a webgadget
The site is packed in "www.ezp" which you can extract to file by the provided ezpack test file unpackfromdisk
(couldn't upload the www source folder to git hub as it had to many files in it, so you can inspect and extract it)
you can then switch from running in memory to file by commenting out app\packAddress =?Index
the source of the site can then be edited and repacked by using the utility function
Civet_Server_Create_Pack(app)

The Preprocessor expects files saved as ".pbh" and you simply tag where you want to inline the output
as <?PB SomeFunction() /> and declare your procedure as runtime. Easy!
delivering an ASP or php like experience

CivetServer.pbi
Is the PB friendly wrapper for civetweb, it's not complete needs some changes but it's a starting point
it has additional dependencies to ezpack.pbi archiver for in memory functions
and has it's own mimetypes.pbi which shouldn't really be required but a few in civetweb are wrong
and it's easier to add them to the mimetypes.pbi as required rather than rebuild civetweb.lib
Civetserver is designed to work from both file and in memory. Civetweb itself doesn't facilitate in memory functionality
so it bypasses a considerable portion of checks civetweb's would normally handle, though the file based functions
are still passed through civetweb's pipeline.
It's usage is demonstrated in civet_server_in_memSSL.pb
basically all you need to do is handle get and post queries and return the appropriate response
you can easily handle ajax calls for instance and return json objects from file or in memory
and with the preprocessor is makes it dead easy to inline your output.


Civetweb.pbi
Is just the raw import of the static libs change the import to which version you want ssl or non ssl
Not all of the functionality of civetweb is tested. see civetweb site for implementation and documentation
The current build is v 1.10 and will be updated to 1.11 when I get around to it.

If it doesn't work it at all you may need to install the msvc 2017 redistributables. which you would distribute
with an application setup.
https://aka.ms/vs/15/release/vc_redist.x86.exe

The project download link
https://github.com/idle-PB/pb_civet_server

The project was commissioned by one of our members who may or may not want to be publically attributed
and I thank him for his custom and generosity in allowing me to share the project with the community
I am looking for contributors and someone for linux and osx who can build the static libs

PM me if your interested in contributing.

Source of civet_server_in_memSSL.pb

Code: Select all

IncludeFile "civetserver.pbi"

Global app.Civet_Server  

Procedure Resize()
  ResizeGadget(2,0,0,WindowWidth(0),WindowHeight(0)) 
EndProcedure 

;If you declare a runtime procedure and add a tag <?PB ElementFillTable() /> in the source html page remamed as .pbh 
;it will call the function and inline the code in the output
Runtime Procedure ElementsFillTable(*app.Civet_Server) 
  Protected sout.s,sum.f,a  
  sout = "<div><H2>Output from ElementsFillTable() Callback</h2></div>"
  sout + "<div class='table-wrapper'><table><thead><tr><th>Name</th><th>Description</th><th>Price</th></tr></thead>" + #LF$
  sout + "<tbody>" + #LF$ 
  For a = 1 To 20 
    sout + "<tr><td>Item " + Str(a) + "</td><td>Ante turpis integer aliquet porttitor.</td><td>" + StrF(a * 2.99,2) + "</td></tr>" + #LF$
    sum + (a * 2.99)  
  Next 
  sout + "</tbody><tfoot><tr><td colspan='2'></td><td>$" + StrF(sum,2) + "</td></tr></tfoot></table></div>"
  ProcedureReturn UTF8(sout) 
  
EndProcedure   

Procedure cbPost(ctx,*request.Civet_Server_Request) ;callbacks are threaded only use locally scoped resources  
  
  Debug "cbpost request type " +  *request\RequestType 
  Debug *request\Uri 
  
  ForEach *request\mquery()
    Debug *request\mquery()\name + " " + *request\mquery()\value 
  Next
  
  *request\Uri = "/elements.pbh"  ;redirect to  
  
  ProcedureReturn 0 ;to continue processing or 200  
  
EndProcedure  

Procedure cbGet(ctx,*request.Civet_Server_Request)
  
  Debug "cbGet request type " +  *request\RequestType 
  Debug *request\Uri 
  Debug GetFilePart(*request\Uri)
  
  ForEach *request\mquery()
    Debug MapKey(*request\mQuery()) + " " + *request\mquery()\name + " " + *request\mquery()\value 
  Next 
  
  If *request\mQuery("play")\value = "Waponez" 
    RunProgram("Http://127.0.0.1/Waponez.html")
    Civet_Server_Send_Response(ctx,"",0,0,"204 No Content") 
    ProcedureReturn 204
  EndIf     
  
  ProcedureReturn 0
EndProcedure   



app\packAddress =?Index  ;address for ezp pack archive of website in datasection. if unset will load from files. 
app\packsize = ?EndIndex - ?Index ;size of the pack to read in  
app\dirRelitive = "www"           ;required     

app\cbpost = @cbpost()   ;set a call back to process the post tokens 
app\cbget = @cbGet()     ;set a call back to process query strings from a get request 

Debug GetPathPart(ProgramFilename()) + app\dirRelitive
app\Server_Settings\document_root = GetPathPart(ProgramFilename()) + app\dirRelitive ;set a root to be safe
app\Server_Settings\access_control_list = "-0.0.0.0/0,+127.0.0.1"  ;block all and then only allow connection from 127.0.0.1
app\Server_Settings\error_log_file = GetPathPart(ProgramFilename()) + "error.log"    ;set logs 
app\Server_Settings\access_log_file = GetPathPart(ProgramFilename()) + "access.log"  ;set logs 
app\Server_Settings\static_file_max_age = "0" ;nothing should be cached when debugging                     
app\Server_Settings\ssl_certificate = GetPathPart(ProgramFilename()) + "ssl_cert.pem" 
app\Server_Settings\enable_keep_alive = "yes" 

;app\Server_CallBacks\begin_request = @cb_begin_request() ;these callbacks are usefull for debug info 
;app\Server_CallBacks\end_request = @cb_end_request()     ;or if you want to overide civetweb, declared in civetweb.pbi 
;app\Server_CallBacks\connection_close = @cb_connection_closed() 
;app\Server_CallBacks\log_Access = @cb_log_access() 
;app\Server_CallBacks\log_message = @cb_log_message()  
;app\Server_CallBacks\http_error = @cb_http_error() 
;app\Server_CallBacks\init_connection = @cb_init_Connection() 
;app\Server_CallBacks\init_context = @cb_init_Context() 
;app\Server_CallBacks\exit_context = @cb_exit_context() 

;This can be used to pack your website to ezp so you can include it in the datasection
;it will be named as the folder of the app\dirRelitive setting with an extension of .ezp eg: www.ezp 
;Civet_Server_Create_Pack(app)

If Civet_Server_Start(app,80,443,"127.0.0.1")
  If OpenWindow(0,0,0,1024,600,"PB_Civet_Server_in_memSSL example of in memory embedded PB_Civet_Server " + app\Server_Settings\listening_ports,#PB_Window_SystemMenu |#PB_Window_SizeGadget |#PB_Window_MaximizeGadget) 
    WebGadget(2,0,0,1024,600,"https://127.0.0.1:"+ app\https_port) 
    BindEvent(#PB_Event_SizeWindow,@Resize()) 
    Repeat
    Until WaitWindowEvent(30) = #PB_Event_CloseWindow
  EndIf
  Civet_Server_Stop(app) 
Else 
  MessageRequester(GetFilePart(ProgramFilename()),"Cant start Server") 
  End
EndIf   

DataSection 
  Index:
  IncludeBinary "www.ezp"
  EndIndex:
EndDataSection 
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: PB_Civet_Server demo

Post by Kwai chang caine »

The two EXE works perfectly on my W10 X64 8)
And the most recent i have installing, it's just the "Microsoft visual C++ 2015" :wink:
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server demo

Post by idle »

I wasn't sure if it'd have a dependency on the 2017 VC redistributables
The static lib was built to target win7 sdk but when it came to resolving external symbols
I had to add two of the 2017 VC libs to the static lib as there was no other way to do it
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server demo

Post by idle »

Added SpiderBasic demo Waponez in the SSL version, it'll load it into your browser as http

Linked in the "What does it do" section under "Host or deploy Spider basic apps"
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: PB_Civet_Server demo

Post by Kwai chang caine »

Waponez works here :D
Thanks 8)
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server demo

Post by idle »

Thanks KCC,

I've run out of time so have posted the project as is v0.8 with static libs and examples for the community to pick over
I would appreciate some feed back.
See first post

Added github project, hope it's right
https://github.com/idle-PB/pb_civet_server
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: PB_Civet_Server

Post by Kwai chang caine »

Very very nice
I have compiled on W10 X64 / v5.61 x86 and that works like usually
I have also tested your unpacker and extract the site with succes :D

After i have compiled on another machine W7 X86 / v5.61 with the same result, always perfect 8)

Splendid job IDLE, i not know this CivetServer but with it, it's very simple to embed a full site in an exe local
Finally, if i have good understand it's exactely the reverse of everybody do, translate all the EXE in WEB (Office, PDF, etc...) :shock:
I like it thanks a lot for sharing this great tool 8)

I wondered, are you forcing to put "www.ezp" in DATA ?
Is it possible to have several packing site and choose the pack you want after have running the apps ?
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server

Post by idle »

you kind of see it a little differently once you've realized it's served from memory.
So yes you could pack an entire website into an exe including pdfs, docs, spider basic apps or any other mime type, though it hasn't got range handling or chunked transfers capabilities added for in memory yet, so you wouldn't be able to seek through a movie for instance but It should do that from file. The in memory function bypasses a lot of civetwebs internal handling, so it's just the bare minimum to make it work and changing civetweb itself isn't a good option.
Kwai chang caine wrote:I wondered, are you forcing to put "www.ezp" in DATA ?
Is it possible to have several packing site and choose the pack you want after have running the apps ?
Yes, it would be possible to abstract it to add dynamic packs though it's probably not really necessary
As you can just add multiple websites to the subdirectory of the sites root.
and use uri paths as /foo/index.html /bar/index.html or a get queries /index.html?s=foo or /index.html?s=bar
It's designed to work as a mirror of your servers root file directory so you can edit and test your websites assets from file while developing it then pack it to .ezp to embed in a datasection, set the address, recompile and deploy it with no file dependencies.

There's a lot of potential for it.
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: PB_Civet_Server

Post by Kwai chang caine »

Aaaah yeeees ! i have not thinking of that :oops:
So when you adding a new page, you must just recompiling the "www.ezp" with your packer and it's done :D
It's new for me, but i'm sure a day i have an idea for use your splendid work, i love applications using WEB and EXE together :D
Thanks for your explanations 8)
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server

Post by idle »

Added a small update to discover available ports should the ones you specify be in use
start up is changed to

Civet_Server_Start(app,80,443,"127.0.0.1")

Plus if you only want to have the server, serve locally to the machine it's running on
you just have to set the ACL -block, +allow
app\Server_Settings\access_control_list = "-0.0.0.0/0,+127.0.0.1"
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5342
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: PB_Civet_Server

Post by Kwai chang caine »

Thanks IDLE 8)
ImageThe happiness is a road...
Not a destination
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server

Post by idle »

I'm currently adding a simple server side preprocessor, it expects tags as

Code: Select all

<?PB FunctionName()/>
And the callback functions are simply

Code: Select all

Runtime Procedure FooBar(*app.civet_server)
    ProcedureReturn UTF8("<h1>Hello FooBar()"</h1>") 
EndProcedure  


So you can use it to do portions of a page like the header or footer or to fill out a table with data
or create one off assets like images or whatever.

You have the server state and any query tokens in the *app pointer so you shouldn't need anything else
question is what to call the pages index.pbh?

It will still take a couple of days to finish integrating and do the sanity checks with an example.
So you will have a choice client side ajax with a js framework of your choice and or server side processing with PB functions
Windows 11, Manjaro, Raspberry Pi OS
Image
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: PB_Civet_Server full stack web solution for PB

Post by idle »

Might be a bit premature and I'm out of time, added the pb tag preprocessor
and basic example.

Preprocessor files are named ".pbh" and your tags are <?PB SomeFunctionName() />

example

Code: Select all

Runtime Procedure ElementsFillTable(*app.Civet_Server) 
  Protected sout.s,sum.f,a  
  sout = "<div><H2>Output from ElementsFillTable() Callback</h2></div>"
  sout + "<div class='table-wrapper'><table><thead><tr><th>Name</th><th>Description</th><th>Price</th></tr></thead>" + #LF$
  sout + "<tbody>" + #LF$ 
  For a = 1 To 20 
    sout + "<tr><td>Item " + Str(a) + "</td><td>Ante turpis integer aliquet porttitor.</td><td>" + StrF(a * 2.99,2) + "</td></tr>" + #LF$
    sum + (a * 2.99)  
  Next 
  sout + "</tbody><tfoot><tr><td colspan='2'></td><td>$" + StrF(sum,2) + "</td></tr></tfoot></table></div>"
  ProcedureReturn UTF8(sout) 
  
EndProcedure  

Windows 11, Manjaro, Raspberry Pi OS
Image
Post Reply