HTML5 Browser

[Kwai chang caine]

Hello Firace
Is your nice tool can a day export whole rendered HTML or/and text by a way (To file, Stdout, etc ...)

[firace]

Hello Firace
Is your nice tool can a day export whole rendered HTML or/and text by a way (To file, Stdout, etc ...)

Hi KCC,
to be sure I understand, could you clarify what you mean by "whole rendered HTML" (perhaps with an example)?

[firace]

downloading = 503 Service Unavailable

Download link should be OK now.

[PureLust]

Hmmm .... looks very interesting ... but get blocked by my Antivirus (Panda).

Not because it's recognized as a virus (scan of exe is fine), but because of behavioral analysis when I try to run it.

[juror]

downloading = 503 Service Unavailable

Download link should be OK now.

Got it.

Thanks again. It is a wonderful "big name" browser alternative.

[Derren]

On startup. It takes 1-2 seconds, then this message appears. Clicking "OK" terminates the program.
I don't see any website loaded or URL in the address bar.

Whatever you did in the last couple of months, I can start the program just fine now.

And I don't know what Microsoft did to their browser. I hate using IE, not only because it's behind web-standards, but because it's slow as a snail. Forkle is not. It loads and navigates websites seemingly faster than the regular IE. WOW

[Kwai chang caine]

Hello Firace
Is your nice tool can a day export whole rendered HTML or/and text by a way (To file, Stdout, etc ...)

Hi KCC,
to be sure I understand, could you clarify what you mean by "whole rendered HTML" (perhaps with an example)?

I use this words because i have see in numerous page of web he is used
I search to export all the final HTML the browser have to writing the page, the page HTML base obviously, but mainly, the include JS, and all the DOM, exately like you can see in the DOM inspector

[firace]

On startup. It takes 1-2 seconds, then this message appears. Clicking "OK" terminates the program.
I don't see any website loaded or URL in the address bar.

Whatever you did in the last couple of months, I can start the program just fine now.

And I don't know what Microsoft did to their browser. I hate using IE, not only because it's behind web-standards, but because it's slow as a snail. Forkle is not. It loads and navigates websites seemingly faster than the regular IE. WOW

Thanks for the great comment! I have spent quite a bit of time on improving performance and usability, as I'm using it as my daily web browser.

[firace]

Hello Firace
Is your nice tool can a day export whole rendered HTML or/and text by a way (To file, Stdout, etc ...)

Hi KCC,
to be sure I understand, could you clarify what you mean by "whole rendered HTML" (perhaps with an example)?

I use this words because i have see in numerous page of web he is used
I search to export all the final HTML the browser have to writing the page, the page HTML base obviously, but mainly, the include JS, and all the DOM, exately like you can see in the DOM inspector

I see. Next build will have two experimental commands to extract DOM dump and iframe DOM dump with keyboard shortcuts. Perhaps you will find them useful.

[Kwai chang caine]

Perhaps you will find them useful.

WHAAAAAAAAT !!!!!!



You can do that with your nice browser ????
Ask to a blindman.... if he want see !!!!!
I search this style of function in all the web, since several years without succes
That can adding a great popularity to your browser for everyone who whant scrapping, and the scrapping it's the futur, because all is on the web now

DOM dump and iframe DOM dump with keyboard shortcuts

Yes it's a good idee for writing a file text for exampler manually, or put in the clipboard text
But if you can, i think it's better to also adding a sending of all the page by PIPE stdout with an option parameter at the running
Like CHROME with his "dump_dom" option, like what, i'm not the only one who search this function in a browser because FF and CHROME have it
But the problem with this two dump it's he run only at the launch of the browser,and furthermore only in headless mode, so invisible
So how can i enter a login password on an invisible page
Furthermore the stdout is only filling at the first launch, the next page ...nothing
And nobody have thinking to always filling the stdout when a change come, mainly the url

Imagine, you do this parameter option "--dumpDOM" or "--dumTEXT" you replace all the several codes of nodejs (pupetter, chrome-remote-interface, etc ...) and another ways, about this subject, it's amazing
A runprogram or console, with "--dumTEXT" option and "roll my hen" (french expression ) we can recording all the text of all sites, if obviously you can read the text of JAVA and FLASH plugin

I don't know what you can do, but a fork between the writing in the browser and the PIPE, and your tools can be the white bread of several scrapping users, and believe me...they are numerous since i read the thousands pages about this subject

Il after, you can adding an option for can sending somes simples commands, style "WriteId(#IdObject") for filling fields, ClickLink(#IdLink"), or better again ExecuteJS("MyJSCommand").... you are the king of the petrol
With this option, you replace the AUTOMATION of IE, the remoting of GeckoDriver and dozen of other methods, each more complicated than the other and impossible to understand by the not "EINSTEIN" peoples
It's not for nothing, that the softwares like AUTOIT exists......it's for remote others program easily....

So thanks a lot by advance for your interest to my needs
I wait your futur release, like a dog waiting for his master... behind the gate with his tail who stirs

[firace]

@KCC: Thanks for your hilarious post
But please don't set your expectations too high, because:
1. I am not sure I have the skills to develop all that
2. I will have limited free time in the next few days/weeks
3. I have to put more focus on other projects, to be able to pay my bills

That said, you can check this quick test build and see if it extracts the data you are after. (Just navigate to your web page, and then use Ctrl+Shift+U to copy the generated DOM to the clipboard)
At least I will know if this catches the data you need, before I go any further.

Test build (temporary link): https://www.radsix.com/forkle_x64_test.zip

Notes:
Only tested on Windows 10 x64 so far
Won't catch text generated by plugins
Won't catch iframes violating SOP (same-origin policy)

[Kwai chang caine]

Yeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeessssssssssssssssssssss !!!!!
That wooooooorks !!!!



It's a miracle ....i have understand now.....
your real name is not FIRACE.....but FIRACLE !!!

Won't catch text generated by plugins

Wrong !!! your splendid jewel catch all the DOM, even the text in the JAVA pluggin
Because i'm not really sure, but there are some shit of JAVA in this page
Thanks to you, i have a little bit of hope now

1. I am not sure I have the skills to develop all that
2. I will have limited free time in the next few days/weeks
3. I have to put more focus on other projects, to be able to pay my bills

Don't worry, i understand you have others things to do, than the littles problems, of again more little, KCC
And i want to already thanks a lot for have create, just for KCC, this provisional function
You are an angel, and i love you nothing for that



Yesterday i have continue my search, like all the days, and also find two tools
QTWEB a browser open source, and apparently with not so big source code
http://www.qtweb.net/
But, he have crash on my bank page

And an HTML parser open source too, but mainly with a simple use, an exe and stdout like i love, and even more a DLL (Perhaps can using in PB)
https://www.html-tidy.org/
Because you imagine, you are the first to have create a webbrowser, who can SIMPLY scrapping, and again more strong.....a part in PB

All yesterday, i have try to retrieve all my olds codes for recompile Firefox
All the day i have try to download the FF source, and after running BOOTSTRAP and it's here that not works, i have a simple problem of PATH and i not have found where

Because like i have already try since several years, nearly all solutions for remote browser, since i have abandonned the IE ole automation who worked fine, but it's IE.....
I have try a new time to recompile FF, after weeks and weeks of works, you never believe me but little KCC succeeded to have his own FF
And even more, i can modify after hours of search in numerous page of code, to reactivate JS in the URL.....
Because FF is in C++ or this style of language, but the big part of the functions is in JS
Like the PB IDE in C and after in PB for several options

You surely know that, but you can easily navigate in a part of internal JS page by the pseudo FF protocol "resource:///"
Write "resource:///" in the FF URL and you see the tree of what you can explore in internal JS, when i have found that, i fall of my chair
For exampler this page about "resource://gre/modules/AboutPagesUtils.jsm"
And after, my search are stopping here....but i know it's possible , because if little KCC can do something...all the members of all the world can also do that

My real goal when i have beginning to recompile FF, it's create a simple fork, without all protections MOZILLA adding years after years

- Reactivate JS injection in URL
- Reactivate Scratchpad and modify it for can handle it easily
- Create a copy of DOM by stdout PIPE, and also a JS injection by stdin
- Delete the locking of permanent addons with Mozilla certificate
- Delete numerous functions for simplify this gaz factory, for just keep the essential, a WEBBROWSER exactely like your splendid FORKLE

But alone, i'm not also strong for do all that apparently, even if i have already have littles success

In fact, just create a simple browser not for navigate all the days with it, but just for remote it easily, and scrapping all is possible with our love PB, by simples commands
Not all this monsters of NodeJS (Pupeeters and numerous others modules), Selenium, Python, etc ....
Just a PIPE stdin/out like in the old time, like this you can use it with all languages
I'm tired to see all this modules, addons, pluggins, GeckoDrivers, XulRunner, big language OOP, for just read one line of text
Then i'm sure it's always possible, even in the last version of FF or other webbrowser to remote it with a simple line, the proof ??? you have do it with your amazing browser
It's time, to return to the simplicity, she have not disappear...she is under tons of useless layers of new codes, protocol, etc ...
In fact adopt the "FRED attitude" keep the powerfull.....let the complexity for the young programmers, and the big heads, who like to masturbate your brain for nothing

And why not create a new wave ????
After all this tons of new technology more and more complex, for have the same thing than before, but in more hard and long
Create the "PAS"....Programming in Attitude Simple
If it's too much hard for beginners....go out

Again thanks a lot for your nice present
Have the best day of the world

[firace]

Yeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeessssssssssssssssssssss !!!!!
That wooooooorks !!!!

...
...

Again thanks a lot for your nice present
Have the best day of the world

Glad to see that it worked and made you so happy!
Miracles do happen from time to time

Unfortunately I can't go much further now (family and medical problems to deal with), but good luck with your search, and maybe I will be able to help more when I have more time.

Have a great day too

[Kwai chang caine]

No problem, thanks for your kind help

[PureLust]

Works fine now after I contacted Panda ... and WOW ... that's impressive !!!

I can imagine, you will not publish the source for that, but would there be any chance, you release this as an DLL or something like this, so one could include this into own Projects (e.g. as replacement for the WebGadget())?

As you might know, the communication with the WebGadget() is very limited (Events, Content, etc.).
With your own written Browser, there might be an horrendous amount of information your Browser could provide.

[EDIT]A little Question:
It looks like, you use the IE11 Engine for your Browser so far.
Do you use the genuine WebGadget for this, or did you create the WebView-Element on your own (e.g. by using API-Calls to IE-Engine, etc.)?