It recurses based on a start point, so if I navigate through a list of posts by say 10 pages and kick it off there I will get another 65k html files and then just copy them in. The date (to do it in years) would be based on the last post as that determins the depth it's sunk into the forum
I have a parser that is then chopping out the unneeded html and the parts that links in gfx and buttons and puts it into SQLite. This works okay but there is still more crap that my parser needs to handle. I have a table with author, subject, date, body etc (cause I REEEEEAAALLY want to search by post title
) and the viewer tool is where I was up to last time I was putting effort into this (a couple of months ago).
I want to create a web form shown locally in the web gadget, you put your search in and sends to a fake url (eg
http://pbforum?title="hello") and the gadget notification callback can catch this, process it, peel the search params out, look up sqlite and return the rows found in a web page with links to the articles which will also point to a fake URL with the DB ID and be caught by the notification callback. I can even put the 15-post-at-a-time pages back together now (or atleast link them up)
This way I can keep the original web formatting of the post for display.
That's the idea but I ran out of time when my 2nd son was born
I'll pick it up again when I get some time. I have a download of tips&tricks only at the moment (done in april), teleport pro can still catch that whole forum in one sweep and it's the main one I want.
I have a filter done for teleport pro to pass these forums so that you can stay within a topic and not follow profile links etc. I have a copy of the tips and tricks forum and pass tool to store it in sqlite and show it in a browser control (still VERY buggy and unfinished though).
