Page 1 of 2

qDoc 1.1

Posted: Fri Dec 09, 2011 2:41 pm
by srod
10th Dec 2011. - Update.

Version 1.1 has the following changes :
  • Upgraded the \ReadString() method to use it's own internal buffers and so should be much faster now.
  • Added two methods to the qDoc_FileObject class : \ReadOLEString() and \WriteOLEString().

    Consider using these new strings if working with a lot of string data as they have the potential to operate much faster than 'regular' strings (particularly when reading the data back from a document). An OLE string is simply written as 4-bytes containing the length of the string (in bytes) followed by the bytes of the string in Unicode format without any null terminator. They are fully cross-platform (despite their name!)

=================

Hi,

for those familiar with my comDoc utility (http://www.purebasic.fr/english/viewtop ... lit=comDoc) here is a fully cross platform library offering the same facilities; namely, compound documents.

For those not familiar with the original comDoc, the following comes from the qDoc manual :
qDoc allows Purebasic applications to create proprietary documents which act like an entire file system in that a single document can contain any number of disparate 'files', files which can be accessed in a ‘random’ fashion. In this way, qDoc can be said to create compound documents.

Each such document itself occupies just a single disc (or memory) file (regardless of how many 'files' it contains!)

For example, imagine a word processing application. Such an application may wish to save not just the text entered by the user within it's documents, but it may also need to save images, charts, spreadsheets and so on within the same document and in such a way that each separate image or chart etc. can easily be retrieved and altered etc. This is exactly what qDoc facilitates and in a fashion almost identical to Purebasic’s file library.

Open a qDocument, open one of it’s files and then proceed to read and/or write bytes, longs, strings… etc. from/to the file.
Features...
  • Windows/Linux x86 and x64 compatible and Mac OSX compatible
  • Unicode compatible
  • Simple to use OOP api
  • Create new 'files' in individual documents
  • Delete files from individual documents
  • Enumerate all items in a compound document
  • An extensive list of methods for working with individual files (ReadByte(), ReadWord(), ... WriteByte(),... WriteString()... etc.)
  • Batched error reporting. Saves checking for errors after each individual write operation etc.
Porting existing comDoc applications to use qDoc will be very straight forward. The only thing that qDoc lacks is a 'CatchDocument' facility. There is no way around this I'm afraid except to dump the document from memory to a temporary disc file and to then load it up either remaining as a disc file or as a memory based document.

Please see the nxSoftware site for the download etc.

Regards.

Re: qDoc

Posted: Fri Dec 09, 2011 6:52 pm
by ts-soft
Thx Stephen, :D

You are using integer for loc() and lof(), is this a limitation?

Greetings - Thomas

Re: qDoc

Posted: Fri Dec 09, 2011 11:54 pm
by electrochrisso
Thanks from me too srod, your code is always good to learn from. :)

Re: qDoc

Posted: Fri Dec 09, 2011 11:56 pm
by netmaestro
Thanks from me too, very useful tool!

Re: qDoc

Posted: Sat Dec 10, 2011 12:42 am
by srod
ts-soft wrote:Thx Stephen, :D

You are using integer for loc() and lof(), is this a limitation?

Greetings - Thomas
Must admit that never occurred to me! :) I will take a look at that tomorrow as I will be taking steps to improve the string processing anyhow.

Thanks all.

Re: qDoc 1.1

Posted: Sat Dec 10, 2011 12:51 pm
by srod
10th Dec 2011. - Update.

Version 1.1 has the following changes :
  • Upgraded the \ReadString() method to use it's own internal buffers and so should be much faster now.
  • Added two methods to the qDoc_FileObject class : \ReadOLEString() and \WriteOLEString().

    Consider using these new strings if working with a lot of string data as they have the potential to operate much faster than 'regular' strings (particularly when reading the data back from a document). An OLE string is simply written as 4-bytes containing the length of the string (in bytes) followed by the bytes of the string in Unicode format without any null terminator. They are fully cross-platform (despite their name!)
============

@ts-soft : I thought about the integer data-types being used for Loc() and Lof() and have decided to leave them be for the time being (it is not a simple matter of switching for quads as of course other changes would need to be made, nothing major though I have to admit!)

The integers will only prove problematic in 32-bit apps which work with extremely large files (in excess of 2GB). I don't mean documents here (which can be up to 14 TB in size), but individual files within documents. No problems with 64-bit apps.

Switching to quads will of course slow things down a wee bit when running under x86 and so I am reluctant to make any changes in this respect. :)

Re: qDoc 1.1

Posted: Sun Dec 11, 2011 6:00 am
by said
[quote="srod"]10th Dec 2011. - Update.

Version 1.1 has the following changes :
  • Upgraded the \ReadString() method to use it's own internal buffers and so should be much faster now.
  • Added two methods to the qDoc_FileObject class : \ReadOLEString() and \WriteOLEString().

    Consider using these new strings if working with a lot of string data as they have the potential to operate much faster than 'regular' strings (particularly when reading the data back from a document). An OLE string is simply written as 4-bytes containing the length of the string (in bytes) followed by the bytes of the string in Unicode format without any null terminator. They are fully cross-platform (despite their name!)
Thanks Srod, very useful tool indeed.
Good to notice you are more involved into cross-platform stuff, i read somewhere on the forum you are about to announce a cross-platform grid soon, is that true?
I believe your know how and coding style are much appreciated within the PB community (at least this what i feel)

said

Re: qDoc 1.1

Posted: Sun Dec 11, 2011 11:52 am
by srod
Well, not exactly; what time I have for coding is split between various projects; but a cross-platform grid is high on that list for sure. 'Any time soon... ?' I have no time frame in mind right now because time is so limited right now. :)

Re: qDoc 1.1

Posted: Mon Dec 12, 2011 8:43 am
by jamirokwai
Hi srod,

great include! Thanks.

Could you add some compression routines, too?
Could come in handy for two reasons:
- files would be smaller
- if you don't like others to poke around inside the file, there is some kind of privacy :-)

Re: qDoc 1.1

Posted: Mon Dec 12, 2011 10:13 am
by srod
Hi jamirokwai,

compression is not something I am in need of at this time. You can always VACUUM a qDoc to make the file smaller. As for privacy; the internal files relevant to an application are binary and, of course, proprietary, and so they are no more readable than any other binary file. If you wish to obfuscate their contents completely then that is something best built in at the application level rather than something qDoc need offer. Sure there are a couple of files inside each qDoc which any viewer can read, but the information contained within these files is of little use to anyone (except qDoc itself of course). :)

The truth is that qDoc, like most of the utilities I release, arose out of my own needs and at the moment it is perfect for what I need it for. If I do find myself fretting over file sizes at some point then I shall then rip the code apart.

:)

Re: qDoc 1.1

Posted: Mon Dec 12, 2011 5:49 pm
by jamirokwai
srod wrote:Hi jamirokwai,

compression is not something I am in need of at this time. You can always VACUUM a qDoc to make the file smaller. As for privacy; the internal files relevant to an application are binary and, of course, proprietary, and so they are no more readable than any other binary file. If you wish to obfuscate their contents completely then that is something best built in at the application level rather than something qDoc need offer. Sure there are a couple of files inside each qDoc which any viewer can read, but the information contained within these files is of little use to anyone (except qDoc itself of course). :)

The truth is that qDoc, like most of the utilities I release, arose out of my own needs and at the moment it is perfect for what I need it for. If I do find myself fretting over file sizes at some point then I shall then rip the code apart.

:)
Hi srod,

sounds familiar. Many of the things, I code, also arise from my needs :-)
Extending your code with obfuscation (e.g. RC4) and compression sounds interesting to me. If I find enough time, I will have a look...
Shouldn't be that difficult! :mrgreen:

Thanks!

Re: qDoc 1.1

Posted: Wed Nov 28, 2012 4:52 pm
by Poshu
Hi srod!
The qdoc part of your site is down, could you provide us with a new download link please?

Re: qDoc 1.1

Posted: Wed Nov 28, 2012 4:59 pm
by srod
Done.

Re: qDoc 1.1

Posted: Wed Nov 28, 2012 5:01 pm
by Poshu
That was fast °_°; Thanks :D

Re: qDoc 1.1

Posted: Wed Nov 28, 2012 5:07 pm
by srod
Poshu wrote:That was fast °_°; Thanks :D
Just happened to be in the neighbourhood! :wink: