PureBasic Forums - English

Hi forum,

i have to handle a lot of data (> 50,000 entries) in a TreeGadget.

When starting my app reads the entries to be displayed from a flatfile. The entries are then to be organized in few major (few 1,000 entries each) and many minor nodes (up to 100-500 entries each).

I thought i could speed this up if i saved the entries of a tree - once constructed - as a separate text file and then reconstruct the tree from this file after restarting the app.

However, for a sample file of about 30,000 entries to be displayed it takes roughly 1.5 minutes of computing time - regardless of the method to construct the tree. Obviously, determing the entries and the level of the entries to be displayed at costs almost no time.

Now i wonder why it takes so long to just add entries to the end of a TreeGadget list. Is there a way to speed this up?? Is there a faster API function for trees???

thanx, searhin

Ok, there are several issues about the TreeGadget:

1st: on PB's side:
The Item handling functions (changing Text/State of Items) for
TreeGadget get slower, the more Items the Gadget contains. However,
the AddGadgetItem() command should not be slow at all at the moment,
as it doesn't contain any code that might be a bottleneck when dealing
with a high number of items.

Maybe you are also using a SetGadgetItemState() or something to change
the new items state or something that slows down the whole thing?

However, the TreeGadget code is beeing improved for v3.70, and all
commands should be faster then.

2nd: on M$ side:
Go have a look at the Remarks section here: http://msdn.microsoft.com/library/defau ... tcount.asp

First of all, this tells me, that the TreeView control on Windows is not
intended for that large amounts of data. I think the concept behind it is
more to dinamically add data to the Control, when the user wants to see
them, and free not needed recources, when the user collapses a Node.

Therefore it provides possibilities, to add subitems, only if they are visible
for example. However, implementing this stuff for purebasic would make
the whole TreeGadget a very complex thing, although parts of it will be
improved now (for example adding Items wherever you want ).

However, as to what I have read, the TreeView control is capable of
holding very large amounts of data, only counting them is buggy

What's the point now? I guess, that because the TreeView control isn't designed to be used with that large amouts of data, that's the reason why
it slows down so much. However, this is just a guess.

Another much bigger problem is, that at the moment, PB get's confused
when addimg more than 32767 Items, because the Item count gets
negative. All Items added behind the 32767th Item will get wrong Item
numbers by PB!


So much about the bad news, let's come to the good ones...

As I allready said, the code is beeing improved, so on v3.7, all commands
should be faster. The limit of data will also be solved, so that you will
hopefully get the whole 32bit range for Item numbers, so this won't be
a problem any more. (the max number of Items will then be 1073741823,
as this is the largest positive number a PB LONG can hold )
Another improvement will be the support of the Position parameter in
AddGadgetItem(), so the TreeGadget will get more dynamic.


My Tip for you now is... stay tuned for 3.7, should work better then

Timo

freak wrote:(the max number of Items will then be 1073741823,
as this is the largest positive number a PB LONG can hold )

Hmm, should be double that +1, shouldn't it

Hehe, damn misstyping on the calculator
I'd better write a program for that

Timo

Acording to my docs it's +2147483647 and not 1073741823, in any case it's should be an unsigned long, not signed.. which is 4294967295.

@freak

many thanx for this extensive reply

well, i'll check my code for SetGadgetState etc. and just wait for PB 3.70 (each new update is so exciting!). Maybe i split the tree so each major node gets its own tree in a PanelGadget (would require quite a lot of work though...)

btw, guess it doesn't matter if a tree's max item number would be 2^32 or 2^30...

> the TreeView control on Windows is not intended for that large
> amounts of data

Exactly. If you look at http://tinyurl.com/cu0k it states: "It is not
recommended that you add more than 32767 nodes to a TreeView
control. If you need more than 32767 nodes, you should consider using
multiple TreeView controls or a control more suited for larger amounts
of data". It also states that "performance will become extremely slow
as you add more and more nodes".

Yep, I was refering to that same article when saying that.

As I said, the concept of TreeView is more adding/removing Items, when
they are needed to be displayed. Look at the M$ Explorer for example, or
Regedit.exe. Regedit never scanns the whole Registry to create the tree, it
only scanns the Keys the user opens, and adds them to the tree.

The problem is, that handling these expand/collapse Events doesn't fit
PB's way to handle events, so I guess it won't be easy to implement that.

Maybe this will come some day, but not for v3.7 However, it's flexibility will
grow...

Timo

> I was refering to that same article when saying that.

Oh, sorry -- our links went to different pages so I didn't realise.