CodeDB 0.3.2b

[idle]

Windows x64 ASM and c backend
CodeDB builds a DB of your procedures and Macros by scanning your drives or paths.
Prefix search for procedures and macros: "GetW"
wildcard searches of procedures or macros: "*Get"
Full text prefix search's with AND OR of the procedures or macros "open AND close"



download
https://atomicwebserver.com/codeDB.zip

DB is stored in temp folder for now

1) un zip to where ever
2) Compile CodeDb.pb with windows x64 ASM backend
3) Create exe and set it as a compiler tool with a short cut of your choice eg "ctrl + f12"
4) Scan a path, drive or ThisPC for all drives
4a) Click file menu add another path or drive

5) Enter procedure prefix in text box
6) click found procedures in list, double click to set to clipboard
7) click file button top right to open in IDE has to be compiler tool to do it

The app is a simple test of IMPBDB (in memory Purebasic DB) which is a WIP and uses JSONRuntimeStrucutres

[Bisonte]

Sorry, the inludefile “IMPBDB.pbi” is missing in the zip archive...

[idle]

Sorry, the inludefile “IMPBDB.pbi” is missing in the zip archive...

It's not ready for release yet so it's only the exe at the moment

[idle]

fixed a few bugs in the parsing
added the code to compile codeDB.pb
Bare in mind the DB engine is a WIP and will only currently works with Windows x64 ASM backend

[Skipper]

Hi Idle,

what does "Delete DB" actually do?

I let the application scan over my entire pc, and it reserved just shy of 49 Mb of memory after the scan.
After deleting the DB, this did not change.

Letting it then re-scan the entire pc once more, memory requirements went up to 62 Mb.

Cheers
Skipper

[idle]

Hi Idle,

what does "Delete DB" actually do?

I let the application scan over my entire pc, and it reserved just shy of 49 Mb of memory after the scan.
After deleting the DB, this did not change.

Letting it then re-scan the entire pc once more, memory requirements went up to 62 Mb.

Cheers
Skipper

At the moment it won't delete until you restart the program
Also when it does a rescan or you add a new path it will inflate the size, It's a WAL (write ahead log) I haven't added the compaction yet. When an item is added that already exists and matches It will store the metadata and a pointer to the existing item rather than the data but it will still keep growing by the size of the metadata until it goes through compaction and dedup. I'm still working through the db engine and have a bit to do before going back and addressing the WAL growth, it's in part that I can only compare an item once it's serialized to JSON not when its in memory as I haven't added the runtime structure reflection required to compare the binary structures at runtime. It's supposedly coming at somewhen but I doubt it's on the high priority list of features to add and I'm also not really in any rush.

I intend to use the CodeDB application to implement full text searches in the DB engine.

I also found that if you scan the folder "ThisPC" you can't open the file in the IDE as it doesn't give you the absolute path, I will look at that.

[Skipper]

As to your last sentence: scanning "This PC" results in double entries on my side: one with an absolute path, the other with a relative path - both pointing to the same resource.

cheers
Skipper

[idle]

I fixed the count and root path issue so it will scan all drives in that case
The DB will still grow on rescans for now as it's supposed to, so you can do rollbacks to prior versions, though I will look at that again later.

The delete is still only done after you close the program and still stored in Appdata\temp

[AZJIO]

I got an error on line 232

Code: Select all

token + strChr(*char\u) 

log

Code: Select all

[09:52:32] [ERROR] CodeDB.pb (String: 232)
[09:52:32] [ ERROR] Memory access is not possible. (write error at 18446744073709551568)
[09:52:36] The program is destroyed.

[idle]

it's reading past the end of the string, not sure what's causing it maybe the file string format?
Maybe I need to add ReadStringFormat and set the flag in ReadString line 272 . I will look in the morning

[AZJIO]

I use the coding #PB_UTF8 in the source, which is by default when opening a file.
The problem can #CRLF$? I used the sources from the Linux folder that use #LF$.

Code: Select all

Procedure.s _findString(str.s,find.s,pos=0)
    Protected *char.Unicode,token.s,p1,s1=33
    p1=pos
    If @str = 0
      ProcedureReturn ""
    EndIf

[idle]

Thanks AZJIO, hopefully I can get on now and implement full text searches.

[idle]

V0.2a

Added wildcard search *foo will find foo anywhere within a procedure or macro name

Added Full Text search of Procedures and Macros with AND and OR, searches are prefix searches, wildcard isn't supported in that mode yet.

DeleteDB doesn't need a restart now.

You might need to nuke the old DB first line 463 then rescan.

[idle]

v0.2.1a

Added Sorting to wildcard and full text searches

full text search takes a while to build and it will raise a message box when it's ready, reporting how many keys were added and the memory size of the index. In my case it's indexed 3,185,104 tokens that sums to 113,661,384 bytes and the result index uses 116,786,192 bytes, this might seem like a lot and well it is but consider a hash table of the same, it would be at least 1.5 times larger as it has to store the keys and allocate more space for the table so it doesn't collided. So that would be 3,185,104 * 1.3 * 16 + 113,661,384 where 16 is the size of a PB_MapElement and 1.3 is the estimated additional space required for the table to operate fully loaded which is generally 70% of the space.

It's starting to be quite handy considering I've got code all over the place and now I can easily find it.

[idle]

0.2.2b

Added Highlighting for Full test search