Page 1 of 1
lightning memory mapped DB
Posted: Thu Aug 02, 2018 5:25 am
by idle
Anyone interested in LMDB, lightning memory mapped DB
I had a nice day with much gnashing of teeth building it with VS for windows and I'm just about to start getting my head around it
and would like some help testing and wrapping it.
https://symas.com/lmdb/
AN ULTRA-FAST, ULTRA-COMPACT, CRASH-PROOF KEY-VALUE EMBEDDED DATA STORE.
Symas LMDB is an extraordinarily fast, memory-efficient database we developed for the OpenLDAP Project. With memory-mapped files, it has the read performance of a pure in-memory database while retaining the persistence of standard disk-based databases.
Bottom line, with only 32KB of object code, LMDB may seem tiny. But it’s the right 32KB. Compact and efficient are two sides of a coin; that’s part of what makes LMDB so powerful.
bench marks
http://www.lmdb.tech/bench/microbench/
I built v0.9.22 from the openldap project.
https://github.com/openldap/openldap/tr ... es/liblmdb
What's it uses, it replaces the need to mess around with SQL for starters key's can be binary or strings
It's persistent by design with full acid transactions and being memory mapped is fast.
So you can easily use it as a DB for you application or to add persistence for your applications state
1st test
https://www.dropbox.com/s/unz1m8gkphcou ... 2.zip?dl=0
Re: lightning memory mapped DB
Posted: Fri Aug 03, 2018 2:02 am
by idle
1st test, don't all rush to say anything constructive, like who needs a DB anyway!
https://www.dropbox.com/s/unz1m8gkphcou ... 2.zip?dl=0
make sure you compile with create temporary exe in source directory or you'll have a db in your root
Re: lightning memory mapped DB
Posted: Fri Aug 03, 2018 11:18 am
by blueb
idle wrote:1st test, don't all rush to say anything constructive, like who needs a DB anyway!
Thanks Idle.
Spent an hour yesterday at the Symas web site reading over the documentation.
Certainly looks usable in PB.
Thanks for converting the library for PureBasic users, now I can dig a little deeper.
Having a multi-CPU machine I'm especially interested in...
Code: Select all
;* The library is fully thread-aware And supports concurrent Read/write
;* access from multiple processes And threads. Data pages use a copy-on-
;* write strategy so no active Data pages are ever overwritten, which
;* also provides resistance To corruption And eliminates the need of any
;* special recovery procedures after a system crash. Writes are fully
;* serialized; only one write transaction may be active at a time, which
;* guarantees that writers can never deadlock. The database Structure is
;* multi-versioned so readers run With no locks; writers cannot block
;* readers, And readers don't block writers.
Thanks again... put me on your list.
Re: lightning memory mapped DB
Posted: Fri Aug 03, 2018 6:04 pm
by Kwai chang caine
Thanks a lot IDLE for this another great job
Like usually i'm surely alone to not all understand several of your precious works, so much they are powerfull
And obviously this one is in the lot
But i have obviously, when even tested it, for see the effect and apparently that works on W10 X64 / v5.61 x86
See yourself :
LMDB 0.9.22: (March 21, 2018)
1001 duplicates
1001
what foo is this? 0 ok!
what foo is this? 1 ok!
what foo is this? 2 ok!
what foo is this? 3 ok!
what foo is this? 4 ok!
what foo is this? 5 ok!
what foo is this? 6 ok!
what foo is this? 7 ok!
.......
etc .....
......
what foo is this? 997 ok!
what foo is this? 998 ok!
what foo is this? 999 ok!
what foo is this? 1000 ok!
For better understand, if it's possible KCC understand something a day
The "lmdb.lib" on your zip is created by you for us? from the original C lib found here ?
https://github.com/LMDB/lmdb
is it that ?
Re: lightning memory mapped DB
Posted: Fri Aug 03, 2018 8:57 pm
by idle
@blueb
It's a great looking lib and has been around for a while, there are numerous c test files in the git sources
which should be easy enough to convert.
@Kcc
No I used the version from
https://github.com/openldap/openldap/tr ... es/liblmdb
the page you linked to is their official lmdb mirror, but I couldn't get that to compile
and even the version from the openldap, I had to do a few function replacements, windows is a 3rd class OS according to the author.
It should be easy to compile on linux at least.
Re: lightning memory mapped DB
Posted: Sat Aug 04, 2018 3:39 pm
by Kwai chang caine
Thanks IDLE for your answer
I'm always impressed when someone can use a LIB create for C and back it in PB
windows is a 3rd class OS according to the author.
