It is currently Tue Jun 18, 2019 11:56 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: lightning memory mapped DB
PostPosted: Thu Aug 02, 2018 5:25 am 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3377
Location: New Zealand
Anyone interested in LMDB, lightning memory mapped DB
I had a nice day with much gnashing of teeth building it with VS for windows and I'm just about to start getting my head around it
and would like some help testing and wrapping it.

https://symas.com/lmdb/

Quote:
AN ULTRA-FAST, ULTRA-COMPACT, CRASH-PROOF KEY-VALUE EMBEDDED DATA STORE.
Symas LMDB is an extraordinarily fast, memory-efficient database we developed for the OpenLDAP Project. With memory-mapped files, it has the read performance of a pure in-memory database while retaining the persistence of standard disk-based databases.
Bottom line, with only 32KB of object code, LMDB may seem tiny. But it’s the right 32KB. Compact and efficient are two sides of a coin; that’s part of what makes LMDB so powerful.


bench marks
http://www.lmdb.tech/bench/microbench/

I built v0.9.22 from the openldap project.
https://github.com/openldap/openldap/tr ... es/liblmdb

What's it uses, it replaces the need to mess around with SQL for starters key's can be binary or strings
It's persistent by design with full acid transactions and being memory mapped is fast.
So you can easily use it as a DB for you application or to add persistence for your applications state

1st test
https://www.dropbox.com/s/unz1m8gkphcou ... 2.zip?dl=0

_________________
Got winter blues?
Enjoy a Caravan Trip into, "The Land of Grey and Pink", wine and punk weed optional!
https://www.youtube.com/watch?v=9hmFzGTxod4


Top
 Profile  
Reply with quote  
 Post subject: Re: lightning memory mapped DB
PostPosted: Fri Aug 03, 2018 2:02 am 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3377
Location: New Zealand
1st test, don't all rush to say anything constructive, like who needs a DB anyway!

https://www.dropbox.com/s/unz1m8gkphcou ... 2.zip?dl=0
make sure you compile with create temporary exe in source directory or you'll have a db in your root

_________________
Got winter blues?
Enjoy a Caravan Trip into, "The Land of Grey and Pink", wine and punk weed optional!
https://www.youtube.com/watch?v=9hmFzGTxod4


Top
 Profile  
Reply with quote  
 Post subject: Re: lightning memory mapped DB
PostPosted: Fri Aug 03, 2018 11:18 am 
Offline
Addict
Addict
User avatar

Joined: Sat Apr 26, 2003 2:15 pm
Posts: 801
Location: Cuernavaca, Mexico
idle wrote:
1st test, don't all rush to say anything constructive, like who needs a DB anyway!
:mrgreen:

Thanks Idle.

Spent an hour yesterday at the Symas web site reading over the documentation.
Certainly looks usable in PB.

Thanks for converting the library for PureBasic users, now I can dig a little deeper.

Having a multi-CPU machine I'm especially interested in...

Code:
;*   The library is fully thread-aware And supports concurrent Read/write
;*   access from multiple processes And threads. Data pages use a copy-on-
;*   write strategy so no active Data pages are ever overwritten, which
;*   also provides resistance To corruption And eliminates the need of any
;*   special recovery procedures after a system crash. Writes are fully
;*   serialized; only one write transaction may be active at a time, which
;*   guarantees that writers can never deadlock. The database Structure is
;*   multi-versioned so readers run With no locks; writers cannot block
;*   readers, And readers don't block writers.


Thanks again... put me on your list.

_________________
- It was too lonely at the top.


Top
 Profile  
Reply with quote  
 Post subject: Re: lightning memory mapped DB
PostPosted: Fri Aug 03, 2018 6:04 pm 
Offline
Addict
Addict
User avatar

Joined: Sun Nov 05, 2006 11:42 pm
Posts: 4453
Location: Lyon - France
Thanks a lot IDLE for this another great job :shock:
Like usually i'm surely alone to not all understand several of your precious works, so much they are powerfull :oops:
And obviously this one is in the lot :wink:

But i have obviously, when even tested it, for see the effect and apparently that works on W10 X64 / v5.61 x86 :D

See yourself :
Quote:
LMDB 0.9.22: (March 21, 2018)
1001 duplicates
1001
what foo is this? 0 ok!
what foo is this? 1 ok!
what foo is this? 2 ok!
what foo is this? 3 ok!
what foo is this? 4 ok!
what foo is this? 5 ok!
what foo is this? 6 ok!
what foo is this? 7 ok!
.......
etc .....
......
what foo is this? 997 ok!
what foo is this? 998 ok!
what foo is this? 999 ok!
what foo is this? 1000 ok!
For better understand, if it's possible KCC understand something a day :oops:
The "lmdb.lib" on your zip is created by you for us? from the original C lib found here ?
https://github.com/LMDB/lmdb
is it that ?

_________________
ImageThe happiness is a road...
Not a destination


Top
 Profile  
Reply with quote  
 Post subject: Re: lightning memory mapped DB
PostPosted: Fri Aug 03, 2018 8:57 pm 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3377
Location: New Zealand
@blueb
It's a great looking lib and has been around for a while, there are numerous c test files in the git sources
which should be easy enough to convert.

@Kcc
No I used the version from
https://github.com/openldap/openldap/tr ... es/liblmdb
the page you linked to is their official lmdb mirror, but I couldn't get that to compile
and even the version from the openldap, I had to do a few function replacements, windows is a 3rd class OS according to the author.

It should be easy to compile on linux at least.

_________________
Got winter blues?
Enjoy a Caravan Trip into, "The Land of Grey and Pink", wine and punk weed optional!
https://www.youtube.com/watch?v=9hmFzGTxod4


Top
 Profile  
Reply with quote  
 Post subject: Re: lightning memory mapped DB
PostPosted: Sat Aug 04, 2018 3:39 pm 
Offline
Addict
Addict
User avatar

Joined: Sun Nov 05, 2006 11:42 pm
Posts: 4453
Location: Lyon - France
Thanks IDLE for your answer 8)
I'm always impressed when someone can use a LIB create for C and back it in PB :shock:
Quote:
windows is a 3rd class OS according to the author.
:lol:

_________________
ImageThe happiness is a road...
Not a destination


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye