It is currently Fri Jul 03, 2020 9:17 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Using adresses as a key for AddMapElements ???
PostPosted: Sat Nov 23, 2019 4:00 pm 
Offline
Enthusiast
Enthusiast

Joined: Fri Oct 16, 2009 10:12 am
Posts: 653
Location: BE
Hi,

Is there a save and fast way to use adresses as a key for maps like this :
AddMapElement(map(), Str(adres))
Chr(47710280) is faster as Str(47710280) but does only two bytes.
And ... how to convert back ?
Val(MapKey(map())) ;<= Converts a string into a quad numeric value.

Thanks.

_________________
Yeah I know, but keep in mind ... Leonardo da Vinci was also an autodidact.


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Thu Dec 12, 2019 7:32 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Thu Apr 30, 2009 5:23 pm
Posts: 328
Location: Côtes d'Azur, France
The only fast and safe way is to use a custom hastable library that accepts numbers as key.
But it depends how fast you need to go, Pb map are pretty good even if str(adress) is somehow slow.

Remark, your adress evolves probably 4 by 4 so you can reduce its key by right shifting it by 2.

_________________
There are 2 methods to program bugless.
But only the third works fine.

Win10, Pb x64 5.71 LTS


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 2:02 am 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3525
Location: New Zealand
the pb map header suggests there is a numeric map key, this works on linux x64

LINUX x64
Code:
ImportC ""
  PB_AddNumericMapElement(*Map,key.i);
  PB_FindNumericMapElement(*Map,Key.i);
EndImport

Global NewMap dummy.i()

 Macro _NewMap(var,size=512) ;linux x64
   EnableASM   
   mov    rdi,size     
   !MOV    r8,rdi
   lea    rcx, var     ;pointer
   !XOr    rdx,rdx
   !MOV    rsi,21       
   !MOV    rdi,8         
   !SUB    rsp,32       
   !CALL   PB_NewMap
   !ADD    rsp,32
   DisableASM
 EndMacro
   
Global *mp
_NewMap(*mp,1024) 

Global *el.Integer
;
*el = PB_AddNumericMapElement(*mp,123)
*el\i = 123
;
*el = PB_AddNumericMapElement(*mp,345)
*el\i = 345
;
*el = PB_FindNumericMapElement(*mp,123)
Debug *el\i
*el = PB_FindNumericMapElement(*mp,345)
Debug *el\i


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 10:02 am 
Offline
Enthusiast
Enthusiast

Joined: Fri Oct 16, 2009 10:12 am
Posts: 653
Location: BE
idle wrote:
the pb map header suggests there is a numeric map key...

What do you mean with this ? I see only strings as map key in the PB-helpfiles.

Your code doesn't works on windows, so ... no can do.

Thanks.

_________________
Yeah I know, but keep in mind ... Leonardo da Vinci was also an autodidact.


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 10:12 am 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3525
Location: New Zealand
Joris wrote:
idle wrote:
the pb map header suggests there is a numeric map key...

What do you mean with this ? I see only strings as map key in the PB-helpfiles.

Your code doesn't works on windows, so ... no can do.

Thanks.


The c headers for maps say it all

PB_NewMap(integer ElementSize, int ElementType, integer *StructureMap, PB_Map **Address, int HashSize);
PB_NewNumericMap(integer ElementSize, integer *StructureMap, PB_Map **Address, int HashSize);

I don't know how long it's been like that but it is clearly an option which just isn't exposed.

Hey, I did clearly say it only worked on Linux x64, I ran out of time to look at it, however Wilbert has taken a look and I expect he will post a solution soon!


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 11:17 am 
Offline
PureBasic Expert
PureBasic Expert

Joined: Sun Aug 08, 2004 5:21 am
Posts: 3622
Location: Netherlands
Joris wrote:
Your code doesn't works on windows, so ... no can do.

Idle means the procedures are there but they aren't exposed so you won't find them in the help file.

Here's an example ...
Code:
;->> Map extension module <<

DeclareModule MapEx
   
  Prototype GetMapPointerI (Map IntegerMap.i())
  Prototype GetMapPointerL (Map LongMap.l())
  Prototype GetMapPointerQ (Map QuadMap.q())
  Prototype GetMapPointerS (Map StringMap.s())
 
  Prototype NumericMapKeyI (Map IntegerMap.i())
  Prototype NumericMapKeyL (Map LongMap.l())
  Prototype NumericMapKeyQ (Map QuadMap.q())
  Prototype NumericMapKeyS (Map StringMap.s())
 
  Global GetMapPointerI.GetMapPointerI
  Global GetMapPointerL.GetMapPointerL
  Global GetMapPointerQ.GetMapPointerQ
  Global GetMapPointerS.GetMapPointerS
 
  Global NumericMapKeyI.NumericMapKeyI
  Global NumericMapKeyL.NumericMapKeyL
  Global NumericMapKeyQ.NumericMapKeyQ
  Global NumericMapKeyS.NumericMapKeyS
 
EndDeclareModule

Module MapEx
 
  DisableDebugger
 
  ;->> Procedures << 
 
  Procedure GetMapPointer(*Map)
    CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
      !mov rdx, [p.p_Map]
      !mov rax, [rdx + 72]
    CompilerElse
      !mov edx, [p.p_Map]
      !mov eax, [edx + 48]
    CompilerEndIf
    ProcedureReturn
  EndProcedure
 
  Procedure NumericMapKey(*Map)
    CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
      !mov rdx, [p.p_Map]
      !mov rdx, [rdx]
      !mov rax, [rdx + 8]
    CompilerElse
      !mov edx, [p.p_Map]
      !mov edx, [edx]
      !mov eax, [edx + 4]
    CompilerEndIf
    ProcedureReturn
  EndProcedure
 
  ;->> Prototype assignments <<
 
  GetMapPointerI = @GetMapPointer()
  GetMapPointerL = @GetMapPointer()
  GetMapPointerQ = @GetMapPointer()
  GetMapPointerS = @GetMapPointer()
 
  NumericMapKeyI = @NumericMapKey()
  NumericMapKeyL = @NumericMapKey()
  NumericMapKeyQ = @NumericMapKey()
  NumericMapKeyS = @NumericMapKey()
 
EndModule



;->> PB imports for numeric integer maps <<

Import ""
  PB_NewNumericMap(ElementSize, *StructureMap, *PtrAddress, HashSize)
  PB_AddNumericMapElement2(Map IntegerMap.i(), Key, ElementCheck = #PB_Map_ElementCheck)
  PB_FindNumericMapElement(Map IntegerMap.i(), Key)
  PB_DeleteNumericMapElement2(Map IntegerMap.i(), Key)
  PB_GetNumericMapElement(Map IntegerMap.i(), Key)
EndImport




; Create a dummy map and free it again
; this way we have a variable that is
; marked as a map
NewMap NumericMap.i()
*NumericMap = MapEx::GetMapPointerI(NumericMap())
FreeMap(NumericMap())

; Create a numeric map for the same variable
PB_NewNumericMap(SizeOf(Integer), 0, *NumericMap, 512)

; Work with the numeric map

PB_AddNumericMapElement2(NumericMap(), 12)
NumericMap() = 123

PB_AddNumericMapElement2(NumericMap(), 34)
NumericMap() = 345

PushMapPosition(NumericMap())

PB_AddNumericMapElement2(NumericMap(), 56)
NumericMap() = 765

PB_AddNumericMapElement2(NumericMap(), 56, #PB_Map_NoElementCheck)
NumericMap() = 567

PopMapPosition(NumericMap())
Debug NumericMap()

PB_FindNumericMapElement(NumericMap(), 56)
Debug NumericMap()

*Int.Integer = PB_GetNumericMapElement(NumericMap(), 12)
Debug *Int\i

ForEach NumericMap()
  Debug "Key:" + Str(MapEx::NumericMapKeyI(NumericMap())) + "  Value: " + Str(NumericMap())
Next

FreeMap(NumericMap())


Unfortunately the imports would need to be changed for a map containing strings. :(
But is shows the procedures themselves are there.

@Fred, is there a reason this functionality is not exposed ?
It would be very useful to be able to use a numeric value as a key.

_________________
macOS 10.15 Catalina, Windows 10


Last edited by wilbert on Sat Dec 14, 2019 6:44 am, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 1:52 pm 
Offline
Enthusiast
Enthusiast

Joined: Fri Oct 16, 2009 10:12 am
Posts: 653
Location: BE
Thanks for the efforts people (masters).

@Wilbert I get this (PB 5.71 LTS on XP 32, yeah old hé) :
One value 345 and then :
"The debugged executable quit unexpectedly."
wilbert wrote:
It would be very useful to be able to use a numeric value as a key.
Indeed.

_________________
Yeah I know, but keep in mind ... Leonardo da Vinci was also an autodidact.


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 2:14 pm 
Offline
PureBasic Expert
PureBasic Expert

Joined: Sun Aug 08, 2004 5:21 am
Posts: 3622
Location: Netherlands
Joris wrote:
@Wilbert I get this (PB 5.71 LTS on XP 32, yeah old hé) :
One value 345 and then :
"The debugged executable quit unexpectedly."

Does it make a difference if you change ImportC to Import ?

An alternative could be to have a faster function to convert a memory pointer to a string.
Or if you don't have to store a lot of values, a tree instead of a map.

_________________
macOS 10.15 Catalina, Windows 10


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 5:03 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sat May 05, 2007 5:31 pm
Posts: 737
Location: Linz, Austria
Thanks wilbert! Any chance you could get this to work with custom structures?
Code:
Structure window
  NewList controls.i()
  hWnd.i
  value.i
EndStructure


wilbert wrote:
@Fred, is there a reason this functionality is not exposed ?
It would be very useful to be able to use a numeric value as a key.
I'd be pretty interested in that, too

_________________
Et cetera is my worst enemy


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Fri Dec 13, 2019 10:17 pm 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3525
Location: New Zealand
chi wrote:
Thanks wilbert! Any chance you could get this to work with custom structures?
Code:
Structure window
  NewList controls.i()
  hWnd.i
  value.i
EndStructure


wilbert wrote:
@Fred, is there a reason this functionality is not exposed ?
It would be very useful to be able to use a numeric value as a key.
I'd be pretty interested in that, too


It can be used with structures, It just needs the structure address filled out, so for strings or any structure with a PB_Object, you just pass in the address

example tested on linux x64 and Windows x86

Code:
Import ""
  PB_NewNumericMap(ElementSize.i,*StructureMap,*Address,HashTabelSize.l);
  PB_FreeMap(*Map);
  PB_ResetMap(*Map);
  PB_ClearMap(*Map);
  PB_MapSize(*Map) ;
 
  PB_PushMapPosition(*map);
  PB_PopMapPosition(*map);
 
  PB_NextMapElement(*Map);
 
  PB_FindNumericMapElement(*Map,Key.i);
  PB_GetNumericMapElement(*Map,Key.i);
 
  PB_AddNumericMapElement(*Map,Key.i);
  PB_AddNumericMapElement2(*Map,Key.i,ElementCheck=#PB_Map_ElementCheck);
 
  PB_DeleteNumericMapElement2(*Map,Key.i);
  PB_DeleteNumericMapElement(*Map);
 
  PB_CopyMap(*Map,*DestinationMap);
  PB_CopyMap2(*Map,*DestinationMap,Clear.l);
  ;
  PB_MapKey(*Map,PreviousPosition.l);
   
EndImport

Macro NewNumericMap(pmap,mapsize,Type) 
  Global structadr
  EnableASM
  CompilerIf TypeOf(Type) = #PB_Structure   
    CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
      lea rax,[s_#Type]
      mov [v_structadr],rax
    CompilerElse
      lea eax,[s_#Type]
      mov [v_structadr],eax
    CompilerEndIf
  CompilerElse
    structadr = 0
  CompilerEndIf 
  DisableASM
  PB_NewNumericMap(SizeOf(Type),structadr,@pmap,mapsize)
EndMacro   


Structure mFoo
  List controls.i()
  i.i
  s.s
EndStructure   

Global *mp,*el.mFoo

NewNumericMap(*mp,512,mfoo)

*el = PB_AddNumericMapElement2(*mp,123)
AddElement(*el\controls())
*el\controls() = 123
AddElement(*el\controls())
*el\controls() = 345
*el\i = 123
*el\s = "Hello"
;
*el = PB_AddNumericMapElement2(*mp,345)
AddElement(*el\controls())
*el\controls() = 678
AddElement(*el\controls())
*el\controls() = 911

*el\i = 345
*el\s = "World"
;
*el = PB_FindNumericMapElement(*mp,123)
ForEach *el\controls()
  Debug *el\controls()
Next   
Debug *el\i
Debug *el\s

*el = PB_FindNumericMapElement(*mp,345)
ForEach *el\controls()
  Debug *el\controls()
Next   
Debug *el\i
Debug *el\s

PB_FreeMap(*mp)


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Sat Dec 14, 2019 3:13 am 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sat May 05, 2007 5:31 pm
Posts: 737
Location: Linz, Austria
This is fantastic! Thank you, idle 8)

Now I can finally get rid of converting the hWnd to a string first...

_________________
Et cetera is my worst enemy


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Sat Dec 14, 2019 4:05 am 
Offline
Addict
Addict
User avatar

Joined: Fri Sep 21, 2007 5:52 am
Posts: 3525
Location: New Zealand
chi wrote:
This is fantastic! Thank you, idle 8)

Now I can finally get rid of converting the hWnd to a string first...


try this, think I've got it sorted so you can use native types but might not work on osx

Code:
;NumericMap hack for structured Numeric keyed Maps
;windows linux osx

Import ""
  PB_NewNumericMap(ElementSize.i,*StructureMap,*Address,HashTabelSize.l);
  PB_FreeMap(*Map)                                                      ;
  PB_ResetMap(*Map)                                                     ;
  PB_ClearMap(*Map)                                                     ;
  PB_MapSize(*Map)                                                      ;
  PB_PushMapPosition(*map)                                              ;
  PB_PopMapPosition(*map)                                               ;
  PB_NextMapElement(*Map)                                               ;
  PB_FindNumericMapElement(*Map,Key.i)                                  ;
  PB_AddNumericMapElement(*Map,Key.i)                                   ;
  PB_AddNumericMapElement2(*Map,Key.i,ElementCheck=#PB_Map_ElementCheck);
  PB_DeleteNumericMapElement2(*Map,Key.i)                               ;
  PB_DeleteNumericMapElement(*Map)                                      ;
  PB_CopyMap(*Map,*DestinationMap)                                      ;
  PB_CopyMap2(*Map,*DestinationMap,Clear.l)                             ;
EndImport

Macro NewNumericMap(pmap,HashTableSize,StructureType,IsStructure=0)
  Global structadr=0
  EnableASM
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    CompilerIf IsStructure
      lea rax,[s_#StructureType]
      mov [v_structadr],rax
    CompilerEndIf
  CompilerElse
    CompilerIf IsStructure 
      lea eax,[s_#StructureType]
      mov [v_structadr],eax
    CompilerEndIf
  CompilerEndIf
  DisableASM
  PB_NewNumericMap(SizeOf(StructureType),structadr,@pmap,HashTableSize)
EndMacro   

Procedure NumericMapkey(*Map)
  Protected ele,key   
  ele = PeekI(*map)
  key = PeekI(ele+SizeOf(Integer))
  ProcedureReturn key
EndProcedure   

CompilerIf #PB_Compiler_IsMainFile
 
  ;Note you need to use a structured element pointer even for native types
 
  Structure foo
    List controls.i()
    i.i
    s.s
  EndStructure   
 
 
  Procedure test()
    Protected mp, *el.foo
    NewNumericMap(mp,512,foo,#PB_Structure) ;create the map and pass map variable, number of elements, the name of the structure 
   
    *el = PB_AddNumericMapElement(mp,123)   
   
    AddElement(*el\controls())
    *el\controls() = 111
    AddElement(*el\controls())
    *el\controls() = 112
    *el\i = 123
    *el\s = "Hello"
    ;
    *el = PB_AddNumericMapElement2(mp,345)
    AddElement(*el\controls())
    *el\controls() = 113
    AddElement(*el\controls())
    *el\controls() = 114
   
    *el\i = 345
    *el\s = "World"
    ;
    *el = PB_FindNumericMapElement(mp,123)
    ForEach *el\controls()
      Debug "list Controls " + *el\controls()
    Next   
    Debug "integer " + *el\i
    Debug "string "  + *el\s
   
    *el = PB_FindNumericMapElement(mp,345)
    ForEach *el\controls()
      Debug "list Controls " + *el\controls()
    Next   
    Debug "integer " + *el\i
    Debug "string "  + *el\s
   
    Debug "+++++++++++"
    Debug "walk the map"
    Debug "+++++++++++"
   
    PB_ResetMap(mp)
    Repeat
      *el = PB_NextMapElement(mp)
      If *el 
        Debug "mapkey " + NumericMapkey(mp)
        ForEach *el\controls()
          Debug "list Controls " + *el\controls()
        Next   
        Debug "integer " + *el\i
        Debug "string "  + *el\s
      EndIf   
    Until *el=0   
   
    Debug "++++++++++++++"
    Debug "clear map"
    PB_ClearMap(mp);
    PB_ResetMap(mp)
    Repeat
      *el = PB_NextMapElement(mp)
      If *el 
        Debug "mapkey " + NumericMapkey(mp)
        ForEach *el\controls()
          Debug "list Controls " + *el\controls()
        Next   
        Debug "integer " + *el\i
        Debug "string "  + *el\s
      EndIf   
    Until *el=0   
    Debug "should be nothing"
   
    PB_FreeMap(mp)
   
    Debug "++++++++++++"
    Debug "test with a native type via pointer" 
    Protected *elf.float   
    NewNumericMap(mp,512,float) ;create the map and pass map variable, number of elements, the name of the structure
    *elf = PB_AddNumericMapElement(mp,123)
    *elf\f = #PI
    *elf = PB_AddNumericMapElement(mp,345)
    *elf\f = 2*#PI
    *elf = PB_FindNumericMapElement(mp,123)
    Debug *elf\f
   
    *elf = PB_FindNumericMapElement(mp,345)
    If *elf
      Debug *elf\f
    EndIf
   
    Debug NumericMapkey(mp)
   
    PB_FreeMap(mp)
   
  EndProcedure
 
  test()
 
CompilerEndIf


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Sat Dec 14, 2019 9:35 am 
Offline
Enthusiast
Enthusiast

Joined: Fri Oct 16, 2009 10:12 am
Posts: 653
Location: BE
wilbert wrote:
Does it make a difference if you change ImportC to Import ?
I tried but no better result.

_________________
Yeah I know, but keep in mind ... Leonardo da Vinci was also an autodidact.


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Thu Dec 19, 2019 9:54 am 
Offline
PureBasic Expert
PureBasic Expert

Joined: Sun Aug 08, 2004 5:21 am
Posts: 3622
Location: Netherlands
Joris wrote:
wilbert wrote:
Does it make a difference if you change ImportC to Import ?
I tried but no better result.

Approximately how many items with a numeric key do you need to store ?
Hundreds, thousands ?

_________________
macOS 10.15 Catalina, Windows 10


Top
 Profile  
Reply with quote  
 Post subject: Re: Using adresses as a key for AddMapElements ???
PostPosted: Thu Dec 19, 2019 12:46 pm 
Offline
Enthusiast
Enthusiast

Joined: Fri Oct 16, 2009 10:12 am
Posts: 653
Location: BE
wilbert wrote:
... Approximately how many items with a numeric key do you need to store ?
Hundreds, thousands ?
wilbert it can vary by user use of the program, but I suppose it will mostly be less then 100, but yeah, that's my guess...

_________________
Yeah I know, but keep in mind ... Leonardo da Vinci was also an autodidact.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye