JSON::eval() 0.1a - EASY json access

Share your advanced PureBasic knowledge/code with the community.
User avatar
Derren
Enthusiast
Enthusiast
Posts: 316
Joined: Sat Jul 23, 2011 1:13 am
Location: Germany

JSON::eval() 0.1a - EASY json access

Post by Derren »

I was sooo excited when I read "json library added" in the 5.3 changelog, but frankly I expected something different than an endless amount of function calls of all kinds, even for not very complex json objects.

What I wanted, was an easy implementation like in Javascript (this might be slightly simplified, but it's really easy to access JsON nodes in JS (obviously, json was made for javascript)):

Code: Select all

json = {foo:bar}; 
print(json.foo); //prints "bar"
What my module does is this:

Code: Select all

Define json.s = "{foo:bar}"
JSON::eval(json)
Debug JSON::Get("foo") ; will debug: bar
Simple. Even this simple object would take "tons" of code in 5.3 PB...

So here is some info and further down the code:

Version 0.1a (first release) 30.09.2014
  • This module does NOT validate the json string. Perhaps there's an easy way to do it with PB (like the XML example demonstrates for example). So if your json string contains errors, you will get weird results.
  • TODO: Currently, only one json object (string) is supported. Will add an ID-system later
  • TODO: Keys are case sensitive. I will add a flag later
  • TODO: Objects in Arrays are not properly parsed
  • There are no types just now. Everything is a string. I might add a type system later. Especially for the array type, although I'm not quite sure how to design it. For now you'd have to explode an array yourself with StringField() and convert any numbers yourself as well.
  • I put some comments, but not everything is commented, especially the latter part where the actual parsing happens. I will work on that.
  • I am NOT an expert on json, nor PB. Please aid and help with tips, suggestions and pointing out coding mistakes wherever you can, if you like. Thank you.

Code: Select all

;============================================================
;= JSON:: 		Module for easy json access
;= Version:		0.1a 
;= Author: 		Derren (german and english purebasic forum)
;= PB Version:	5.22 LTS  x86 (written in)
;= Infothread:	http://purebasic.fr/english/viewtopic.php?f=12&t=60656
;============================================================

DeclareModule JSON
	#JSON_Node_Delimiter$ = "\" ; The delimiter of object levels, e.g. "." in Javascript or "->" in php. 
	; In PB the closest thing would be the "\" in structures, so that is what I'm using here but feel free to change it.
	
	Declare eval(input.s)	; Should return 0. If not, there's an error in object depth, e.g. "{key:value,key2:{object}" (closing paranthesis missing)
	Declare.s GET(key.s)	; Returns the value for the given key as a string
	
	Enumeration
		#Json_Type_Generic
		#Json_Type_NULL
		#Json_Type_Bool
		#Json_Type_Num
		#Json_Type_String
		#Json_Type_Array
	EndEnumeration 
	
EndDeclareModule 	



Module JSON
	Structure json
		type.i	;type of value (see Enumeration in Declare block above). Not in use yet
		Data.s	;the value as a string
		key.s 	;just stores the key of the map for when you iterate through the map with ForEach
	EndStructure	
	
	;---private functions
	Procedure.s _Strip(string.s) ;Trims spaces, tabs, encasing double quotes
		string = Trim(string)
		string = Trim(string, Chr(9))
		string = Trim(string, Chr(34))
		ProcedureReturn string
	EndProcedure 
	
	;--- private declarations
	NewMap JSON_Map_Internal.json()
	
	
	;---public functions 		
	
	Procedure.s GET(key.s)
		Shared JSON_Map_Internal()
		ProcedureReturn JSON_Map_Internal(key)\data		
	EndProcedure 
	
	Procedure eval(input.s)
		Shared JSON_Map_Internal()
		
		Protected *c.Character 	= @input	; Pointer with the size of a char at the adress of the input string ( = first char in string)
		
		Protected objectLevel.i				; Depth of objects, i.e. {bla:"level_1", blub {meep:"level_2, another:"level_2"}}
		
		Protected *key_begin 		= *c		; Marks the beginning of a key, based on the delimiters
		Protected *key_end						; Marks the end of a key ^^
		Protected *value_begin		= *c		; Same for the value ^^
		Protected *value_end					; ^^
		
		Protected inString.i					; bool to determine if a delimiter is inside a string and thus should be ignored
		Protected inArray.i					; bool to determine whether or not a comma is in an array or outside where it marks a new key pair
		Protected keyExists.i					; This is set to true when a key exists before a value. An easy workaround for double values found when "}," is encountered
		
		Protected KEY.s 						; The actual key used to access a node. This variable is used to build the key which is then used in the map
		Protected TYPE.i						; The type of the value
		
		Protected counter.i 					; counter variable used in For loop		
		
		Dim lastKeyOnLevel.s(1)
		
		
		While *c\c ! 0										; While EndOfString <> 0
			Select *c\c 
					
				Case '"'									; Hitting ": Either entering or leaving string 
					inString = Abs(inString - 1)			; -> Alternate inside/outside state
					
				Case '{'
					If Not inString	
						*key_begin = *c
						objectLevel +1
						ReDim lastKeyOnLevel(objectLevel)	
					EndIf 
					
				Case '}'
					If Not inString
						*value_end = *c
						objectLevel -1
					EndIf 
					
				Case '['
					If Not inString
						inArray=#True
					EndIf 
					
				Case ']'
					If Not inString
						inArray=#False
					EndIf 
					
				Case ':'
					If Not inString
						*key_end = *c
						*value_begin = *c
					EndIf 
					
				Case ','
					If Not inString And Not inArray
						*value_end = *c
						*key_begin = *c
					EndIf 
					
			EndSelect  
			
			
			
			If *c = *key_end
				*key_begin + SizeOf(Character)				
				
				lastKeyOnLevel(objectLevel) = _Strip(  PeekS(*key_begin, (*key_end - *key_begin)/SizeOf(Character)) )
				
				KEY=""
				For counter = 1 To objectLevel - 1
					KEY + lastKeyOnLevel(counter) + #JSON_Node_Delimiter$
				Next 
				KEY + _Strip(  PeekS(*key_begin, (*key_end - *key_begin)/SizeOf(Character)) )
				
				keyExists = #True 
			EndIf 
			
			
			
			If *c = *value_end And keyExists=#True
				*value_begin+SizeOf(character)
				
				keyExists=#False 
				JSON_Map_Internal(KEY)\data = _Strip(  PeekS(*value_begin, (*value_end - *value_begin)/SizeOf(Character)) )
				
				JSON_Map_Internal(KEY)\key = KEY
			EndIf 
			
			*c + SizeOf(Character) ; Next Character
		Wend 
		
		ProcedureReturn objectLevel ;Should be 0. If not, the json string contained an error.
	EndProcedure 	
EndModule 

;-
;-EXAMPLE

Define json.s = "{"  ;copied from Wikipedia and edited to add more variety
json + "	'Company': 'Xema',"
json + "	'Number': '1234-5678-9012-3456',"
json + "	'Exponent': 2e+6,"
json + "	'Currency': 'EURO',"
json + "	'Person': {"
json + "		'Name': 'Mustermann',"
json + "		'First Name': 'Max',"
json + "		'male': true,"
json + "		'Hobbys': [ 'Hiking', 'Golf', 'Reading' ],"
json + "		'Age': 42,"
json + "		'Children': [{name:julia},{name:hans}],"
json + "		'Spouse': null"
json + "	}"
json + "}":ReplaceString(json, "'", Chr(34), #PB_String_InPlace)


JSON::eval(json)

Debug "Company: "+			JSON::GET("Company")
Debug "Number: "+			JSON::GET("Number")
Debug "Person: "+			JSON::GET("Person") + "<< empty, as it's an object and not a real value"
Debug "Person -> Name: "+	JSON::GET("Person\Name")
Debug "Person -> Hobbys: "+	JSON::GET("Person\Hobbys")
Debug JSON::GET("Person\Children\name") ;this is not a proper key, it should be an array index. This needs to be fixed
firace
Addict
Addict
Posts: 946
Joined: Wed Nov 09, 2011 8:58 am

Re: JSON::eval() 0.1a - EASY json access

Post by firace »

Thanks a lot for sharing this. I feel exactly the same as you regarding PB's JSON library...

However I tried to use your module with a real-world example but could not access any values.
Note: The json.s string below is the exact response I get from a google API query.

Code: Select all


Define json.s = "{'responseData': {'results':[{'GsearchResultClass':'GwebSearch','unescapedUrl':'http://powerbasic.com/','url':'http://powerbasic.com/','visibleUrl':'powerbasic.com','cacheUrl':'http://www.google.com/search?q\u003dcache:rTRGSFWIrzEJ:powerbasic.com','title':'\u003cb\u003ePowerBASIC\u003c/b\u003e: Basic Compilers','titleNoFormatting':'PowerBASIC: Basic Compilers','content':'Welcome to \u003cb\u003ePowerBASIC\u003c/b\u003e! Shop online for \u003cb\u003ePowerBASIC\u003c/b\u003e compilers, BASIC \ncompilers and programming tools for Windows and DOS. Extensive tech support \nand ...'}],'cursor':{'resultCount':'24,100','pages':[{'start':'0','label':1},{'start':'1','label':2},{'start':'2','label':3},{'start':'3','label':4},{'start':'4','label':5},{'start':'5','label':6},{'start':'6','label':7},{'start':'7','label':8}],'estimatedResultCount':'24100','currentPageIndex':0,'moreResultsUrl':'http://www.google.com/search?oe\u003dutf8\u0026ie\u003dutf8\u0026source\u003duds\u0026start\u003d0\u0026hl\u003den\u0026q\u003dpowerbasic','searchResultTime':'0.23'}}, 'responseDetails': null, 'responseStatus': 200}"

ReplaceString(json, "'", Chr(34), #PB_String_InPlace)


JSON::eval(json)

Debug "Company: "+         JSON::GET("responseData\results\GsearchResultClass")   ;; empty?

destiny
User
User
Posts: 29
Joined: Wed Jul 15, 2015 12:58 pm
Location: CCCP

Re: JSON::eval() 0.1a - EASY json access

Post by destiny »

Derren, thanx for the idea to start from.
Did you extended your code or finished on that?
Is there possible to cycle through keys that exists like through array elements or object elements? (for (ArrayMembers = JSONArraySize(Value) - 1; For i = 0 To ArrayMembers ... Next i)

PS: 5.31 includes JSON library/procedures. Maybe you can use them?
User avatar
falsam
Enthusiast
Enthusiast
Posts: 632
Joined: Wed Sep 21, 2011 9:11 am
Location: France
Contact:

Re: JSON::eval() 0.1a - EASY json access

Post by falsam »

Ho ho, I did not see this topic. Destiny thank you for your question. Derren it's great. Thank.

I adopt this code.

➽ Windows 11 64-bit - PB 6.21 x64 - AMD Ryzen 7 - NVIDIA GeForce GTX 1650 Ti

Sorry for my bad english and the Dunning–Kruger effect 🤪
destiny
User
User
Posts: 29
Joined: Wed Jul 15, 2015 12:58 pm
Location: CCCP

Re: JSON::eval() 0.1a - EASY json access

Post by destiny »

falsam wrote:I adopt this code.
It's really great. I appreciate it and will be thankful if you adopt this code to latest 5.31 procedures and make possible of sorting values inside this variable by key and/or value and cycle through arrays/objects inside variables inside this variable. Please, post code or link to your solution/example in my thread http://forums.purebasic.fr/english/view ... 13&t=62595
Thank you very much. I try to adopt it by muself but with no luck. I'm not experienced pb coder, i'm new :( and learning.
Little John
Addict
Addict
Posts: 4777
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Re: JSON::eval() 0.1a - EASY json access

Post by Little John »

Another approach.
//edit: updated and extended on 2015-07-17

Code: Select all

; PB 5.31

DeclareModule JSON
   Declare.s Get (jn.i, path$, quote$="'")
   Declare.i Set (jn.i, path$, value$, quote$="'")
EndDeclareModule


Module JSON
   EnableExplicit
   
   Procedure.i _GetJSONValue (jn.i, path$)
      ; -- internal function
      Protected jv.i, depth.i, level.i, index.i, field$
      
      jv = JSONValue(jn)
      
      If jv <> 0 And path$ <> ""
         depth = CountString(path$, "\") + 1
         For level = 1 To depth
            field$ = StringField(path$, level, "\")
            If Left(field$, 1) = "[" And Right(field$, 1) = "]"
               index = Val(Mid(field$, 2, Len(field$)-2))
               If JSONType(jv) = #PB_JSON_Array And index >= 0 And index < JSONArraySize(jv)
                  jv = GetJSONElement(jv, index)
               Else
                  jv = 0
               EndIf
            Else   
               If JSONType(jv) = #PB_JSON_Object
                  jv = GetJSONMember(jv, field$)   ; 0 if the given 'field$' does not exist in the object
               Else
                  jv = 0
               EndIf
            EndIf
         Next   
      EndIf
      
      ProcedureReturn jv
   EndProcedure
   
   
   Procedure.s Get (jn.i, path$, quote$="'")
      ; in : jn    : JSON number, e.g. generated by ParseJSON()
      ;      path$ : path to the JSON element that is to be retrieved
      ;      quote$: character that is regarded as quote
      ; out: value of the desired element
      Protected jv.i, ret$=""
      
      If jn
         jv = _GetJSONValue(jn, path$)
         If jv
            Select JSONType(jv)
               Case #PB_JSON_String 
                  ret$ = quote$ + GetJSONString(jv) + quote$
               Case #PB_JSON_Boolean 
                  If GetJSONBoolean(jv) = #True
                     ret$ = "true"
                  Else
                     ret$ = "false"
                  EndIf   
               Case #PB_JSON_Null 
                  ret$ = "null" 
               Case #PB_JSON_Number
                  ret$ = StrD(GetJSONDouble(jv)) 
            EndSelect      
         EndIf
      EndIf
      
      ProcedureReturn ret$
   EndProcedure
   
   
   Macro _IsNumber (_string_)
      ; -- internal macro
      Bool(_string_ <> "" And (Val(_string_) <> Val(_string_+"1") Or ValF(_string_) <> ValD(_string_+"1")))
   EndMacro
   
   Procedure.i Set (jn.i, path$, value$, quote$="'")
      ; in : jn    : JSON number, e.g. generated by ParseJSON()
      ;      path$ : path to the JSON element that is to be changed
      ;      value$: new value of the desired element
      ;      quote$: character that is regarded as quote
      ; out: #True on success, #False on error
      Protected jv.i, ret.i=#False
      
      If jn
         jv = _GetJSONValue(jn, path$)
         If jv
            If Left(value$, 1) = quote$ And Right(value$, 1) = quote$
               SetJSONString(jv, Mid(value$, 2, Len(value$)-2))
               ret = #True
            ElseIf value$ = "true"
               SetJSONBoolean(jv, #True)
               ret = #True
            ElseIf value$ = "false"
               SetJSONBoolean(jv, #False)
               ret = #True
            ElseIf value$ = "null"
               SetJSONNull(jv)
               ret = #True
            ElseIf _IsNumber(value$)
               SetJSONDouble(jv, ValD(value$))
               ret = #True
            EndIf  
         EndIf
      EndIf
      
      ProcedureReturn ret
   EndProcedure
EndModule


CompilerIf #PB_Compiler_IsMainFile
   ; ==== Demo ====
   
   EnableExplicit
   
   Procedure ShowJSON (jn.i)
      Protected i.i, hobby$
      
      Debug "Company: "          + JSON::Get(jn, "Company")
      Debug "Number: "           + JSON::Get(jn, "Number")
      Debug "Person: "           + JSON::Get(jn, "Person") + "     << empty, as it's an object and not a real value"
      Debug "Person -> Name: "   + JSON::Get(jn, "Person\Name")
      Debug "Person -> Hobbys: "
      i = 0
      Repeat
         hobby$ = JSON::Get(jn, "Person\Hobbys\[" + i + "]")
         If hobby$ = ""
            Break
         EndIf   
         Debug "       " + hobby$
         i + 1
      ForEver
      Debug "Person -> Children: " + 
            JSON::Get(jn, "Person\Children\[0]\name") + ", " +
            JSON::Get(jn, "Person\Children\[1]\name")
   EndProcedure
   
   
   Define json$, jn.i
   
   ; -- some simple cases
   json$ = "'abc'"
   ReplaceString(json$, "'", #DQUOTE$, #PB_String_InPlace)
   jn = ParseJSON(#PB_Any, json$)
   Debug JSON::Get(jn, "")
   
   Debug JSON::Set(jn, "", "'Hello!'")
   Debug JSON::Get(jn, "")
   
   Debug JSON::Set(jn, "", "true")
   Debug JSON::Get(jn, "")
   
   Debug JSON::Set(jn, "", "-12")
   Debug JSON::Get(jn, "")
   
   Debug JSON::Set(jn, "", "7.3")
   Debug JSON::Get(jn, "")
   
   FreeJSON(jn)
   
   Debug ""
   Debug "==================================="
   Debug ""
   
   ; -- an object
   json$ = "{"
   json$ + "   'Company': 'Xema',"
   json$ + "   'Number': '1234-5678-9012-3456',"
   json$ + "   'Exponent': 2e+6,"
   json$ + "   'Currency': 'EURO',"
   json$ + "   'Person': {"
   json$ + "      'Name': 'Mustermann',"
   json$ + "      'First Name': 'Max',"
   json$ + "      'male': true,"
   json$ + "      'Hobbys': [ 'Hiking', 'Golf', 'Reading' ],"
   json$ + "      'Age': 42,"
   json$ + "      'Children': [{'name': 'Julia'}, {'name': 'Hans'}],"
   json$ + "      'Spouse': null"
   json$ + "   }"
   json$ + "}"
   ReplaceString(json$, "'", #DQUOTE$, #PB_String_InPlace)
   jn = ParseJSON(#PB_Any, json$)
   
   ShowJSON(jn)
   
   Debug "-----------------------------------"
   
   Debug JSON::Set(jn, "Company", "'CoolToys unlimited'")
   Debug JSON::Set(jn, "Number", "007")
   Debug JSON::Set(jn, "Person\Name", "'Smith'")
   Debug JSON::Set(jn, "Person\Hobbys\[0]", "'Tennis'")
   
   ShowJSON(jn)
   Debug ""
   Debug ComposeJSON(jn, #PB_JSON_PrettyPrint)
   FreeJSON(jn)
CompilerEndIf
Last edited by Little John on Fri Jul 17, 2015 7:12 am, edited 1 time in total.
destiny
User
User
Posts: 29
Joined: Wed Jul 15, 2015 12:58 pm
Location: CCCP

Re: JSON::eval() 0.1a - EASY json access

Post by destiny »

Little John, it's awesome!
No Enumerations, no Structures, no map(slots)! Just compact, clean and simple enough code that simplifying a lot of things.
I need some time to adopt it to my tasks, push away cloning and freeing of json variable each time GetJSON called (because i have a tons of incoming data and each call to variable make clone then kill clone, so i need to speed up this just by push such things away and clone then free json variable after all work done with it). And make possible of sorting data by value (like "Julia", "Hans" in example) and by keys (like "Name", "First Name", "male", "Hobbys" etc). I think i can do it in a week or so :-D
I've read you using purifier to understand how much time any part of the code takes. Do you know (according to yout experience) how much difference in time will be if i leave everything as is and each time will create a clone then kill it when when accessing a variable inside json (for example, 100'000 times of accessing GetJSON(json, "Person\Children\[1]\name") in example) against accessing to the same variable without clone then killing of this variable?

And another question - does it make a sense of changing json data to a pointer *? And if yes - then how?

PS: thinked about it and understand that this is for easy _access_, not for storing/holding data... :(
Is there any chance to store that data to variable and then access to any value in this variable like in your example?
Last edited by destiny on Thu Jul 16, 2015 9:31 pm, edited 1 time in total.
Little John
Addict
Addict
Posts: 4777
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Re: JSON::eval() 0.1a - EASY json access

Post by Little John »

Hi destiny, you are welcome!
destiny wrote:I've read you using purifier to understand how much time any part of the code takes.
Not quite. ;-)

The Purifier is a built-in debugging tool for finding memory errors,
see at the end of this page:
http://www.purebasic.com/documentation/ ... tools.html

The Purefiler is a tool by Didelphodon that makes time profiling of our programs possible,
see
http://www.purebasic.fr/english/viewtopic.php?f=27&t=43727
destiny wrote:Do you know (according to yout experience) how much difference in time will be if i leave everything as is and each time will create a clone then kill it when when accessing a variable inside json (for example, 100'000 times of accessing GetJSON(json, "Person\Children\[1]\name") in example) against accessing to the same variable without clone then killing of this variable?
I don't know, please do any time profiling yourself.
Anyway, I have changed the code now, so that ParseJSON() is not called by the Get() function anymore.
destiny wrote:And another question - does it make a sense of changing json data to a pointer *? And if yes - then how?
In my above code, I cannot see where it would make sense.
For instance, when you add

Code: Select all

Debug jn
then a number such as 35308896 will be shown, and it's pretty much the same with jv.
This indicates, that the variables jn and jv already are pointers.
However, I can't answer in general whether or not it would make sense to use pointers in your code.
destiny wrote:PS: thinked about it and understand that this is for easy _access_, not for storing/holding data... :(
Is there any chance to store that data to variable and then access to any value in this variable like in your example?
I have updated and extended the above code, besides other things now there is also a Set() procedure.
Some things are still not implemented, e.g. the possibility to add new elements to an array.
Post Reply