Page 3 of 4
Re: JSON encoder and decoder
Posted: Thu Mar 28, 2013 4:35 pm
by mariosk8s
Got another patch for you.
You're not JSON escaping keys.
Code: Select all
--- a/Json2.pbi 2013-02-19 17:08:17.716628088 +0100
+++ b/Json2.pbi 2013-03-28 16:32:33.984799955 +0100
@@ -253,11 +253,12 @@
Case 't' ; tabulator
string + Chr(9)
Case 'u' ; 4 hex digits
+ *c + SizeOf(CHARACTER) ; eat the 'u'
If nullByte - *c > 4 * SizeOf(CHARACTER)
hexDigit = "$"
;For i = 1 To 4
hexDigit + PeekS(*c, 4)
- *c + SizeOf(CHARACTER) * 4
+ *c + SizeOf(CHARACTER) * 3 ; only 3 the 4th eaten below
;Next
string + Chr(Val(hexDigit))
Else
@@ -522,6 +523,18 @@
EndIf
EndProcedure
+Procedure.s JSON_encodeString(str.s)
+ Protected tmpString.s = ReplaceString(str, "\", "\\") ; \
+ tmpString = ReplaceString(tmpString, Chr(8), "\b") ; backspace
+ tmpString = ReplaceString(tmpString, Chr(14), "\f") ; formfeed
+ tmpString = ReplaceString(tmpString, Chr(10), "\n") ; new line
+ tmpString = ReplaceString(tmpString, Chr(13), "\r") ; carriage return
+ tmpString = ReplaceString(tmpString, Chr(9), "\t") ; tabulator
+ tmpString = ReplaceString(tmpString, Chr(34), "\" + Chr(34)) ; "
+ tmpString = ReplaceString(tmpString, "/", "\/") ; /
+ ProcedureReturn tmpString
+EndProcedure
+
Procedure.s JSON_encode(*obj.jsonObj, spaces.i = 0, type.i = #JSON_Type_Undefined)
Protected tmpString.s
Protected i.i
@@ -557,17 +570,8 @@
Case #JSON_Type_Integer
JSON_addString(Str(*obj\i))
Case #JSON_Type_String
- tmpString = ReplaceString(*obj\s, "\", "\\") ; \
- tmpString = ReplaceString(tmpString, Chr(8), "\b") ; backspace
- tmpString = ReplaceString(tmpString, Chr(14), "\f") ; formfeed
- tmpString = ReplaceString(tmpString, Chr(10), "\n") ; new line
- tmpString = ReplaceString(tmpString, Chr(13), "\r") ; carriage return
- tmpString = ReplaceString(tmpString, Chr(9), "\t") ; tabulator
- tmpString = ReplaceString(tmpString, Chr(34), "\" + Chr(34)) ; "
- tmpString = ReplaceString(tmpString, "/", "\/") ; /
-
JSON_addString(Chr(34))
- JSON_addString(tmpString)
+ JSON_addString(JSON_encodeString(*obj\s))
JSON_addString(Chr(34))
Case #JSON_Type_Array
JSON_addString("[")
@@ -595,7 +599,7 @@
NextMapElement(*obj\o())
JSON_addString(Space(spaces + 2))
JSON_addString(Chr(34))
- JSON_addString(MapKey(*obj\o()))
+ JSON_addString(JSON_encodeString(MapKey(*obj\o())))
JSON_addString(Chr(34))
JSON_addString(" : ")
CompilerIf Defined(JSON_UseObjectPointer, #PB_Constant)
This patch includes my previous patch since it hasn't made it into the original post yet.
Re: JSON encoder and decoder
Posted: Thu Apr 18, 2013 10:54 pm
by PMV
update 19.04.2013
+ bugfix: encoding for unicode escaped characters (\u) failed
+ bugfix: map keys were not escaped
+ improved: escaping of strings in JSON_encode() now faster
+ improved: test code uses new possibility of CompilerIf #IsMain
Thanks again
mariosk8s for bug hunting
I still have not much time for testing, but i hope the new
escape-function is really faster then the previous solution
with ReplaceString().
Re: JSON encoder and decoder
Posted: Fri Apr 19, 2013 7:07 am
by Kukulkan
Thanks PMV. Very good functions!
Kukulkan
Re: JSON encoder and decoder
Posted: Sat Jul 27, 2013 3:17 am
by Fangbeast
Sorry for the beginner question but, when I have a loaded string, how do I list all the available fields if I don't know in advance what they are?
Tried the below but am getting read errors at address 4 starting at "ResetMap(*out\o())"
Code: Select all
; 9780718149789 the navigator
; 071814788x polar shift
; 0552151696 digital fortress
; 0552149519 the davinci code
; 0552151769 pic deception point
; 0732911206 the lady of the sorrows
IncludeFile "JsonLib.pb"
Define GoogleJsonFile.i = OpenFile(#PB_Any, "GoogleBookResults.txt")
Define JsonString.s = ""
Define FileStringFormat.i = ReadStringFormat(GoogleJsonFile.i)
Define.s URLString.s
Define.s CacheFile.s
If InitNetwork()
URLString.s = "https://www.googleapis.com/books/v1/volumes?projection=full&q=isbn:9780718149789"
CacheFile.s = GetCurrentDirectory() + "GoogleBookResults.txt"
If URLDownloadToFile_(0, URLString.s, CacheFile.s, 0, 0) = #S_OK
Debug "Success"
While Eof(GoogleJsonFile.i) = #False
JsonString.s + ReadString(GoogleJsonFile.i, FileStringFormat.i) + Chr(13) + Chr(10)
Wend
CloseFile(GoogleJsonFile.i)
Define *out.jsonObj = JSON_decode(JsonString.s)
Debug MapSize(*out\o())
ResetMap(*out\o())
While NextMapElement(*out\o())
Debug MapKey(*out\o())
Debug *out\o()\o("url")\s
Wend
Else
Debug "Failed"
EndIf
Else
End
EndIf
**EDIT** Silly me. The cachefile was being truncated at 0 bytes. D'oh! I'll just add a check for filesize.
Re: JSON encoder and decoder
Posted: Sat Jul 27, 2013 5:44 am
by USCode
Looks like C!
Very cool, thanks for sharing!
Re: JSON encoder and decoder
Posted: Sat Jul 27, 2013 10:32 am
by Fangbeast
Okay, I don't know what I am doing but am having trouble reading the output of the below string. I used my routine above to download it from google and I *thought* I could do something simple like: Debug *out\o("textSnippet")\s to show the "textSnippet": " data??
What am I doing wrong?
Code: Select all
{
"kind": "books#volumes",
"totalItems": 1,
"items": [
{
"kind": "books#volume",
"id": "189DPgAACAAJ",
"etag": "M7ZWVKH/6Gc",
"selfLink": "https://www.googleapis.com/books/v1/volumes/189DPgAACAAJ",
"volumeInfo": {
"title": "The Navigator",
"subtitle": "A Kurt Austin Adventure, a Novel from the NUMA Files",
"authors": [
"Clive Cussler"
],
"publisher": "Michael Joseph",
"publishedDate": "2007-05-20",
"description": "Years ago, an ancient Phoenician statue known as the Navigator was stolen from the Baghdad Museum, and there are men who would do anything to get their hands on it. The first victim is a crooked antiquities dealer, murdered in cold blood. Their second very nearly is a UN investigator who, were it not for the timely assistance of Austin and Zavala, would now be at the bottom of a very watery grave. What's so special about this statue? Austin wonders. The search for answers will take the NUMA team on an astonishing odyssey through time and space, one that encompasses no less than the lost treasures of King Solomon, a mysterious packet of documents personally encoded by Thomas Jefferson, and a top secret scientific project that could change the world forever. And that's before the surprises really begin . . .",
"industryIdentifiers": [
{
"type": "ISBN_10",
"identifier": "0718149785"
},
{
"type": "ISBN_13",
"identifier": "9780718149789"
}
],
"pageCount": 437,
"printType": "BOOK",
"averageRating": 3.0,
"ratingsCount": 27,
"contentVersion": "preview-1.0.0",
"imageLinks": {
"smallThumbnail": "http://bks7.books.google.com.au/books?id=189DPgAACAAJ&printsec=frontcover&img=1&zoom=5&source=gbs_api",
"thumbnail": "http://bks7.books.google.com.au/books?id=189DPgAACAAJ&printsec=frontcover&img=1&zoom=1&source=gbs_api"
},
"language": "en",
"previewLink": "http://books.google.com.au/books?id=189DPgAACAAJ&dq=isbn:9780718149789&hl=&cd=1&source=gbs_api",
"infoLink": "http://books.google.com.au/books?id=189DPgAACAAJ&dq=isbn:9780718149789&hl=&source=gbs_api",
"canonicalVolumeLink": "http://books.google.com.au/books/about/The_Navigator.html?hl=&id=189DPgAACAAJ"
},
"saleInfo": {
"country": "AU",
"saleability": "NOT_FOR_SALE",
"isEbook": false
},
"accessInfo": {
"country": "AU",
"viewability": "NO_PAGES",
"embeddable": false,
"publicDomain": false,
"textToSpeechPermission": "ALLOWED",
"epub": {
"isAvailable": false
},
"pdf": {
"isAvailable": false
},
"webReaderLink": "http://books.google.com.au/books/reader?id=189DPgAACAAJ&hl=&printsec=frontcover&output=reader&source=gbs_api",
"accessViewStatus": "NONE"
},
"searchInfo": {
"textSnippet": "Years ago, an ancient Phoenician statue known as the Navigator was stolen from the Baghdad Museum, and there are men who would do anything to get their hands on it."
}
}
]
}
Re: JSON encoder and decoder
Posted: Sat Jul 27, 2013 5:37 pm
by StarBootics
Hello Fangbeast,
I have play a little with your code and I finally make it to work. Since I'm on linux I have loaded the json file directly from my computer not over the internet but it should work.
Code: Select all
; 9780718149789 the navigator
; 071814788x polar shift
; 0552151696 digital fortress
; 0552149519 the davinci code
; 0552151769 pic deception point
; 0732911206 the lady of the sorrows
IncludeFile "JsonLib.pb"
; Define GoogleJsonFile.i = OpenFile(#PB_Any, "GoogleBookResults.txt")
; Define JsonString.s = ""
; Define FileStringFormat.i = ReadStringFormat(GoogleJsonFile.i)
Define.s URLString.s
Define.s CacheFile.s
; If InitNetwork()
;
; URLString.s = "https://www.googleapis.com/books/v1/volumes?projection=full&q=isbn:9780718149789"
;
; CacheFile.s = GetCurrentDirectory() + "GoogleBookResults.txt"
; If URLDownloadToFile_(0, URLString.s, CacheFile.s, 0, 0) = #S_OK
If ReadFile(0, "book.json")
Debug "Success"
While Eof(GoogleJsonFile.i) = #False
JsonString.s + ReadString(GoogleJsonFile.i, FileStringFormat.i) + Chr(13) + Chr(10)
Wend
CloseFile(GoogleJsonFile.i)
Define *out.jsonObj = JSON_decode(JsonString.s)
Debug "MapSize : " + Str(MapSize(*out\o()))
ResetMap(*out\o())
While NextMapElement(*out\o())
JSON_Debug(*out.jsonObj, MapKey(*out\o()))
; Debug MapKey(*out\o()) + " : " + *out\o()\o("url")\s
Wend
Else
Debug "Failed"
EndIf
; Else
;
; End
;
; EndIf
Then the debugger output :
Code: Select all
Success
MapSize : 3
totalItems (object) : {
totalItems(int) : 1
kind (string) : books#volumes
items (array) : [
1. (object) : {
searchInfo (object) : {
textSnippet (string) : Years ago, an ancient Phoenician statue known as the Navigator was stolen from the Baghdad Museum, and there are men who would do anything to get their hands on it.
}
volumeInfo (object) : {
language (string) : en
averageRating (float) : 3
canonicalVolumeLink (string) : http://books.google.com.au/books/about/The_Navigator.html?hl=&id=189DPgAACAAJ
description (string) : Years ago, an ancient Phoenician statue known as the Navigator was stolen from the Baghdad Museum, and there are men who would do anything to get their hands on it. The first victim is a crooked antiquities dealer, murdered in cold blood. Their second very nearly is a UN investigator who, were it not for the timely assistance of Austin and Zavala, would now be at the bottom of a very watery grave. What's so special about this statue? Austin wonders. The search for answers will take the NUMA team on an astonishing odyssey through time and space, one that encompasses no less than the lost treasures of King Solomon, a mysterious packet of documents personally encoded by Thomas Jefferson, and a top secret scientific project that could change the world forever. And that's before the surprises really begin . . .
subtitle (string) : A Kurt Austin Adventure, a Novel from the NUMA Files
title (string) : The Navigator
publishedDate (string) : 2007-05-20
ratingsCount(int) : 27
infoLink (string) : http://books.google.com.au/books?id=189DPgAACAAJ&dq=isbn:9780718149789&hl=&source=gbs_api
previewLink (string) : http://books.google.com.au/books?id=189DPgAACAAJ&dq=isbn:9780718149789&hl=&cd=1&source=gbs_api
pageCount(int) : 437
printType (string) : BOOK
imageLinks (object) : {
smallThumbnail (string) : http://bks7.books.google.com.au/books?id=189DPgAACAAJ&printsec=frontcover&img=1&zoom=5&source=gbs_api
thumbnail (string) : http://bks7.books.google.com.au/books?id=189DPgAACAAJ&printsec=frontcover&img=1&zoom=1&source=gbs_api
}
industryIdentifiers (array) : [
1. (object) : {
type (string) : ISBN_10
identifier (string) : 0718149785
}
2. (object) : {
type (string) : ISBN_13
identifier (string) : 9780718149789
}
]
publisher (string) : Michael Joseph
authors (array) : [
1. (string) : Clive Cussler
]
contentVersion (string) : preview-1.0.0
}
selfLink (string) : https://www.googleapis.com/books/v1/volumes/189DPgAACAAJ
accessInfo (object) : {
country (string) : AU
textToSpeechPermission (string) : ALLOWED
pdf (object) : {
isAvailable (false)
}
epub (object) : {
isAvailable (false)
}
accessViewStatus (string) : NONE
viewability (string) : NO_PAGES
publicDomain (false)
embeddable (false)
webReaderLink (string) : http://books.google.com.au/books/reader?id=189DPgAACAAJ&hl=&printsec=frontcover&output=reader&source=gbs_api
}
kind (string) : books#volume
saleInfo (object) : {
isEbook (false)
country (string) : AU
saleability (string) : NOT_FOR_SALE
}
id (string) : 189DPgAACAAJ
etag (string) : M7ZWVKH/6Gc
}
]
}
Now how you can use information just loaded it's a different story but if you take the time to study how the JSON_Debug() work I'm pretty sure someone like you will be able to do it.
Regards
StarBootics
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 2:01 am
by Fangbeast
Thanks StarBootics, studying the json lib, I didn't really understand it, that's why I had no idea how/why to get the output. Thanks for the pointer to the right direction.
I'm missing something though. How do I get access directly to my elements with the type in the loop as shown here in the original jsonlib page.
Code: Select all
Debug "direct Access:"
Debug "Nummer: " + *out\o("Nummer")\s
Debug "Vorname: " + *out\o("Inhaber")\o("Vorname")\s
Debug "Vorliebe 2: " + *out\o("Inhaber")\o("Vorlieben")\a(2)\s
Debug " "
I should be able to do this?
Debug "Text snippet: " + *out\o("textSnippet")\s
but nothing is returned
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 12:08 pm
by PMV
The "textSnippet" is part of the "items"-Array, isn't it?
So you need to go through it first
it should be something like this then:
Code: Select all
Debug *out\o("items")\a(0)\o("textSnippet")\s
MFG PMV
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 1:15 pm
by Fangbeast
PMV wrote:The "textSnippet" is part of the "items"-Array, isn't it?
So you need to go through it first
it should be something like this then:
Code: Select all
Debug *out\o("items")\a(0)\o("textSnippet")\s
MFG PMV
Sorry to be so thick, that didn't work. I find it hard to follow code where every variable is a single letter. If I understood it, I would use long variables that I could understand (grin), in the tradition of PureBasic.
This bit looks like ASCII art (*out\o("items")\a(0)\o)!!
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 2:46 pm
by PMV
Normally, i just write variables as long as possible, but in this case,
it would result in much to long lines. I prefere lines with 70 chars instead
of 700
But it is just an include, you can make your own
one by replacing the letters with the complete words.
If you understand JSON, it shouldn't be unpossible
o = object
a = array
s = string
... and so on
MFG PMV
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 8:55 pm
by Fangbeast
PMV wrote:Normally, i just write variables as long as possible, but in this case,
it would result in much to long lines. I prefere lines with 70 chars instead
of 700
But it is just an include, you can make your own
one by replacing the letters with the complete words.
If you understand JSON, it shouldn't be unpossible
o = object
a = array
s = string
... and so on
MFG PMV
Hohoho, a comedian yet. No, I don't understand json and no idea what I am doing wrong LOL. But thanks for the suggestion:):)
Re: JSON encoder and decoder
Posted: Sun Jul 28, 2013 9:37 pm
by PMV
Fangbeast wrote:PMV wrote:The "textSnippet" is part of the "items"-Array, isn't it?
So you need to go through it first
it should be something like this then:
Code: Select all
Debug *out\o("items")\a(0)\o("textSnippet")\s
MFG PMV
Sorry to be so thick, that didn't work.
Of course, because there is another object surrounding the textSnippet
Right is this:
Code: Select all
*out\o("items")\a(0)\o("searchInfo")\o("textSnippet")\s
1. root element (*out)
2. object "items" that contains an array
3. array has just one element, so index is 0
4. object "SearchInfo" that has just one
5. object in it that is your "textSnippet"
6. access the data, in this case it is a string so it is the structure-field "s"
MFG PMV
Re: JSON encoder and decoder
Posted: Mon Jul 29, 2013 12:33 pm
by Fangbeast
I bow to your munificense and help. My tiny brain is happy! I should bowwow an animated graphic from KCC as thanks to you:):)
Bugger! I am getting a syntax error with that line. I forgot to put something in front of it LOL!
Fixed!
D'oh!! I finally thought to capture the file from the browser instead of the program. Finally I can see from the indenting which section belongs to which object!
Re: JSON encoder and decoder
Posted: Thu Aug 01, 2013 5:33 am
by Fangbeast
One last question please PMV. This section
Code: Select all
epub (object) : {
isAvailable (false)
}
I thought I could get it like this :
Code: Select all
Debug "epub is available: " + *out\o("items")\a(0)\o("accessInfo")\o("epub")\o("isAvailable")\s
But no response as it is a true/false return (not a string \s type) and I don't know how to handle it?
Otherwise I have all the other fields coming back now (Phew!)