In PB: Parsing/Wrintg XML faster than JSON?

Lebostein · **Joined:** Fri Jun 11, 2004 7:07 am **Posts:** 609

I wonder why. All other speed tests in the web with both formats and different parsers say, that JSON handling is much faster than XML, but in PureBasic the XML parser gives better results while loading and saving with small and huge files. Is there a slow JSON parser inside PB?

Lebostein · **Joined:** Fri Jun 11, 2004 7:07 am **Posts:** 609

Small test code:

Code:

EnableExplicit
RandomSeed(12345)
#max = 20000

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
SaveXML(0, "test.xml")
xs = ElapsedMilliseconds() - t0
FreeXML(0)

; Load XML
CreateXML(0)
t0 = ElapsedMilliseconds()
LoadXML(0, "test.xml")
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
SaveJSON(0, "test.json")
js = ElapsedMilliseconds() - t0
FreeJSON(0)

; Load JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
LoadJSON(0, "test.json")
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Save XML: " + Str(xs) + #LF$
msg$ + "Save JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Load XML: " + Str(xl) + #LF$
msg$ + "Load JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)

my results (smallest values after 3 turns):

Save XML: 255 (~ 9,1 MB written data)
Save JSON: 294 (~ 6,7 MB only, but slower)

Load XML: 353 (~ 9,1 MB read data)
Load JSON: 433 (~ 6,7 MB only, but slower)

NicTheQuick · **Posted:** Wed Jun 06, 2018 11:44 am

On Linux I get these results:

Quote:

Save XML: 213
Save JSON: 214

Load XML: 207
Load JSON: 167

So saving is the same speed but reading is faster with JSON. I repeated the test several times and the results were pretty much the same every time.

Edit:
With #max = 200000 this is the result

Quote:

Save XML: 2178
Save JSON: 5277

Load XML: 2391
Load JSON: 5041

So here JSON is definitely slower.

davido · **Posted:** Wed Jun 06, 2018 1:29 pm

Decided that I would check this out on my system:
Windows 10, PB 5.70b1
It didn't work - got this error:

[13:24:49] Executable started.
[13:24:50] [ERROR] Line: 50
[13:24:50] [ERROR] Invalid XML Node. (0)

Any one, perchance, know why?

kenmo · **Joined:** Tue Dec 23, 2003 3:54 am **Posts:** 1705

Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)

I get: JSON is ~4x slower to Insert, and ~3x faster to Extract

Code:

EnableExplicit
;RandomSeed(12345)
#max = 20000

CompilerIf (#PB_Compiler_Debugger)
  CompilerError "Turn off Debugger for accurate timing"
CompilerEndIf

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
xs = ElapsedMilliseconds() - t0
SaveXML(0, "test.xml")
FreeXML(0)

; Load XML
LoadXML(0, "test.xml")
t0 = ElapsedMilliseconds()
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
js = ElapsedMilliseconds() - t0
SaveJSON(0, "test.json")
FreeJSON(0)

; Load JSON
LoadJSON(0, "test.json")
t0 = ElapsedMilliseconds()
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Insert XML: " + Str(xs) + #LF$
msg$ + "Insert JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Extract XML: " + Str(xl) + #LF$
msg$ + "Extract JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)

Lebostein · **Joined:** Fri Jun 11, 2004 7:07 am **Posts:** 609

kenmo wrote:

Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)

1. And you know, that the Save*() and Load*() commands are only writing and saving bytes? I think (I don't know it) these commands are a part of the parsers and can contain prepare / postprocessing algorithms (in memory) before writing / after reading....
2. If you turn off the RandomSeed(), every run of the program generates different data structures and you can not compare it with each other...

kenmo · **Joined:** Tue Dec 23, 2003 3:54 am **Posts:** 1705

Yes I agree, the Save/Load commands probably call the Compose/Parse functions, which are another thing to test. It's not just direct copying to filesystem.

I guess my point was: XML is not faster at every operation. Maybe most...

Oh, the RandomSeed I commented out to see different cases.

NicTheQuick · **Posted:** Wed Jun 06, 2018 4:27 pm

Use /dev/shm/test.xml on Linux to test against memory instead of harddisk.

PureBasic Forum

In PB: Parsing/Wrintg XML faster than JSON?

Who is online