In PB: Parsing/Wrintg XML faster than JSON?

Everything else that doesn't fall into one of the other PB categories.
Lebostein
Addict
Addict
Posts: 807
Joined: Fri Jun 11, 2004 7:07 am

In PB: Parsing/Wrintg XML faster than JSON?

Post by Lebostein »

I wonder why. All other speed tests in the web with both formats and different parsers say, that JSON handling is much faster than XML, but in PureBasic the XML parser gives better results while loading and saving with small and huge files. Is there a slow JSON parser inside PB?
Last edited by Lebostein on Wed Jun 06, 2018 11:03 am, edited 1 time in total.
Lebostein
Addict
Addict
Posts: 807
Joined: Fri Jun 11, 2004 7:07 am

Re: In PB: Parsing/Wrintg XML faster than JSON

Post by Lebostein »

Small test code:

Code: Select all

EnableExplicit
RandomSeed(12345)
#max = 20000

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
SaveXML(0, "test.xml")
xs = ElapsedMilliseconds() - t0
FreeXML(0)

; Load XML
CreateXML(0)
t0 = ElapsedMilliseconds()
LoadXML(0, "test.xml")
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
SaveJSON(0, "test.json")
js = ElapsedMilliseconds() - t0
FreeJSON(0)

; Load JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
LoadJSON(0, "test.json")
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Save XML: " + Str(xs) + #LF$
msg$ + "Save JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Load XML: " + Str(xl) + #LF$
msg$ + "Load JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)
my results (smallest values after 3 turns):

Save XML: 255 (~ 9,1 MB written data)
Save JSON: 294 (~ 6,7 MB only, but slower)

Load XML: 353 (~ 9,1 MB read data)
Load JSON: 433 (~ 6,7 MB only, but slower)
User avatar
NicTheQuick
Addict
Addict
Posts: 1224
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by NicTheQuick »

On Linux I get these results:
Save XML: 213
Save JSON: 214

Load XML: 207
Load JSON: 167
So saving is the same speed but reading is faster with JSON. I repeated the test several times and the results were pretty much the same every time.

Edit:
With #max = 200000 this is the result
Save XML: 2178
Save JSON: 5277

Load XML: 2391
Load JSON: 5041
So here JSON is definitely slower.
Last edited by NicTheQuick on Wed Jun 06, 2018 1:30 pm, edited 1 time in total.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
davido
Addict
Addict
Posts: 1890
Joined: Fri Nov 09, 2012 11:04 pm
Location: Uttoxeter, UK

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by davido »

Decided that I would check this out on my system:
Windows 10, PB 5.70b1
It didn't work - got this error:

[13:24:49] Executable started.
[13:24:50] [ERROR] Line: 50
[13:24:50] [ERROR] Invalid XML Node. (0)

Any one, perchance, know why?
DE AA EB
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by kenmo »

Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)

I get: JSON is ~4x slower to Insert, and ~3x faster to Extract

Code: Select all

EnableExplicit
;RandomSeed(12345)
#max = 20000

CompilerIf (#PB_Compiler_Debugger)
  CompilerError "Turn off Debugger for accurate timing"
CompilerEndIf

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
xs = ElapsedMilliseconds() - t0
SaveXML(0, "test.xml")
FreeXML(0)

; Load XML
LoadXML(0, "test.xml")
t0 = ElapsedMilliseconds()
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
js = ElapsedMilliseconds() - t0
SaveJSON(0, "test.json")
FreeJSON(0)

; Load JSON
LoadJSON(0, "test.json")
t0 = ElapsedMilliseconds()
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Insert XML: " + Str(xs) + #LF$
msg$ + "Insert JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Extract XML: " + Str(xl) + #LF$
msg$ + "Extract JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)
Lebostein
Addict
Addict
Posts: 807
Joined: Fri Jun 11, 2004 7:07 am

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by Lebostein »

kenmo wrote:Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)
1. And you know, that the Save*() and Load*() commands are only writing and saving bytes? I think (I don't know it) these commands are a part of the parsers and can contain prepare / postprocessing algorithms (in memory) before writing / after reading....
2. If you turn off the RandomSeed(), every run of the program generates different data structures and you can not compare it with each other...
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by kenmo »

Yes I agree, the Save/Load commands probably call the Compose/Parse functions, which are another thing to test. It's not just direct copying to filesystem.

I guess my point was: XML is not faster at every operation. Maybe most...

Oh, the RandomSeed I commented out to see different cases.
User avatar
NicTheQuick
Addict
Addict
Posts: 1224
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by NicTheQuick »

Use /dev/shm/test.xml on Linux to test against memory instead of harddisk.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Lebostein
Addict
Addict
Posts: 807
Joined: Fri Jun 11, 2004 7:07 am

Re: In PB: Parsing/Wrintg XML faster than JSON?

Post by Lebostein »

Same problem here. In all my applications I added both mehtods (JSON and XML). And no matter what I test, XML is always faster in PureBasic than JSON, both in parsing and composing. And I also keep reading that XML is supposed to be faster. There also to me the question arises whether the JSON parser in PureBasic does not run optimally...
Post Reply