In PB: Parsing/Wrintg XML faster than JSON?
In PB: Parsing/Wrintg XML faster than JSON?
I wonder why. All other speed tests in the web with both formats and different parsers say, that JSON handling is much faster than XML, but in PureBasic the XML parser gives better results while loading and saving with small and huge files. Is there a slow JSON parser inside PB?
Last edited by Lebostein on Wed Jun 06, 2018 11:03 am, edited 1 time in total.
Re: In PB: Parsing/Wrintg XML faster than JSON
Small test code:
my results (smallest values after 3 turns):
Save XML: 255 (~ 9,1 MB written data)
Save JSON: 294 (~ 6,7 MB only, but slower)
Load XML: 353 (~ 9,1 MB read data)
Load JSON: 433 (~ 6,7 MB only, but slower)
Code: Select all
EnableExplicit
RandomSeed(12345)
#max = 20000
Procedure.s getrandomstring(): Protected output$, i
output$ = ""
For i = 50 To Random(500, 100)
output$ + Chr(Random(126,32))
Next i
ProcedureReturn output$
EndProcedure
Structure xx
string.s
float.d
EndStructure
Structure test
Array myarray.xx(#max)
List mylist.xx()
Map mymap.xx()
EndStructure
Define tree.test, i, t0, xs, xl, js, jl, msg$
; Fill the structure with random values
For i = 0 To #max
tree\myarray(i)\float = Random(100000) / #PI
tree\myarray(i)\string = getrandomstring()
AddElement(tree\mylist())
tree\mylist()\float = Random(100000) / #PI
tree\mylist()\string = getrandomstring()
AddMapElement(tree\mymap(), Str(i))
tree\mymap()\float = Random(100000) / #PI
tree\mymap()\string = getrandomstring()
Next i
; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
SaveXML(0, "test.xml")
xs = ElapsedMilliseconds() - t0
FreeXML(0)
; Load XML
CreateXML(0)
t0 = ElapsedMilliseconds()
LoadXML(0, "test.xml")
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)
; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
SaveJSON(0, "test.json")
js = ElapsedMilliseconds() - t0
FreeJSON(0)
; Load JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
LoadJSON(0, "test.json")
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)
msg$ + "Save XML: " + Str(xs) + #LF$
msg$ + "Save JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Load XML: " + Str(xl) + #LF$
msg$ + "Load JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)
Save XML: 255 (~ 9,1 MB written data)
Save JSON: 294 (~ 6,7 MB only, but slower)
Load XML: 353 (~ 9,1 MB read data)
Load JSON: 433 (~ 6,7 MB only, but slower)
- NicTheQuick
- Addict
- Posts: 1226
- Joined: Sun Jun 22, 2003 7:43 pm
- Location: Germany, Saarbrücken
- Contact:
Re: In PB: Parsing/Wrintg XML faster than JSON?
On Linux I get these results:
Edit:
With #max = 200000 this is the result
So saving is the same speed but reading is faster with JSON. I repeated the test several times and the results were pretty much the same every time.Save XML: 213
Save JSON: 214
Load XML: 207
Load JSON: 167
Edit:
With #max = 200000 this is the result
So here JSON is definitely slower.Save XML: 2178
Save JSON: 5277
Load XML: 2391
Load JSON: 5041
Last edited by NicTheQuick on Wed Jun 06, 2018 1:30 pm, edited 1 time in total.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Re: In PB: Parsing/Wrintg XML faster than JSON?
Decided that I would check this out on my system:
Windows 10, PB 5.70b1
It didn't work - got this error:
[13:24:49] Executable started.
[13:24:50] [ERROR] Line: 50
[13:24:50] [ERROR] Invalid XML Node. (0)
Any one, perchance, know why?
Windows 10, PB 5.70b1
It didn't work - got this error:
[13:24:49] Executable started.
[13:24:50] [ERROR] Line: 50
[13:24:50] [ERROR] Invalid XML Node. (0)
Any one, perchance, know why?
DE AA EB
Re: In PB: Parsing/Wrintg XML faster than JSON?
Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)
I get: JSON is ~4x slower to Insert, and ~3x faster to Extract
(Filesystem access is slower than in-memory processing!)
I get: JSON is ~4x slower to Insert, and ~3x faster to Extract
Code: Select all
EnableExplicit
;RandomSeed(12345)
#max = 20000
CompilerIf (#PB_Compiler_Debugger)
CompilerError "Turn off Debugger for accurate timing"
CompilerEndIf
Procedure.s getrandomstring(): Protected output$, i
output$ = ""
For i = 50 To Random(500, 100)
output$ + Chr(Random(126,32))
Next i
ProcedureReturn output$
EndProcedure
Structure xx
string.s
float.d
EndStructure
Structure test
Array myarray.xx(#max)
List mylist.xx()
Map mymap.xx()
EndStructure
Define tree.test, i, t0, xs, xl, js, jl, msg$
; Fill the structure with random values
For i = 0 To #max
tree\myarray(i)\float = Random(100000) / #PI
tree\myarray(i)\string = getrandomstring()
AddElement(tree\mylist())
tree\mylist()\float = Random(100000) / #PI
tree\mylist()\string = getrandomstring()
AddMapElement(tree\mymap(), Str(i))
tree\mymap()\float = Random(100000) / #PI
tree\mymap()\string = getrandomstring()
Next i
; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
xs = ElapsedMilliseconds() - t0
SaveXML(0, "test.xml")
FreeXML(0)
; Load XML
LoadXML(0, "test.xml")
t0 = ElapsedMilliseconds()
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)
; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
js = ElapsedMilliseconds() - t0
SaveJSON(0, "test.json")
FreeJSON(0)
; Load JSON
LoadJSON(0, "test.json")
t0 = ElapsedMilliseconds()
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)
msg$ + "Insert XML: " + Str(xs) + #LF$
msg$ + "Insert JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Extract XML: " + Str(xl) + #LF$
msg$ + "Extract JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)
Re: In PB: Parsing/Wrintg XML faster than JSON?
1. And you know, that the Save*() and Load*() commands are only writing and saving bytes? I think (I don't know it) these commands are a part of the parsers and can contain prepare / postprocessing algorithms (in memory) before writing / after reading....kenmo wrote:Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)
2. If you turn off the RandomSeed(), every run of the program generates different data structures and you can not compare it with each other...
Re: In PB: Parsing/Wrintg XML faster than JSON?
Yes I agree, the Save/Load commands probably call the Compose/Parse functions, which are another thing to test. It's not just direct copying to filesystem.
I guess my point was: XML is not faster at every operation. Maybe most...
Oh, the RandomSeed I commented out to see different cases.
I guess my point was: XML is not faster at every operation. Maybe most...
Oh, the RandomSeed I commented out to see different cases.
- NicTheQuick
- Addict
- Posts: 1226
- Joined: Sun Jun 22, 2003 7:43 pm
- Location: Germany, Saarbrücken
- Contact:
Re: In PB: Parsing/Wrintg XML faster than JSON?
Use /dev/shm/test.xml on Linux to test against memory instead of harddisk.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Re: In PB: Parsing/Wrintg XML faster than JSON?
Same problem here. In all my applications I added both mehtods (JSON and XML). And no matter what I test, XML is always faster in PureBasic than JSON, both in parsing and composing. And I also keep reading that XML is supposed to be faster. There also to me the question arises whether the JSON parser in PureBasic does not run optimally...