It is currently Sat Aug 08, 2020 4:48 am

View unanswered posts | View active topics


Board index » PureBasic » General Discussion

All times are UTC + 1 hour




Post new topic Reply to topic  Page 1 of 1
  [ 8 posts ] 
  Print view Previous topic | Next topic 
Author Message
Lebostein
 Post subject: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 10:25 am 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Fri Jun 11, 2004 7:07 am
Posts: 635
I wonder why. All other speed tests in the web with both formats and different parsers say, that JSON handling is much faster than XML, but in PureBasic the XML parser gives better results while loading and saving with small and huge files. Is there a slow JSON parser inside PB?


Last edited by Lebostein on Wed Jun 06, 2018 11:03 am, edited 1 time in total.

Top
 Profile  
Reply with quote  
Lebostein
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON
PostPosted: Wed Jun 06, 2018 10:59 am 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Fri Jun 11, 2004 7:07 am
Posts: 635
Small test code:

Code:
EnableExplicit
RandomSeed(12345)
#max = 20000

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
SaveXML(0, "test.xml")
xs = ElapsedMilliseconds() - t0
FreeXML(0)

; Load XML
CreateXML(0)
t0 = ElapsedMilliseconds()
LoadXML(0, "test.xml")
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
SaveJSON(0, "test.json")
js = ElapsedMilliseconds() - t0
FreeJSON(0)

; Load JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
LoadJSON(0, "test.json")
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Save XML: " + Str(xs) + #LF$
msg$ + "Save JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Load XML: " + Str(xl) + #LF$
msg$ + "Load JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)


my results (smallest values after 3 turns):

Save XML: 255 (~ 9,1 MB written data)
Save JSON: 294 (~ 6,7 MB only, but slower)

Load XML: 353 (~ 9,1 MB read data)
Load JSON: 433 (~ 6,7 MB only, but slower)
Top
 Profile  
Reply with quote  
NicTheQuick
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 11:44 am 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sun Jun 22, 2003 7:43 pm
Posts: 567
Location: Germany, Saarbrücken
On Linux I get these results:
Quote:
Save XML: 213
Save JSON: 214

Load XML: 207
Load JSON: 167

So saving is the same speed but reading is faster with JSON. I repeated the test several times and the results were pretty much the same every time.

Edit:
With #max = 200000 this is the result
Quote:
Save XML: 2178
Save JSON: 5277

Load XML: 2391
Load JSON: 5041

So here JSON is definitely slower.

_________________
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.

Last edited by NicTheQuick on Wed Jun 06, 2018 1:30 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
davido
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 1:29 pm 
Offline
Addict
Addict

Joined: Fri Nov 09, 2012 11:04 pm
Posts: 1773
Location: Uttoxeter, UK
Decided that I would check this out on my system:
Windows 10, PB 5.70b1
It didn't work - got this error:

[13:24:49] Executable started.
[13:24:50] [ERROR] Line: 50
[13:24:50] [ERROR] Invalid XML Node. (0)

Any one, perchance, know why?

_________________
DE AA EB

Top
 Profile  
Reply with quote  
kenmo
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 1:44 pm 
Offline
Addict
Addict
User avatar

Joined: Tue Dec 23, 2003 3:54 am
Posts: 1839
Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)

I get: JSON is ~4x slower to Insert, and ~3x faster to Extract

Code:
EnableExplicit
;RandomSeed(12345)
#max = 20000

CompilerIf (#PB_Compiler_Debugger)
  CompilerError "Turn off Debugger for accurate timing"
CompilerEndIf

Procedure.s getrandomstring(): Protected output$, i
  output$ = ""
  For i = 50 To Random(500, 100)
    output$ + Chr(Random(126,32))
  Next i
  ProcedureReturn output$
EndProcedure

Structure xx
  string.s
  float.d
EndStructure

Structure test
  Array myarray.xx(#max)
  List mylist.xx()
  Map mymap.xx()
EndStructure

Define tree.test, i, t0, xs, xl, js, jl, msg$

; Fill the structure with random values
For i = 0 To #max
  tree\myarray(i)\float = Random(100000) / #PI
  tree\myarray(i)\string = getrandomstring()
  AddElement(tree\mylist())
  tree\mylist()\float = Random(100000) / #PI
  tree\mylist()\string = getrandomstring()
  AddMapElement(tree\mymap(), Str(i))
  tree\mymap()\float = Random(100000) / #PI
  tree\mymap()\string = getrandomstring()
Next i

; Save XML
CreateXML(0)
t0 = ElapsedMilliseconds()
InsertXMLStructure(RootXMLNode(0), tree, test)
xs = ElapsedMilliseconds() - t0
SaveXML(0, "test.xml")
FreeXML(0)

; Load XML
LoadXML(0, "test.xml")
t0 = ElapsedMilliseconds()
ExtractXMLStructure(MainXMLNode(0), tree, test)
xl = ElapsedMilliseconds() - t0
FreeXML(0)

; Save JSON
CreateJSON(0)
t0 = ElapsedMilliseconds()
InsertJSONStructure(JSONValue(0), tree, test)
js = ElapsedMilliseconds() - t0
SaveJSON(0, "test.json")
FreeJSON(0)

; Load JSON
LoadJSON(0, "test.json")
t0 = ElapsedMilliseconds()
ExtractJSONStructure(JSONValue(0), tree, test)
jl = ElapsedMilliseconds() - t0
FreeJSON(0)

msg$ + "Insert XML: " + Str(xs) + #LF$
msg$ + "Insert JSON: " + Str(js) + #LF$
msg$ + #LF$
msg$ + "Extract XML: " + Str(xl) + #LF$
msg$ + "Extract JSON: " + Str(jl) + #LF$
MessageRequester("X", msg$)
SetClipboardText(msg$)

_________________
On GitHub: PB Includes - IDE Tools - Color Themes - IDE Branches - TabBarGadget Mods

Top
 Profile  
Reply with quote  
Lebostein
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 4:02 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Fri Jun 11, 2004 7:07 am
Posts: 635
kenmo wrote:
Slightly changed the code, so the file save/load isn't included in the timing.
(Filesystem access is slower than in-memory processing!)

1. And you know, that the Save*() and Load*() commands are only writing and saving bytes? I think (I don't know it) these commands are a part of the parsers and can contain prepare / postprocessing algorithms (in memory) before writing / after reading....
2. If you turn off the RandomSeed(), every run of the program generates different data structures and you can not compare it with each other...
Top
 Profile  
Reply with quote  
kenmo
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 4:09 pm 
Offline
Addict
Addict
User avatar

Joined: Tue Dec 23, 2003 3:54 am
Posts: 1839
Yes I agree, the Save/Load commands probably call the Compose/Parse functions, which are another thing to test. It's not just direct copying to filesystem.

I guess my point was: XML is not faster at every operation. Maybe most...

Oh, the RandomSeed I commented out to see different cases.

_________________
On GitHub: PB Includes - IDE Tools - Color Themes - IDE Branches - TabBarGadget Mods

Top
 Profile  
Reply with quote  
NicTheQuick
 Post subject: Re: In PB: Parsing/Wrintg XML faster than JSON?
PostPosted: Wed Jun 06, 2018 4:27 pm 
Offline
Enthusiast
Enthusiast
User avatar

Joined: Sun Jun 22, 2003 7:43 pm
Posts: 567
Location: Germany, Saarbrücken
Use /dev/shm/test.xml on Linux to test against memory instead of harddisk.

_________________
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.

Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 8 posts ] 

Board index » PureBasic » General Discussion

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 28 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

 


Powered by phpBB © 2008 phpBB Group
subSilver+ theme by Canver Software, sponsor Sanal Modifiye