The test code can be found near the end of the post.
After the test results are some comments on the test.
The buffer size is in bytes and in KB alignments.
The timings are in milliseconds, so 19086 = 19.086 seconds.
In the test I set core affinity (found in Taskmanager if you right click a process) to the second core on my 3 core cpu and unchecked core 0 and 2.FileBuffersSize() results:
Buffer: 4096
ReadByte: 19086
ReadChar: 10309
ReadWord: 10034
ReadLong: 5405
ReadQuad: 3311
ReadFloat: 5602
ReadDouble: 3392
ReadData: 484
Buffer: 8192
ReadByte: 18874
ReadChar: 10068
ReadWord: 9862
ReadLong: 5213
ReadQuad: 3109
ReadFloat: 5385
ReadDouble: 3240
ReadData: 397
Buffer: 16384
ReadByte: 18716
ReadChar: 9983
ReadWord: 9730
ReadLong: 5094
ReadQuad: 3009
ReadFloat: 5267
ReadDouble: 3108
ReadData: 328
Buffer: 32768
ReadByte: 18698
ReadChar: 9959
ReadWord: 9729
ReadLong: 5087
ReadQuad: 3000
ReadFloat: 5273
ReadDouble: 3111
ReadData: 404
Buffer: 65536
ReadByte: 18677
ReadChar: 9945
ReadWord: 9740
ReadLong: 5071
ReadQuad: 2987
ReadFloat: 5249
ReadDouble: 3095
ReadData: 383
Buffer: 131072
ReadByte: 18674
ReadChar: 9969
ReadWord: 9697
ReadLong: 5058
ReadQuad: 2980
ReadFloat: 5243
ReadDouble: 3084
ReadData: 367
Buffer: 262144
ReadByte: 18664
ReadChar: 9955
ReadWord: 9725
ReadLong: 5066
ReadQuad: 2980
ReadFloat: 5247
ReadDouble: 3081
ReadData: 400
Buffer: 524288
ReadByte: 18843
ReadChar: 9996
ReadWord: 9779
ReadLong: 5132
ReadQuad: 3037
ReadFloat: 5306
ReadDouble: 3125
ReadData: 443
Buffer: 1048576
ReadByte: 18984
ReadChar: 10143
ReadWord: 9921
ReadLong: 5208
ReadQuad: 3128
ReadFloat: 5396
ReadDouble: 3226
ReadData: 576
ReadData size was the same as the filebuffer size in the tests.
As you can see, nothing beats ReadData() so the conclusion is that if you know what to read and how much you need, then use ReadData()
It is advised that the ReadData chunks and filebuffersize matches.
In general one can see that the larger the filebuffersize the faster the reads are, however one can see that at a certain point (on my system)
it hit a limit. This could possibly be the OS filesystem or the hard drive buffer limit, and the gain began to be less.
It seems that 32KB, 64KB and 128KB seems to be among the better general choices.
Certain data might benefit from other buffersizes, especially in the case of ReadData, like when streaming audio or video or other data.
My advice is to sync up the sizes to create a double, tripple or a cascading buffering effect, thus you avoid overbuffering or underbuffering which might reduce performance.
Another thing to keep in mind is that the larger the buffer, the less disk reads are needed, and the less filesystem access and thus less CPU use.
If a lot of disk activity is expected or simultaneous processing then larger buffers are advised.
A large test file is advised.
I used the dxsdk_march2008.exe (Direct X march 2008 installer) which was 452751KB.
Feel free to post your own speed tests in this thread,
maybe we can find a nice buffersize "default" that is the most optimal across many systems. The PureBasic default is currently set at 4096 bytes (4KB).
Code: Select all
DisableDebugger
EnableExplicit
Procedure.s test(bufsize.i,file$)
#File1=1
Protected start.l,stop.l,text$,*mem
text$="Buffer: "+Str(bufsize)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadByte(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadByte: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadCharacter(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadChar: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadWord(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadWord: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadLong(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadLong: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadQuad(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadQuad: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadFloat(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadFloat: "+Str(stop-start)+#CRLF$
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadDouble(#File1)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
text$+"ReadDouble: "+Str(stop-start)+#CRLF$
*mem=AllocateMemory(bufsize)
If *mem
If ReadFile(#File1,file$)
FileBuffersSize(#File1,bufsize)
start=timeGetTime_()
While Eof(#File1)=#False
ReadData(#File1,*mem,bufsize)
Wend
stop=timeGetTime_()
CloseFile(#File1)
EndIf
FreeMemory(*mem)
EndIf
text$+"ReadData: "+Str(stop-start)+#CRLF$
text$+#CRLF$
ProcedureReturn text$
EndProcedure
timeBeginPeriod_(1)
Define file$,text$
file$="E:\Downloads\dxsdk_march2008.exe"
text$="FileBuffersSize() results:"+#CRLF$
text$+test(4096,file$)
text$+test(8192,file$)
text$+test(16384,file$)
text$+test(32768,file$)
text$+test(65536,file$)
text$+test(131072,file$)
text$+test(262144,file$)
text$+test(524288,file$)
text$+test(1048576,file$)
MessageRequester("test()",text$)
timeEndPeriod_(1)
There will always be the odd system where nothing makes sense heh.