[Implemented] FileSize: Try this speed test

BackupUser · Post by **BackupUser** » Sat Mar 30, 2002 9:43 pm

Restored from previous forum. Originally posted by PB.

UPDATE: Please try my speed test further down... I think the FileSize command
is very slow and might need improvement. Thanks!

PB - Registered PureBasic Coder

Edited by - PB on 31 March 2002 19:03:16

BackupUser · Post by **BackupUser** » Sat Mar 30, 2002 10:03 pm

Restored from previous forum. Originally posted by skypa.

Try using the GetFileSize_ API instead of the PB command.

BackupUser · Post by **BackupUser** » Sat Mar 30, 2002 10:19 pm

Restored from previous forum. Originally posted by PB.

> Try using the GetFileSize_ API instead of the PB command.

I replaced the offending line above with this code, but it was still just
as slow, unfortunately:

Code: Select all

b$=Space(128) : f=OpenFile_(dir$+a$,b$,#OF_READ)
If f0 : fs=GetFileSize_(f,0) : _lclose_(f) : EndIf

PB - Registered PureBasic Coder

BackupUser · Post by **BackupUser** » Sat Mar 30, 2002 10:33 pm

Restored from previous forum. Originally posted by tinman.

> Try using the GetFileSize_ API instead of the PB command.

I replaced the offending line above with this code, but it was still just
as slow, unfortunately:

1) How fast does Windows take if you get the properties for the
C:\Test directory?

2) Have you tried not doing dir$+a$ in the command call? Perhaps
replace it with a fixed size buffer, preload the first part with dir$
and then poke a$ into the middle of the array each time round the
loop.

--
It's not minimalist - I'm increasing efficiency by reducing input effort.

BackupUser · Post by **BackupUser** » Sat Mar 30, 2002 10:42 pm

Restored from previous forum. Originally posted by freak.

My suggestion would have been to open the file, and use the
Lof() command, but I tried it, and it is as slow as the other command.

Timo

A debugged program is one for which you have not yet found the conditions that make it fail.

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 2:00 am

Restored from previous forum. Originally posted by Pupil.

I'm using the following code to read a folder that has only 288 files in it, yet
the routine takes 4-5 seconds, yuck! If I remove the FileSize command, the
routine is INSTANT, so FileSize is the bottleneck. Is is possible to make
it faster at all? If not, that's cool, but I'm hoping there's a way to speed
it up somehow...

PB - Registered PureBasic Coder

I've tried this also but my directory contained 20k of files, it took forever

I'm not familiar with the inner structure of the filesystem, but wouldn't it be faster if you in one go could get both name and size. I.e. have something like this:

Code: Select all

dir$="C:\Test\"
  If ExamineDirectory(0,dir$,"*.*")
  Repeat
    type=NextDirectoryEntry()
    If type=1 ; File.
      a$=DirectoryEntryName()
      fs=DirectoryEntrySize() ; this command doesn't exist but would be nice to have;)
      t=t+fs
    EndIf
  Until type=0
  MessageRequester("Total","Total file sizes: "+Str(t),0)
EndIf

Perhaps we even could get the file attributes with:
DirectoryEntryAttribute()
and set which entrys we want from a directory with something like the following:

Code: Select all

#PB_EntryName = 0
#PB_EntrySize = 1
#PB_EntryAttribute = 2

If ExamineDirectory(0, "C:\", "*.*", #PB_EntryName|#PB_EntrySize|#PB_EntryAttribute)
  Repeat
    If NextDirectoryEntry() = 1
      a$ = DirectoryEntryName()
      size.l = DirectoryEntrySize()
      attrib.l = DirectoryEntryAttribute()
      ...
    EndIf
  until ...
EndIf

Edited by - Pupil on 31 March 2002 02:18:19

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 10:51 am

Restored from previous forum. Originally posted by PB.

> How fast does Windows take if you get the properties for the directory?

Virtually instant, just like my example with FileSize removed.

> Have you tried not doing dir$+a$ in the command call?

The dir$+a$ is not the problem, because if you replace a=FileSize(dir$+a$) with
tmp$=dir$+a$, so that FileSize is NOT called, the routine is then instant, so
it's definitely FileSize causing the slowdown, not the combining of strings.

PB - Registered PureBasic Coder

Edited by - PB on 31 March 2002 11:52:22

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 4:11 pm

Restored from previous forum. Originally posted by blueb.

Don't know if it's slow.

I tried it on my development folder (c:\BobDev)

From the properties form:
It contains:
347 megabytes
3700 files
306 folders

I have a P-III 800 and it's less than a second (virtually instant)
It returns 29,071,843 which I assume is the size of type 1 files.

Regards,
--Bob

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 5:38 pm

Restored from previous forum. Originally posted by skypa.

Same by me. I used PB's code to examine the total filesize of my Windows Dir, and it takes nearly 1 second.

Windows XP, AMD K6/2 400(!), 330 MB RAM

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 5:59 pm

Restored from previous forum. Originally posted by PB.

> I used PB's code to examine the total filesize of my Windows Dir,
> and it takes nearly 1 second.

***** PB's SPEED TEST *****

Please try this test for me and let me know if the average results shown are
nearly equal. For me, they differ greatly, and I'm talking like 29500 with FileSize,
compared to a tiny 20 without...

Code: Select all

; Make dir$ any folder with approx 500 files.
dir$="C:\Windows\"
;
Procedure TestSpeed(fs)
  Shared dir$
  If ExamineDirectory(0,dir$,"*.*")0
    clock1=GetTickCount_()
    Repeat
      type=NextDirectoryEntry()
      If type=1 ; File.
        a$=DirectoryEntryName()
        If fs=1
          a=FileSize(dir$+a$) ; BOTTLENECK?
        EndIf
      EndIf
    Until type=0 ; No more entries.
    clock2=GetTickCount_()
    ProcedureReturn clock2-clock1
  EndIf
EndProcedure
;
FileSizeY=0 ; Average speed WITH FileSize.
FileSizeN=0 ; Average speed WITHOUT FileSize.
;
; Test WITH FileSize.
b=0 : For r=1 To 3 : b=b+TestSpeed(1) : Next : FileSizeY=b/3
;
; Test WITHOUT FileSize.
b=0 : For r=1 To 3 : b=b+TestSpeed(0) : Next : FileSizeN=b/3
;
MessageRequester("Averages","With: "+str(FileSizeY)+Chr(13)+"Without: "+str(FileSizeN),0)

PB - Registered PureBasic Coder

Edited by - PB on 31 March 2002 19:39:21

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 6:43 pm

Restored from previous forum. Originally posted by skypa.

"With:50
Without: 3"

From Beginning to MessageBox -> virtual instant

[edit] maybe it has something to do with FAT32(bigger cluster) and NTSF

Edited by - skypa on 31 March 2002 19:48:00

BackupUser · Post by **BackupUser** » Sun Mar 31, 2002 8:10 pm

Restored from previous forum. Originally posted by fred.

Ok, I see the problem

. So as Pupil suggested it, 2 commands needs to be added: DirectoryEntrySize() and DirectoryEntryAttributes() and all should be fine. For 3.1. BTW, the v3.0 is planned for the next week... Wait'n'see..

Fred - AlphaSND