ListFiles - scan for files easily

Share your advanced PureBasic knowledge/code with the community.
User avatar
kenmo
Addict
Addict
Posts: 2033
Joined: Tue Dec 23, 2003 3:54 am

ListFiles - scan for files easily

Post by kenmo »

Are you tired of writing and re-writing the same old code for scanning directories, over and over, again and again?

Well worry no more! There's finally a better way!


... but seriously, I decided to finally write myself a tidy, reusable function for finding files within a given folder.

So I wrote ListFiles(), which takes a base folder, a string List (which the results are written to), an optional file extension filter, and six optional flags:
* Recursive mode (default off)
* Relative paths in results (default absolute paths)
* Include folders in results (default off)
* Include ONLY folders in results (default off)
* Include files with no type (since you can't list "no type" in the extension filter)
* Preserve previous contents of list (cleared by default)
* Exclude hidden files (included by default)
* Return ONLY hidden files (default off)

The code is decently commented, so you should be able to understand it.

I could have forced it all into one procedure, but I split it into two for a few reasons.

The second half of the code is a GUI demo I added so you can directly run it and see how it works.
(Not intended to be a useful program! Just a demo of the searches you could run from your own software.)

Suggestions are welcome. Enjoy.

PS. It SHOULD be cross-platform but is only tested on Windows x86.



Code: Select all

; +---------------+-------------------+
; | ListFiles.pbi |       kenmo       |
; +---------------+-------------------+
; |    4.01.2011  . Creation (PureBasic 4.51)
; |    7.24.2011  . Added NoHidden and HiddenOnly flags, changed AllowNoExt behavior
; 
; 
; Count.i   = ListFiles( Directory.s, List Results.s() [, Extensions.s [, Flags.i ] ] )
; 
; 
; 
; Directory:  Base folder to scan for files
;              - Trailing separator (\ or /) is recommended, but not needed
; 
; Results:    List of type string, which will be filled with found files
; 
; Extensions: Optional, lets you only scan for files of certain types
;              - Separate extensions by spaces (txt doc log) or 'requester style' (*.txt;*.doc;*.log)
;              - Wildcards (* and ?) will be ignored
; 
; Flags:      Optional, can be a combination of the following
;                #ListFiles_Recursive:   Scans all sub-directories along with the base folder
;                #ListFiles_Preserve:    Keeps existing elements in the Results list (they are cleared, otherwise)
;                #ListFiles_Relative:    Returns filenames relative to the base folder (absolute paths, otherwise)
;                #ListFiles_Folders:     Includes any found folders in the Results list
;                #ListFiles_FoldersOnly: Includes folders and excludes normal files from Results
;                #ListFiles_AllowNoExt:  Includes files that have no extension (enables extension filter)
;                #ListFiles_NoHidden:    Excludes files/folders marked hidden (included, otherwise) (Windows only!)
;                #ListFiles_HiddenOnly:  Includes only files/folders marked hidden (Windows only!)
; 
; 
; Count:      Returned value is the total number of files (and/or folders) found
;              - If the specified path is blank or cannot be examined, the result is -1
;              - The Results list may be larger than this, if you preserve prior elements
; 




;-
;- USER Constants

; Flags for ListFiles()
#ListFiles_Recursive   = $01 ; Recursively scans all sub-directories
#ListFiles_Preserve    = $02 ; Does not clear the specified list before scanning
#ListFiles_Relative    = $04 ; Returns results relative to the provided path
#ListFiles_Folders     = $08 ; Includes directories in scan results
#ListFiles_FoldersOnly = $10 ; Excludes files from scan results (forces IncludeFolders flag)
#ListFiles_AllowNoExt  = $20 ; Includes files with no extension (enables extension filter, even if empty)

; Hidden Flags, Windows OS only
CompilerIf (#PB_Compiler_OS = #PB_OS_Windows)
  #ListFiles_NoHidden    = $40 ; Excludes files/folders marked 'hidden' (normally included)
  #ListFiles_HiddenOnly  = $80 ; Includes only files/folders marked 'hidden'
CompilerElse
  #ListFiles_NoHidden    = $00 ; Not available
  #ListFiles_HiddenOnly  = $00 ; Not available
CompilerEndIf





;-
;- INTERNAL Constants

; Internal flags (not needed by user)
#xListFiles_FileFilter     = $8000

; OS-dependent path separator
CompilerIf (#PB_Compiler_OS = #PB_OS_Windows)
  #xListFiles_Separator = "\"
CompilerElse
  #xListFiles_Separator = "/"
CompilerEndIf





;-
;- INTERNAL Procedures

; Internal scanning procedure (not needed by user)
Procedure.i xListFiles(Root.s, Relative.s, List Results.s(), Extensions.s, Flags.i)
  Protected Count.i, ThisPath.s, Dir.i, AddThis.i
  Protected Name.s,  Type.i,     Ext.s, Recurse.i
  
  ThisPath = Root + Relative
  Dir = ExamineDirectory(#PB_Any, ThisPath, "")
  If (Dir)
    
    ; Clear previous results
    If (Not ((Flags & #ListFiles_Preserve) Or (Relative)))
      ClearList(Results())
    EndIf
    
    Count = 0
    While (NextDirectoryEntry(Dir))
      
      ; Check directory entry
      Name    =  DirectoryEntryName(Dir)
      Type    =  DirectoryEntryType(Dir)
      AddThis = #False
      Recurse = #False
      
      ; Process folders
      If (Type = #PB_DirectoryEntry_Directory)
      
        ; Check for real directories
        If ((Name <> ".") And (Name <> ".."))
          Name + #xListFiles_Separator
          
          ; Add to Results, if specified
          If (Flags & #ListFiles_Folders)
            AddThis = #True
          EndIf
          
          ; Set Recurse flag
          If (Flags & #ListFiles_Recursive)
            Recurse = #True
          EndIf
          
        EndIf
        
      ; Process files
      Else
      
        ; Exclude from Results, if specified
        If (Not (Flags & #ListFiles_FoldersOnly))
          AddThis = #True
          
          ; Check extension, if specified
          If (Flags & #xListFiles_FileFilter)
            Ext = LCase(GetExtensionPart(Name))
            
            If (Ext)
              ; Exclude unspecified file types
              If (Not FindString(Extensions, ";" + Ext + ";", 1))
                AddThis = #False
              EndIf
            Else
              ; Exclude files with no type
              If (Not (Flags & #ListFiles_AllowNoExt))
                AddThis = #False
              EndIf
            EndIf
            
          EndIf
          
        EndIf
        
      EndIf
      
      ; Check 'hidden' flag
      If (AddThis)
        If ((Flags & #ListFiles_NoHidden) And (DirectoryEntryAttributes(Dir) & #PB_FileSystem_Hidden))
          AddThis = #False
        ElseIf ((Flags & #ListFiles_HiddenOnly) And (Not (DirectoryEntryAttributes(Dir) & #PB_FileSystem_Hidden)))
          AddThis = #False
        EndIf
      EndIf
      
      ; Add entry to Results
      If (AddThis)
        AddElement(Results())
        If (Flags & #ListFiles_Relative)
          Results() = Relative + Name
        Else
          Results() = Root + Relative + Name
        EndIf
        Count + 1
      EndIf
      
      ; Scan sub-folder, add sub-count to current count
      If (Recurse)
        Recurse = xListFiles(Root, Relative + Name, Results(), Extensions, Flags)
        If (Recurse > 0)
          Count + Recurse
        EndIf
      EndIf
      
    Wend
    
    FinishDirectory(Dir)
  Else
    ; Invalid path
    Count = -1
  EndIf
  
  ProcedureReturn (Count)
EndProcedure





;-
;- USER Procedures

Procedure.i ListFiles(Directory.s, List Results.s(), Extensions.s = "", Flags.i = #Null)
  Protected Count.i
  
  ; Check for non-empty directory
  If (Directory)
    
    ; Append trailing path separator
    If (Right(Directory, 1) <> #xListFiles_Separator)
      Directory + #xListFiles_Separator
    EndIf
    
    ; Re-format extension list into 'ext1;ext2;ext3' format
    Extensions =       RemoveString(Extensions, ".")
    Extensions =       RemoveString(Extensions, "?")
    Extensions = LCase(RemoveString(Extensions, "*"))
    ReplaceString(Extensions, "|", ";", #PB_String_InPlace)
    ReplaceString(Extensions, " ", ";", #PB_String_InPlace)
    Extensions = Trim(Extensions, ";")
    
    ; Extension string determines FileFilter flag
    If (Extensions)
      Extensions = ";" + Extensions + ";"
      Flags | ( #xListFiles_FileFilter)
    Else
      Flags & (~#xListFiles_FileFilter)
    EndIf
    
    ; AllowNoExt flag forces FileFilter flag
    If (Flags & #ListFiles_AllowNoExt)
      Flags | ( #xListFiles_FileFilter)
    EndIf
    
    ; FoldersOnly flag forces Folders flag
    If (Flags & #ListFiles_FoldersOnly)
      Flags | ( #ListFiles_Folders)
    EndIf
    
    ; Call function to scan base directory
    Count = xListFiles(Directory, "", Results(), Extensions, Flags)
    
  Else
  
    ; Invalid operation
    Count = -1
    
  EndIf
  
  ProcedureReturn (Count)
EndProcedure

;-



















;-
;- EXAMPLE Program

;  Quick and dirty demo program: delete if you IncludeFile this from another source!

CompilerIf (1)

DisableExplicit

CompilerIf (#PB_Compiler_OS = #PB_OS_Windows)
  #Warning = #MB_ICONWARNING
  #Info    = #MB_ICONINFORMATION
CompilerElse
  #Warning = #Null
  #Info    = #Null
CompilerEndIf

Global NewList MyResults.s()

OpenWindow(0, 0, 0, 620, 360, "ListFiles() Demo", #PB_Window_ScreenCentered|#PB_Window_MinimizeGadget)

Frame3DGadget(0, 10, 10, 600, 150, "Test Parameters")
  TextGadget(1, 20, 33, 80, 20, "Directory:", #PB_Text_Center)
    StringGadget(2, 100, 30, 480, 20, "")
    ButtonGadget(3, 580, 30, 20, 20, "...")
  TextGadget(4, 20, 58, 80, 20, "Extensions:", #PB_Text_Center)
    StringGadget(5, 100, 55, 480, 20, "")
    ButtonGadget(6, 580, 55, 20, 20, "?")
  CheckBoxGadget(7, 20, 80, 140, 20, "Recursive search")
    CheckBoxGadget(8, 20, 100, 140, 20, "Relative filenames")
  CheckBoxGadget(9, 160, 80, 140, 20, "Include folders")
    CheckBoxGadget(10, 160, 100, 140, 20, "Return ONLY folders")
  CheckBoxGadget(11, 300, 80, 140, 20, "Allow files with no type")
    CheckBoxGadget(12, 300, 100, 140, 20, "Preserve existing results")
  CheckBoxGadget(17, 440, 80, 140, 20, "Exclude 'hidden' items")
    CheckBoxGadget(18, 440, 100, 140, 20, "Return ONLY 'hidden'")
  ButtonGadget(13, 220, 125, 80, 25, "Search", #PB_Button_Default)
    ButtonGadget(14, 320, 125, 80, 25, "Clear")

Frame3DGadget(15, 10, 170, 600, 180, "Results")
  ListIconGadget(16, 20, 190, 580, 150, "File", 400)
    AddGadgetColumn(16, 1, "Size", 140)

AddKeyboardShortcut(0, #PB_Shortcut_Return, 0)

SetActiveGadget(2)







Procedure Clear()
  ClearList(MyResults())
  ClearGadgetItems(16)
EndProcedure

Procedure.s Browse()
  Path.s = PathRequester("Choose Path", GetGadgetText(2))
  If (Path)
    SetGadgetText(2, Path)
  EndIf
  ProcedureReturn (Path)
EndProcedure

Procedure Search()
  Define Path.s, Exts.s, Size.i, St.s, Flags.i, Result.i
  
  ClearGadgetItems(16)
  
  Path    = GetGadgetText(2)
  Exts    = GetGadgetText(5)
  Flags   = GetGadgetState( 7) * #ListFiles_Recursive
    Flags + GetGadgetState( 8) * #ListFiles_Relative
    Flags + GetGadgetState( 9) * #ListFiles_Folders
    Flags + GetGadgetState(10) * #ListFiles_FoldersOnly
    Flags + GetGadgetState(11) * #ListFiles_AllowNoExt
    Flags + GetGadgetState(12) * #ListFiles_Preserve
    Flags + GetGadgetState(17) * #ListFiles_NoHidden
    Flags + GetGadgetState(18) * #ListFiles_HiddenOnly
  
  If (Path = "")
    Path = Browse()
  EndIf
  
  If (Path)
  
    If (Right(Path, 1) <> #xListFiles_Separator)
      Path + #xListFiles_Separator
    EndIf
  
    Result = ListFiles(Path, MyResults(), Exts, Flags)
    
    ForEach (MyResults())
      If (Flags & #ListFiles_Relative)
        Size = FileSize(Path + MyResults())
      Else
        Size = FileSize(MyResults())
      EndIf
      
      If (Size = -2)
        St = MyResults() + #LF$ + "      . dir"
      Else
        St = MyResults() + #LF$ + Str(Size)
      EndIf
      
      AddGadgetItem(16, -1, St)
    Next
    
    If (Result = -1)
      MessageRequester("Warning", "ListFiles returned -1 (empty or invalid path).", #Warning)
    Else
      MessageRequester("Info", "Found files: " + Str(Result), #Info)
    EndIf
  
  EndIf
EndProcedure




Repeat
  
  Event = WaitWindowEvent()
  If (Event = #PB_Event_CloseWindow)
    Quit = #True
  ElseIf (Event = #PB_Event_Gadget)
    Select (EventGadget())
      Case 3 : Browse()
      Case 6 : St.s = "Enter file types to include, separated by spaces:" + #LF$ + "      txt doc log"
               St + #LF$ + #LF$ + "or 'requester style':" + #LF$ + "      *.txt;*.doc;*.log"
               St + #LF$ + #LF$ + "Note: wildcards (* and ?) will be ignored."
               MessageRequester("Extensions", St, #Info)
        
      Case 13 : Search()
      Case 14 : Clear()
    EndSelect
  ElseIf (Event = #PB_Event_Menu)
    Search()
  EndIf
  
Until (Quit)

CompilerEndIf

;-
Last edited by kenmo on Fri Apr 17, 2015 4:45 pm, edited 5 times in total.
MachineCode
Addict
Addict
Posts: 1482
Joined: Tue Feb 22, 2011 1:16 pm

Re: ListFiles - scan for files easily

Post by MachineCode »

kenmo wrote:Are you tired of writing and re-writing the same old code for scanning directories, over and over, again and again?
Nope. I just IncludeFile my own 10-line procedure when it's needed. No writing or re-writing at all. But thanks for asking. :twisted:
Microsoft Visual Basic only lasted 7 short years: 1991 to 1998.
PureBasic: Born in 1998 and still going strong to this very day!
User avatar
djes
Addict
Addict
Posts: 1806
Joined: Sat Feb 19, 2005 2:46 pm
Location: Pas-de-Calais, France

Re: ListFiles - scan for files easily

Post by djes »

Yes, I'm tired, you're right!!! Thanks for sharing :)
kvitaliy
Enthusiast
Enthusiast
Posts: 162
Joined: Mon May 10, 2010 4:02 pm

Re: ListFiles - scan for files easily

Post by kvitaliy »

kenmo wrote: There's finally a better way!
Total Commander ->Menu->Commands->Search File
rsts
Addict
Addict
Posts: 2736
Joined: Wed Aug 24, 2005 8:39 am
Location: Southwest OH - USA

Re: ListFiles - scan for files easily

Post by rsts »

Pretty tough audience tonight :D

Nice, useful, documented.

Thanks for sharing.
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4789
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: ListFiles - scan for files easily

Post by Fangbeast »

kvitaliy wrote:
kenmo wrote: There's finally a better way!
Total Commander ->Menu->Commands->Search File
Total Commander = Commercial

Kenmo's solution = free
Amateur Radio/VK3HAF, (D-STAR/DMR and more), Arduino, ESP32, Coding, Crochet
User avatar
kenmo
Addict
Addict
Posts: 2033
Joined: Tue Dec 23, 2003 3:54 am

Re: ListFiles - scan for files easily

Post by kenmo »

kvitaliy wrote:
kenmo wrote: There's finally a better way!
Total Commander ->Menu->Commands->Search File
Sure, but Total Commander is a standalone program!

The purpose of this code is a user-friendly procedure to include in your own PB software!
User avatar
Kwai chang caine
Always Here
Always Here
Posts: 5494
Joined: Sun Nov 05, 2006 11:42 pm
Location: Lyon - France

Re: ListFiles - scan for files easily

Post by Kwai chang caine »

Yes ..thanks a lot to sharing 8)
ImageThe happiness is a road...
Not a destination
User avatar
luis
Addict
Addict
Posts: 3893
Joined: Wed Aug 31, 2005 11:09 pm
Location: Italy

Re: ListFiles - scan for files easily

Post by luis »

kvitaliy wrote: Total Commander ->Menu->Commands->Search File
This is a programmer's forum. Programmers writes programs.

This is a tip for programmers, he's not asking a final user to buy PureBasic and compile the source above to list some files.

This proves the roll-eyes icon had its rightful place ...
"Have you tried turning it off and on again ?"
A little PureBasic review
User avatar
Andre
PureBasic Team
PureBasic Team
Posts: 2137
Joined: Fri Apr 25, 2003 6:14 pm
Location: Germany (Saxony, Deutscheinsiedel)
Contact:

Re: ListFiles - scan for files easily

Post by Andre »

Well done. :)

It run's on PB for MacOS (x86) too, only the GUI should be a bit bigger there (because of the larger system font).
Bye,
...André
(PureBasicTeam::Docs & Support - PureArea.net | Order:: PureBasic | PureVisionXP)
IdeasVacuum
Always Here
Always Here
Posts: 6426
Joined: Fri Oct 23, 2009 2:33 am
Location: Wales, UK
Contact:

Re: ListFiles - scan for files easily

Post by IdeasVacuum »

Outstanding code, thank you very much for sharing kenmo. I like the demo too, icing on the cake. :D
IdeasVacuum
If it sounds simple, you have not grasped the complexity.
jamba
Enthusiast
Enthusiast
Posts: 144
Joined: Fri Jan 15, 2010 2:03 pm
Location: Triad, NC
Contact:

Re: ListFiles - scan for files easily

Post by jamba »

thanks for sharing. I wish I had seen this yesterday, I was pretty much writing code to do the same thing! this is very useful

Oh well. mine is working well, but I will gladly peruse yours and see if I can borrow from it to improve what I have :D
yours does has a few more options
-Jon

Fedora user
But I work with Win7
User avatar
utopiomania
Addict
Addict
Posts: 1655
Joined: Tue May 10, 2005 10:00 pm
Location: Norway

Re: ListFiles - scan for files easily

Post by utopiomania »

Hey you!

Are you tired of writing and re-writing the same old code for scanning directories, over and over, again and again?

Well worry no more! There's finally a better way!
No, I'm not. I posted this back in 2005, and use the same code today.
User avatar
kenmo
Addict
Addict
Posts: 2033
Joined: Tue Dec 23, 2003 3:54 am

Re: ListFiles - scan for files easily

Post by kenmo »

utopiomania wrote:No, I'm not. I posted this back in 2005, and use the same code today.
Nice, looks like a lot of similar features (I considered adding Hidden File exclusion after posting). You should update it with your PB 4.xx version!

Great minds think alike right?

(Who knows, I could have seen your code ages ago and been subconsciously inspired!)
User avatar
idle
Always Here
Always Here
Posts: 5836
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: ListFiles - scan for files easily

Post by idle »

Thanks for sharing the more the better.
Windows 11, Manjaro, Raspberry Pi OS
Image
Post Reply