Page 1 of 1
[Solved] Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 2:57 pm
by BarryG
Hi all, I remember seeing a procedure for getting a list of files on a disk without recursion, but can't find it now. You could call the procedure and it never called itself, but was able to get all subfolder files too. I didn't save it for some reason, but now I need it. Anyone remember how it was done? Thanks.
[Edit] I need to get each filename on-the-fly to do something with them; not just get a big list at the end.
[Edit 2] Found the post I was looking for - and it was by Freak! ->
viewtopic.php?p=297865#p297865
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 3:20 pm
by RASHAD
Hi BarryG
Code: Select all
dir c: /w /s >> c:\users\barry??\list.txt
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 3:24 pm
by Marc56us
Yes, and to have full filename
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 3:24 pm
by BarryG
Sorry, see my first post edit.
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 4:26 pm
by kpeters58
BarryG wrote:Sorry, see my first post edit.
I don't understand...
So take Rashad's list and iterate over it, doing to each/some file(s) whatever you need to do.
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 4:39 pm
by BarryG
Waiting until the list populates (from "dir") takes too long (I have almost half a million files on a 2 TB slow USB drive), and results in a 15 MB text file to parse. I need to start displaying the files immediately, in real-time. I can't make the user wait 20 seconds for their file list.
Plus, the user needs to be able to cancel the list build at any time, which "dir" won't let me do while that long 20-second wait is happening.
Plus, I need to filter by size and/or date, so it means touching all the files anyway. Better just to do one big non-recursive loop and filter them on-the-fly.
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 4:47 pm
by NicTheQuick
In principle you can just use a recursing version and then handling the stack by yourself. Or you also can define a callback which is called for every file. I will try something. Gimme a few minutes.
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 4:57 pm
by jacdelad
Use two lists. Put the base path in the first list. Start a loop that processes all elements of the list, not a for-loop (use a while-loop, since you don't know how many entries it will finally contain). Whole processing said elements write the files into the second list, subdirs into the first. When done, process the next subdir in the first list, if available.
Code: Select all
EnableExplicit
Define counter,examine,NewList List1.s(),NewList List2.s(),dirtemp.s
AddElement(List1())
List1()="D:\PureBasic"
While ListSize(List1())>0
SelectElement(List1(),0)
dirtemp=List1()
examine=ExamineDirectory(#PB_Any,dirtemp,"*.*")
If examine
While NextDirectoryEntry(examine)
Select DirectoryEntryType(examine)
Case #PB_DirectoryEntry_Directory
If Len(ReplaceString(DirectoryEntryName(examine),".",""))>0
AddElement(List1())
List1()=dirtemp+"\"+DirectoryEntryName(examine)
EndIf
Case #PB_DirectoryEntry_File
AddElement(List2())
List2()=dirtemp+"\"+DirectoryEntryName(examine)
Debug List2()
EndSelect
Wend
FinishDirectory(examine)
EndIf
SelectElement(List1(),0)
DeleteElement(List1())
Wend
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 5:02 pm
by Axolotl
Hi,
if I understand your question right, than this could be of help:
Code: Select all
Procedure MakeDirectoryListwithoutRecursion(Directory$) ;' returns number of found items
Protected NewList folders$() ;' temp directories
Protected dir$, name$, count, QuitLoop, DirID = 0
If Directory$ = "" ;' missing start directory
Directory$ = PathRequester("Please select the drive and directory to catalogue", "")
If Directory$ = ""
ProcedureReturn 0 ;' nothing to examine
EndIf
EndIf
; ClearGadgetItemList(#GADGET_Items)
Debug "Start with " + Directory$
ClearList(folders$())
AddElement(folders$()) ;' init first element
folders$() = RTrim(Directory$, "\") ;' remove trailing backslash
count = 0
Repeat
If ListSize(folders$()) = 0 : Break : EndIf
FirstElement(folders$())
dir$ = folders$()
DeleteElement(folders$(), 1) ;' current is the next element
If ExamineDirectory(DirID, dir$, "*.*")
While NextDirectoryEntry(DirID)
If DirectoryEntryType(DirID) = #PB_DirectoryEntry_File
name$ = DirectoryEntryName(DirID)
Debug "List[" + RSet(Str(count), 4) + "] = '" + dir$ + "\" + name$ + "'"
; AddGadgetItem(#GADGET_Items, - 1, dir$ + name$)
; FlushEvents()
; SetStatus(1, "File(s) " + Str(count))
; SendMessage_(GadgetID(#GADGET_Items), #LVM_ENSUREVISIBLE, count, 0)
count + 1
Else ; = #PB_DirectoryEntry_Directory
name$ = DirectoryEntryName(DirID)
If name$ <> ".." And name$ <> "." ;' i dont want the special windows directories
AddElement(folders$()) ;' keep directory for further processing
folders$() = dir$ + "\" + name$ ;:Debug "add to folders() = '" + folders$() + "' " + ListSize(folders$())
EndIf
EndIf
Wend
EndIf
Until ListSize(folders$()) = 0
ProcedureReturn count
EndProcedure ;()
;' call some sample directories for test -- I made only some short ones :-)
;Debug MakeDirectoryListwithoutRecursion("C:\temp\")
;Debug MakeDirectoryListwithoutRecursion(<StartDirectory>)
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 5:03 pm
by NicTheQuick
This should do the trick:
Code: Select all
EnableExplicit
#PATH_SEP = "/"
Structure iterStack
hDir.i
path.s
EndStructure
Procedure iterateFiles(path.s)
Protected NewList stack.iterStack()
Protected fullpath.s, name.s, nextLevel.i = #False
AddElement(stack())
stack()\path = path
stack()\hDir = 0
While FirstElement(stack())
With stack()
If \hDir = 0
\hDir = ExamineDirectory(#PB_Any, \path, "*")
EndIf
If \hDir
nextLevel = #False
While NextDirectoryEntry(\hDir)
name = DirectoryEntryName(\hDir)
If name = ".." Or name = "."
Continue
EndIf
fullpath = \path + #PATH_SEP + name
Debug fullpath ; Here you will get you file!
If DirectoryEntryType(\hDir) = #PB_DirectoryEntry_Directory
AddElement(stack())
stack()\path = fullpath
stack()\hDir = 0
nextLevel = #True
Break
EndIf
Wend
If nextLevel
Continue
EndIf
FinishDirectory(\hDir)
EndIf
DeleteElement(stack())
EndWith
Wend
EndProcedure
iterateFiles("/home/nicolas/tmp")
I do not remember what the correct constant is for the operating system dependent path separator, so I named it #PATH_SEP.
Re: Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 5:10 pm
by BarryG
Thanks everyone - your codes do the job indeed, and I've edited my first post with the code I was looking for (I found it!).
Re: [Solved] Get all files on disk without recursion?
Posted: Thu Feb 25, 2021 5:23 pm
by NicTheQuick
I've implemented a Depth-first search if this is what you are looking for. That's the reason why I also need to push the handle of ExamineDirectory to the stack. This way it is comparable to a naive recursion search.
Re: [Solved] Get all files on disk without recursion?
Posted: Fri Feb 26, 2021 4:07 am
by BarryG
Thanks, NicTheQuick. Also, the constant for the path separator is #PS$.
Re: [Solved] Get all files on disk without recursion?
Posted: Fri Feb 26, 2021 4:49 am
by AZJIO
NicTheQuick
A week ago I posted the same
code, but I have no stack deletion, but an array is used. You can embed the nesting depth, for example, request a depth of 2 or 3 levels. In the archive, I made the formation of a list including, for example, size and date. If you enable sorting, it works slowly, while I think how to optimize.
Your function is called iterateFiles, but it contains files and folders. I used mine and your function, I ended up with different lists. I sorted and compared in Meld, the difference is in adding folders.
I made a change to my code and began to receive only files in the specified folder. Now you can use the number as the search depth.
Code: Select all
; AZJIO
; https://www.purebasic.fr/english/viewtopic.php?p=566355#p566355
EnableExplicit
Procedure FileSearch(*Result.string, sPath.s, Mask$ = "*", depth=130)
Protected Len, *Point
Protected NewList Files.s()
Protected sName.s, c = 0
Protected Dim aExaDir(depth)
Protected Dim aSePath.s(depth)
If Right(sPath, 1) <> #PS$
sPath + #PS$
EndIf
aSePath(c) = sPath
aExaDir(c) = ExamineDirectory(#PB_Any, sPath, Mask$)
If Not aExaDir(c)
ProcedureReturn
EndIf
Repeat
While NextDirectoryEntry(aExaDir(c))
sName=DirectoryEntryName(aExaDir(c))
If sName = "." Or sName = ".."
Continue
EndIf
If DirectoryEntryType(aExaDir(c)) = #PB_DirectoryEntry_Directory
If c >= depth
Continue
EndIf
sPath = aSePath(c)
c + 1
aSePath(c) = sPath + sName + #PS$
aExaDir(c) = ExamineDirectory(#PB_Any, aSePath(c), Mask$)
If Not aExaDir(c)
c - 1
EndIf
Else
If AddElement(Files())
Files() = aSePath(c) + sName
EndIf
EndIf
Wend
FinishDirectory(aExaDir(c))
c - 1
Until c < 0
Debug "Depth = " + Str(depth)
Debug "Count = " + Str(ListSize(Files()))
Len=0
ForEach Files()
Len + Len(Files())+2
Next
*Result\s = Space(Len)
*Point = @*Result\s
ForEach Files()
CopyMemoryString(Files()+#CRLF$, @*Point)
Next
ClearList(Files())
EndProcedure
Define StartTime=ElapsedMilliseconds()
Define Path$ = "/home/user"
Define Result.string
FileSearch(@Result, Path$, "*", 0) ; Wildcard, *.exe
Define Time.s = "Time = " + Str(ElapsedMilliseconds()-StartTime) + " ms"
Debug Time
Debug "_______________"
Debug Result\s