[Solved] Get all files on disk without recursion?
[Solved] Get all files on disk without recursion?
Hi all, I remember seeing a procedure for getting a list of files on a disk without recursion, but can't find it now. You could call the procedure and it never called itself, but was able to get all subfolder files too. I didn't save it for some reason, but now I need it. Anyone remember how it was done? Thanks.
[Edit] I need to get each filename on-the-fly to do something with them; not just get a big list at the end.
[Edit 2] Found the post I was looking for - and it was by Freak! -> viewtopic.php?p=297865#p297865
[Edit] I need to get each filename on-the-fly to do something with them; not just get a big list at the end.
[Edit 2] Found the post I was looking for - and it was by Freak! -> viewtopic.php?p=297865#p297865
Last edited by BarryG on Thu Feb 25, 2021 5:11 pm, edited 3 times in total.
Re: Get all files on disk without recursion?
Hi BarryG
Code: Select all
dir c: /w /s >> c:\users\barry??\list.txt
Egypt my love
Re: Get all files on disk without recursion?
Yes, and to have full filename
Code: Select all
dir /s /b
Re: Get all files on disk without recursion?
Sorry, see my first post edit.
Re: Get all files on disk without recursion?
I don't understand...BarryG wrote:Sorry, see my first post edit.
So take Rashad's list and iterate over it, doing to each/some file(s) whatever you need to do.
PB 5.73 on Windows 10 & OS X High Sierra
Re: Get all files on disk without recursion?
Waiting until the list populates (from "dir") takes too long (I have almost half a million files on a 2 TB slow USB drive), and results in a 15 MB text file to parse. I need to start displaying the files immediately, in real-time. I can't make the user wait 20 seconds for their file list.
Plus, the user needs to be able to cancel the list build at any time, which "dir" won't let me do while that long 20-second wait is happening.
Plus, I need to filter by size and/or date, so it means touching all the files anyway. Better just to do one big non-recursive loop and filter them on-the-fly.
Plus, the user needs to be able to cancel the list build at any time, which "dir" won't let me do while that long 20-second wait is happening.
Plus, I need to filter by size and/or date, so it means touching all the files anyway. Better just to do one big non-recursive loop and filter them on-the-fly.
Last edited by BarryG on Thu Feb 25, 2021 4:49 pm, edited 2 times in total.
- NicTheQuick
- Addict
- Posts: 1224
- Joined: Sun Jun 22, 2003 7:43 pm
- Location: Germany, Saarbrücken
- Contact:
Re: Get all files on disk without recursion?
In principle you can just use a recursing version and then handling the stack by yourself. Or you also can define a callback which is called for every file. I will try something. Gimme a few minutes.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Re: Get all files on disk without recursion?
Use two lists. Put the base path in the first list. Start a loop that processes all elements of the list, not a for-loop (use a while-loop, since you don't know how many entries it will finally contain). Whole processing said elements write the files into the second list, subdirs into the first. When done, process the next subdir in the first list, if available.
Code: Select all
EnableExplicit
Define counter,examine,NewList List1.s(),NewList List2.s(),dirtemp.s
AddElement(List1())
List1()="D:\PureBasic"
While ListSize(List1())>0
SelectElement(List1(),0)
dirtemp=List1()
examine=ExamineDirectory(#PB_Any,dirtemp,"*.*")
If examine
While NextDirectoryEntry(examine)
Select DirectoryEntryType(examine)
Case #PB_DirectoryEntry_Directory
If Len(ReplaceString(DirectoryEntryName(examine),".",""))>0
AddElement(List1())
List1()=dirtemp+"\"+DirectoryEntryName(examine)
EndIf
Case #PB_DirectoryEntry_File
AddElement(List2())
List2()=dirtemp+"\"+DirectoryEntryName(examine)
Debug List2()
EndSelect
Wend
FinishDirectory(examine)
EndIf
SelectElement(List1(),0)
DeleteElement(List1())
Wend
Last edited by jacdelad on Thu Feb 25, 2021 5:13 pm, edited 1 time in total.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Re: Get all files on disk without recursion?
Hi,
if I understand your question right, than this could be of help:
if I understand your question right, than this could be of help:
Code: Select all
Procedure MakeDirectoryListwithoutRecursion(Directory$) ;' returns number of found items
Protected NewList folders$() ;' temp directories
Protected dir$, name$, count, QuitLoop, DirID = 0
If Directory$ = "" ;' missing start directory
Directory$ = PathRequester("Please select the drive and directory to catalogue", "")
If Directory$ = ""
ProcedureReturn 0 ;' nothing to examine
EndIf
EndIf
; ClearGadgetItemList(#GADGET_Items)
Debug "Start with " + Directory$
ClearList(folders$())
AddElement(folders$()) ;' init first element
folders$() = RTrim(Directory$, "\") ;' remove trailing backslash
count = 0
Repeat
If ListSize(folders$()) = 0 : Break : EndIf
FirstElement(folders$())
dir$ = folders$()
DeleteElement(folders$(), 1) ;' current is the next element
If ExamineDirectory(DirID, dir$, "*.*")
While NextDirectoryEntry(DirID)
If DirectoryEntryType(DirID) = #PB_DirectoryEntry_File
name$ = DirectoryEntryName(DirID)
Debug "List[" + RSet(Str(count), 4) + "] = '" + dir$ + "\" + name$ + "'"
; AddGadgetItem(#GADGET_Items, - 1, dir$ + name$)
; FlushEvents()
; SetStatus(1, "File(s) " + Str(count))
; SendMessage_(GadgetID(#GADGET_Items), #LVM_ENSUREVISIBLE, count, 0)
count + 1
Else ; = #PB_DirectoryEntry_Directory
name$ = DirectoryEntryName(DirID)
If name$ <> ".." And name$ <> "." ;' i dont want the special windows directories
AddElement(folders$()) ;' keep directory for further processing
folders$() = dir$ + "\" + name$ ;:Debug "add to folders() = '" + folders$() + "' " + ListSize(folders$())
EndIf
EndIf
Wend
EndIf
Until ListSize(folders$()) = 0
ProcedureReturn count
EndProcedure ;()
;' call some sample directories for test -- I made only some short ones :-)
;Debug MakeDirectoryListwithoutRecursion("C:\temp\")
;Debug MakeDirectoryListwithoutRecursion(<StartDirectory>)
Mostly running PureBasic <latest stable version and current alpha/beta> (x64) on Windows 11 Home
- NicTheQuick
- Addict
- Posts: 1224
- Joined: Sun Jun 22, 2003 7:43 pm
- Location: Germany, Saarbrücken
- Contact:
Re: Get all files on disk without recursion?
This should do the trick:
I do not remember what the correct constant is for the operating system dependent path separator, so I named it #PATH_SEP.
Code: Select all
EnableExplicit
#PATH_SEP = "/"
Structure iterStack
hDir.i
path.s
EndStructure
Procedure iterateFiles(path.s)
Protected NewList stack.iterStack()
Protected fullpath.s, name.s, nextLevel.i = #False
AddElement(stack())
stack()\path = path
stack()\hDir = 0
While FirstElement(stack())
With stack()
If \hDir = 0
\hDir = ExamineDirectory(#PB_Any, \path, "*")
EndIf
If \hDir
nextLevel = #False
While NextDirectoryEntry(\hDir)
name = DirectoryEntryName(\hDir)
If name = ".." Or name = "."
Continue
EndIf
fullpath = \path + #PATH_SEP + name
Debug fullpath ; Here you will get you file!
If DirectoryEntryType(\hDir) = #PB_DirectoryEntry_Directory
AddElement(stack())
stack()\path = fullpath
stack()\hDir = 0
nextLevel = #True
Break
EndIf
Wend
If nextLevel
Continue
EndIf
FinishDirectory(\hDir)
EndIf
DeleteElement(stack())
EndWith
Wend
EndProcedure
iterateFiles("/home/nicolas/tmp")
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Re: Get all files on disk without recursion?
Thanks everyone - your codes do the job indeed, and I've edited my first post with the code I was looking for (I found it!).
- NicTheQuick
- Addict
- Posts: 1224
- Joined: Sun Jun 22, 2003 7:43 pm
- Location: Germany, Saarbrücken
- Contact:
Re: [Solved] Get all files on disk without recursion?
I've implemented a Depth-first search if this is what you are looking for. That's the reason why I also need to push the handle of ExamineDirectory to the stack. This way it is comparable to a naive recursion search.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Re: [Solved] Get all files on disk without recursion?
Thanks, NicTheQuick. Also, the constant for the path separator is #PS$.
Re: [Solved] Get all files on disk without recursion?
NicTheQuick
A week ago I posted the same code, but I have no stack deletion, but an array is used. You can embed the nesting depth, for example, request a depth of 2 or 3 levels. In the archive, I made the formation of a list including, for example, size and date. If you enable sorting, it works slowly, while I think how to optimize.
Your function is called iterateFiles, but it contains files and folders. I used mine and your function, I ended up with different lists. I sorted and compared in Meld, the difference is in adding folders.
I made a change to my code and began to receive only files in the specified folder. Now you can use the number as the search depth.
A week ago I posted the same code, but I have no stack deletion, but an array is used. You can embed the nesting depth, for example, request a depth of 2 or 3 levels. In the archive, I made the formation of a list including, for example, size and date. If you enable sorting, it works slowly, while I think how to optimize.
Your function is called iterateFiles, but it contains files and folders. I used mine and your function, I ended up with different lists. I sorted and compared in Meld, the difference is in adding folders.
I made a change to my code and began to receive only files in the specified folder. Now you can use the number as the search depth.
Code: Select all
; AZJIO
; https://www.purebasic.fr/english/viewtopic.php?p=566355#p566355
EnableExplicit
Procedure FileSearch(*Result.string, sPath.s, Mask$ = "*", depth=130)
Protected Len, *Point
Protected NewList Files.s()
Protected sName.s, c = 0
Protected Dim aExaDir(depth)
Protected Dim aSePath.s(depth)
If Right(sPath, 1) <> #PS$
sPath + #PS$
EndIf
aSePath(c) = sPath
aExaDir(c) = ExamineDirectory(#PB_Any, sPath, Mask$)
If Not aExaDir(c)
ProcedureReturn
EndIf
Repeat
While NextDirectoryEntry(aExaDir(c))
sName=DirectoryEntryName(aExaDir(c))
If sName = "." Or sName = ".."
Continue
EndIf
If DirectoryEntryType(aExaDir(c)) = #PB_DirectoryEntry_Directory
If c >= depth
Continue
EndIf
sPath = aSePath(c)
c + 1
aSePath(c) = sPath + sName + #PS$
aExaDir(c) = ExamineDirectory(#PB_Any, aSePath(c), Mask$)
If Not aExaDir(c)
c - 1
EndIf
Else
If AddElement(Files())
Files() = aSePath(c) + sName
EndIf
EndIf
Wend
FinishDirectory(aExaDir(c))
c - 1
Until c < 0
Debug "Depth = " + Str(depth)
Debug "Count = " + Str(ListSize(Files()))
Len=0
ForEach Files()
Len + Len(Files())+2
Next
*Result\s = Space(Len)
*Point = @*Result\s
ForEach Files()
CopyMemoryString(Files()+#CRLF$, @*Point)
Next
ClearList(Files())
EndProcedure
Define StartTime=ElapsedMilliseconds()
Define Path$ = "/home/user"
Define Result.string
FileSearch(@Result, Path$, "*", 0) ; Wildcard, *.exe
Define Time.s = "Time = " + Str(ElapsedMilliseconds()-StartTime) + " ms"
Debug Time
Debug "_______________"
Debug Result\s