Page 1 of 1
Help in modifying search procedure to run in a Thread
Posted: Sat Jan 21, 2023 5:18 am
by viiartz
Hi All, Just wanted to ask for help in modifying the below procedure "SearchDirectory()" to run as a thread.
I found the code in the forum (written by collectordave) and I have been using it to search for photo types specified in the RegEx criteria. The code does a good job in finding files except when searching large (2TB etc) hard drives the GUI freezes. The code still run all the way and control is given back to the GUI at the end. I figured if I ran the search in a thread that will not happen hence the help.
Code: Select all
Global NewList FilesFolders.s()
Global SearchFolder.s
Global SearchRegEX.i
#INITIAL_PATH = "c:\"
Procedure SearchDirectory(dir$, List dList.s(), level.l = 0)
Define EntryName.s
NewList FolderList.s()
Static FileCount.i
If (level = 0)
ClearList(dList())
EndIf
If Right(dir$, 1) <> #PS$
dir$ + #PS$
EndIf
If ExamineDirectory(0, dir$, "")
While NextDirectoryEntry(0)
EntryName = DirectoryEntryName(0)
Select DirectoryEntryType(0)
Case #PB_DirectoryEntry_Directory
If Left(EntryName,1) <> "."
AddElement(FolderList())
FolderList() = EntryName + #PS$
EndIf
Case #PB_DirectoryEntry_File
If Left(EntryName,1) <> "."
If MatchRegularExpression(SearchRegEx,EntryName)
AddElement(dList())
FileCount = FileCount + 1
dList() = dir$ + EntryName
EndIf
EndIf
EndSelect
Wend
EndIf
If ListSize(FolderList())
ForEach FolderList()
SearchDirectory(dir$ + FolderList(), dList(), level + 1)
Next
EndIf
EndProcedure
Define Word.s
;Search for all image files
filter.s ="jpg|rw2|cr2|nef|fpx|dcr|pcd|pef|raf|rwl|srw|arw|raw|kdc|iiq|fff|orf|eip|cib|kqp|dng|mrw|mdc|x3f|nop|exf|sr2|crw|ari|3fr|mos|lfr|r3d|mef|nrw|ndd|erf|ra2|dc2|ptx|srf|st7|sd0|sd1|st5|gray|grey|ce2|cmt|craw|st4|st6|gry|nwb|ce1|rwz|3pr|st8|mfw|kc2|olr|cap|stx|dcs|ycbcra|bay|heic|heif"
;Search for .png and .gif only
Word.s = "\.(?i)(" + filter + ")(?-i)$"
SearchRegEx = CreateRegularExpression(#PB_Any, Word)
;Define the topmost folder to start search
SearchFolder.s = PathRequester("Please choose a SOURCE path", #INITIAL_PATH)
SearchDirectory(SearchFolder, FilesFolders())
ForEach FilesFolders()
Debug FilesFolders()
Next
Re: Help in modifying search procedure to run in a Thread
Posted: Sat Jan 21, 2023 10:15 am
by infratec
Only short tested:
Code: Select all
CompilerIf Not #PB_Compiler_Thread
CompilerError "Enable threadsave in compiler options"
CompilerEndIf
EnableExplicit
Enumeration #PB_Event_FirstCustomValue
#OwnEvent_SearchDirectory_Start
#OwnEvent_SearchDirectory_FileCount
#OwnEvent_SearchDirectory_Finished
EndEnumeration
Structure SearchDirectoryParameter_Structure
Window.i
Thread.i
Exit.i
Level.i
SearchRegEx.i
SearchFolder$
List FileFolderList$()
FileCount.i
EndStructure
#INITIAL_PATH = "c:\"
Procedure SearchDirectory(*Parameter.SearchDirectoryParameter_Structure)
Protected EntryName$, Dir.i, StartSearchFolder$
NewList FolderList$()
If *Parameter\Level = 0
ClearList(*Parameter\FileFolderList$())
EndIf
If Right(*Parameter\SearchFolder$, 1) <> #PS$
*Parameter\SearchFolder$ + #PS$
EndIf
Dir = ExamineDirectory(#PB_Any, *Parameter\SearchFolder$, "")
If Dir
While NextDirectoryEntry(Dir) And Not *Parameter\Exit
EntryName$ = DirectoryEntryName(Dir)
Select DirectoryEntryType(Dir)
Case #PB_DirectoryEntry_Directory
If Left(EntryName$, 1) <> "."
AddElement(FolderList$())
FolderList$() = EntryName$ + #PS$
EndIf
Case #PB_DirectoryEntry_File
If Left(EntryName$, 1) <> "."
If MatchRegularExpression(*Parameter\SearchRegEx, EntryName$)
AddElement(*Parameter\FileFolderList$())
*Parameter\FileCount + 1
*Parameter\FileFolderList$() = *Parameter\SearchFolder$ + EntryName$
;PostEvent(#OwnEvent_SearchDirectory_FileCount, *Parameter\Window, 0, 0, *Parameter\FileCount)
EndIf
EndIf
EndSelect
Wend
FinishDirectory(Dir)
EndIf
ForEach FolderList$()
If Not *Parameter\Exit
StartSearchFolder$ = *Parameter\SearchFolder$
*Parameter\SearchFolder$ + FolderList$()
*Parameter\Level + 1
SearchDirectory(*Parameter)
*Parameter\SearchFolder$ = StartSearchFolder$
EndIf
Next
;PostEvent(#OwnEvent_SearchDirectory_Finished)
EndProcedure
Define Filter$, Word$
Define SearchDirectoryParameter.SearchDirectoryParameter_Structure
;Search for all image files
Filter$ ="jpg|rw2|cr2|nef|fpx|dcr|pcd|pef|raf|rwl|srw|arw|raw|kdc|iiq|fff|orf|eip|cib|kqp|dng|mrw|mdc|x3f|nop|exf|sr2|crw|ari|3fr|mos|lfr|r3d|mef|nrw|ndd|erf|ra2|dc2|ptx|srf|st7|sd0|sd1|st5|gray|grey|ce2|cmt|craw|st4|st6|gry|nwb|ce1|rwz|3pr|st8|mfw|kc2|olr|cap|stx|dcs|ycbcra|bay|heic|heif"
;Search for .png and .gif only
Word$ = "\.(?i)(" + Filter$ + ")(?-i)$"
SearchDirectoryParameter\SearchRegEx = CreateRegularExpression(#PB_Any, Word$)
;Define the topmost folder to start search
SearchDirectoryParameter\SearchFolder$ = PathRequester("Please choose a SOURCE path", #INITIAL_PATH)
If SearchDirectoryParameter\SearchFolder$
SearchDirectoryParameter\Thread = CreateThread(@SearchDirectory(), @SearchDirectoryParameter)
WaitThread(SearchDirectoryParameter\Thread)
ForEach SearchDirectoryParameter\FileFolderList$()
Debug SearchDirectoryParameter\FileFolderList$()
Next
EndIf
You can also modify it to use child threads instead of recursion.
But that's more tricky.
See also:
viewtopic.php?p=563042
viewtopic.php?p=539371
Re: Help in modifying search procedure to run in a Thread
Posted: Tue Jan 24, 2023 8:54 am
by Oso
infratec wrote: Sat Jan 21, 2023 10:15 am
Only short tested:
I just wanted to say, infratec, that your work converting to a threaded version from Saturday was appreciated by me at least. I particularly liked the idea of passing the thread's id. back into its own structure...
Code: Select all
\/
SearchDirectoryParameter\Thread = CreateThread(@SearchDirectory(), @SearchDirectoryParameter)
... I'm just about to use the technique myself, in fact
Thanks for this.
Re: Help in modifying search procedure to run in a Thread
Posted: Tue Jan 24, 2023 12:45 pm
by AZJIO
You don't have to use a regular expression. Use
SplitL to compare against each element of the list.