Page 1 of 1
Recursively packing folders
Posted: Thu May 08, 2025 4:14 pm
by Catdaddy
I'm coding up a simple little routine that will pack a given folder into a ZIP file. The ZIP file will have to faithfully recreate the entire folder when unzipped, including empty directories. The best I could do was to have a procedure recursively walk through the folder and record everything it finds to a linked list. Then, I simply go down the list and add each item to the pack file. Unzipping is easy enough as long as I create folder paths on the fly as I'm extracting files.
This works great and I'm completely happy with it. But, that first step of populating the list first with all of the entries, and THEN do the actual packing seems inefficient. If I could recursively walk through the folder AND pack the entries at the same time that would be awesome. But, I am having troubles with that for some reason. Because the procedure has to call itself throughout the operation when a new folder is found, I think I am mucking up the call. This is what I've got:
Code: Select all
EnableExplicit
UseZipPacker()
Procedure PackFolder(Folder.s, Pack.i)
If ExamineDirectory(0, Folder, "*.*")
While NextDirectoryEntry(0)
If DirectoryEntryType(0) = #PB_DirectoryEntry_File
AddPackFile(Pack, DirectoryEntryName(0), RemoveString(DirectoryEntryName(0), Folder))
Else
;AddPackDirectory(Pack, RemoveString(DirectoryEntryName(0), Folder))
AddPackDirectory(Pack, DirectoryEntryName(0))
PackFolder(DirectoryEntryName(0), Pack)
EndIf
Wend
FinishDirectory(0)
EndIf
EndProcedure
CompilerIf #PB_Compiler_IsMainFile
Define.i Pack
Pack = 0
CreatePack(Pack, "CampusData.zip", #PB_PackerPlugin_Zip, 5)
PackFolder("C:\CampusData\", Pack)
ClosePack(Pack)
CompilerEndIf
C:\CampusData is just a folder with a tree of subdirectories beneath it. I thought to pass a pre-created, opened pack file would be best so that I'm not overwriting the pack file each time I call the procedure. When I try to run this it pauses for about five seconds and I get a "Executable suddenly stopped for unknown reason".
Has anyone had better luck with this?
Re: Recursively packing folders
Posted: Thu May 08, 2025 4:49 pm
by Axolotl
well, just a brief comment to your recursive call
In my understanding you need a
new number to identify the new directory listing in every call. See my additions to your code below.
Code: Select all
EnableExplicit
UseZipPacker()
Procedure PackFolder(Folder.s, Pack.i, NDir=0) ; <= Axolotl: change for recursive calls
If ExamineDirectory(NDir, Folder, "*.*") ; <= Axolotl
While NextDirectoryEntry(NDir)
If DirectoryEntryType(NDir) = #PB_DirectoryEntry_File
AddPackFile(Pack, DirectoryEntryName(NDir), RemoveString(DirectoryEntryName(NDir), Folder))
Else
;AddPackDirectory(Pack, RemoveString(DirectoryEntryName(NDir), Folder))
AddPackDirectory(Pack, DirectoryEntryName(NDir))
PackFolder(DirectoryEntryName(NDir), Pack, NDir+1) ; <= Axolotl: recursive call with new/incremented Number
EndIf
Wend
FinishDirectory(NDir)
EndIf
EndProcedure
CompilerIf #PB_Compiler_IsMainFile
Define.i Pack
Pack = 0
CreatePack(Pack, "CampusData.zip", #PB_PackerPlugin_Zip, 5)
PackFolder("C:\CampusData\", Pack)
ClosePack(Pack)
CompilerEndIf
Re: Recursively packing folders
Posted: Fri May 09, 2025 9:46 am
by SMaag
here you can find how to list files and folders with recursive methode or with a List()
https://github.com/Maagic7/PureBasicFra ... eSystem.pb
Re: Recursively packing folders
Posted: Fri May 09, 2025 10:06 am
by Fred
Or use #PB_Any for such things, it's easier.
Re: Recursively packing folders
Posted: Fri May 09, 2025 10:19 am
by Quin
Here's my method for doing this, written for a basic BreifLZ packer:
Code: Select all
Procedure.i RecurseDirectory(Path$, List Files$())
Protected ID.i
ID = ExamineDirectory(#PB_Any, Path$, "*.*")
If Not ID : ProcedureReturn #False : EndIf
While NextDirectoryEntry(ID)
If DirectoryEntryName(ID) = ".." Or DirectoryEntryName(ID) = "." : Continue : EndIf
If DirectoryEntryType(ID) = #PB_DirectoryEntry_Directory : RecurseDirectory(Path$ + #PS$ + DirectoryEntryName(ID), Files$()) : EndIf
If DirectoryEntryType(ID) = #PB_DirectoryEntry_File
AddElement(Files$())
Files$() = Path$ + #PS$ + DirectoryEntryName(ID)
EndIf
Wend
FinishDirectory(ID)
ProcedureReturn #True
EndProcedure
Re: Recursively packing folders
Posted: Fri May 09, 2025 8:42 pm
by Catdaddy
I see in my anxiousness that I overlooked re-numbering the directory number when recursively calling the procedure. That makes sense now as it would fall into an endless loop of searching the same folder over and over again. And I always forget about #PB_Any, which is incredibly handy for enumerating things for me. And... I just noticed that I forgot to filter out "." and "..". Well, back to the old drawing board.
Thank you, everyone, for your patience and your help.
Re: Recursively packing folders
Posted: Sat May 10, 2025 1:52 pm
by AZJIO
Not recursive, but also adds folders. The notion of recursiveness is not meant by calling a procedure from the procedure itself, but by going through all the folders. You can also use folder depth and filter-mask for file extension. Here you first get a list of files and then add them to the archive.
Also see
here
Code: Select all
EnableExplicit
Procedure FileSearch(List Files.s(), dir.s, mask.s = "", depth = 130, lendir = 0)
Protected Name.s, c
Protected Dim SearchPath.s(depth)
If Right(dir, 1) <> #PS$
dir + #PS$
EndIf
SearchPath(c) = dir
If Not ExamineDirectory(c, dir, "")
ProcedureReturn
EndIf
Repeat
While NextDirectoryEntry(c)
Name = DirectoryEntryName(c)
If Name = "." Or Name = ".."
Continue
EndIf
If DirectoryEntryType(c) = #PB_DirectoryEntry_Directory
If c >= depth
Continue
EndIf
dir = SearchPath(c)
c + 1
SearchPath(c) = dir + Name + #PS$
If Not ExamineDirectory(c, SearchPath(c), "")
c - 1
EndIf
Else
If (Not Asc(mask) Or (Asc(mask) And GetExtensionPart(Name) = mask)) And AddElement(Files())
If lendir
Files() = Mid(SearchPath(c) + Name, lendir)
Else
Files() = SearchPath(c) + Name
EndIf
EndIf
EndIf
Wend
FinishDirectory(c)
c - 1
Until c < 0
EndProcedure
UseLZMAPacker()
Define NewList Files.s()
Define Path$ = "C:\PB\Source\Current\"
Define length = Len(Path$) + 1
FileSearch(Files(), Path$)
; FileSearch(Files(), Path$, "png", 130, Len(Path$) + 1)
; Debug "Count = " + Str(ListSize(Files()))
If CreatePack(0, "config-archive.7z", #PB_PackerPlugin_Lzma)
ForEach Files()
; Debug Files()
AddPackFile(0, Files(), Mid(Files(), length))
Next
ClosePack(0)
EndIf
Re: Recursively packing folders
Posted: Sat May 10, 2025 2:22 pm
by Catdaddy
AZJIO wrote: Sat May 10, 2025 1:52 pm
Not recursive, but also adds folders. The notion of recursiveness is not meant by calling a procedure from the procedure itself, but by going through all the folders. You can also use folder depth and filter-mask for file extension. Here you first get a list of files and then add them to the archive.
Also see
here
Wow, thank you, AZJIO. That works brilliantly! It took me a bit to wrap my head around your ExamineDirectory() use but that seems to be a much cleaner approach. I also like the fact that the LZMA packer works seamlessly with 7Zip. The ZIP packer is kind of confusing in the way it stores files and folder paths and does not unzip properly with 7Zip or the Windows zip folder methods. Thank you for your help.
Re: Recursively packing folders
Posted: Sat May 10, 2025 5:50 pm
by AZJIO
You can get the list of files in different ways.
FileDirListCmd
Recursive
RegExp
I shouldn't have redone the current function without #PB_Any. Even though it works fine, it still has a problem if ExamineDirectory() has already been executed, so use
my previous use case for this function.