Recursively packing folders

[Catdaddy]

I'm coding up a simple little routine that will pack a given folder into a ZIP file. The ZIP file will have to faithfully recreate the entire folder when unzipped, including empty directories. The best I could do was to have a procedure recursively walk through the folder and record everything it finds to a linked list. Then, I simply go down the list and add each item to the pack file. Unzipping is easy enough as long as I create folder paths on the fly as I'm extracting files.

This works great and I'm completely happy with it. But, that first step of populating the list first with all of the entries, and THEN do the actual packing seems inefficient. If I could recursively walk through the folder AND pack the entries at the same time that would be awesome. But, I am having troubles with that for some reason. Because the procedure has to call itself throughout the operation when a new folder is found, I think I am mucking up the call. This is what I've got:

Code: Select all

EnableExplicit
UseZipPacker()

Procedure PackFolder(Folder.s, Pack.i)
	If ExamineDirectory(0, Folder, "*.*")
		While NextDirectoryEntry(0)
			If DirectoryEntryType(0) = #PB_DirectoryEntry_File
				AddPackFile(Pack, DirectoryEntryName(0), RemoveString(DirectoryEntryName(0), Folder))
			Else
				;AddPackDirectory(Pack, RemoveString(DirectoryEntryName(0), Folder))
				AddPackDirectory(Pack, DirectoryEntryName(0))
				PackFolder(DirectoryEntryName(0), Pack)
			EndIf
		Wend
		FinishDirectory(0)
	EndIf
EndProcedure



CompilerIf #PB_Compiler_IsMainFile
	Define.i Pack
	Pack = 0
	CreatePack(Pack, "CampusData.zip", #PB_PackerPlugin_Zip, 5)
	PackFolder("C:\CampusData\", Pack)
	ClosePack(Pack)
CompilerEndIf

C:\CampusData is just a folder with a tree of subdirectories beneath it. I thought to pass a pre-created, opened pack file would be best so that I'm not overwriting the pack file each time I call the procedure. When I try to run this it pauses for about five seconds and I get a "Executable suddenly stopped for unknown reason".

Has anyone had better luck with this?

[Axolotl]

well, just a brief comment to your recursive call
In my understanding you need a new number to identify the new directory listing in every call. See my additions to your code below.

Code: Select all

EnableExplicit
UseZipPacker()

Procedure PackFolder(Folder.s, Pack.i, NDir=0)  ; <= Axolotl: change for recursive calls 
	If ExamineDirectory(NDir, Folder, "*.*")      ; <= Axolotl 
		While NextDirectoryEntry(NDir)
			If DirectoryEntryType(NDir) = #PB_DirectoryEntry_File
				AddPackFile(Pack, DirectoryEntryName(NDir), RemoveString(DirectoryEntryName(NDir), Folder)) 
			Else
				;AddPackDirectory(Pack, RemoveString(DirectoryEntryName(NDir), Folder))
				AddPackDirectory(Pack, DirectoryEntryName(NDir))
				PackFolder(DirectoryEntryName(NDir), Pack, NDir+1)  ; <= Axolotl: recursive call with new/incremented Number 
			EndIf
		Wend
		FinishDirectory(NDir)
	EndIf
EndProcedure

CompilerIf #PB_Compiler_IsMainFile
	Define.i Pack
	Pack = 0
	CreatePack(Pack, "CampusData.zip", #PB_PackerPlugin_Zip, 5)
	PackFolder("C:\CampusData\", Pack)
	ClosePack(Pack)
CompilerEndIf

[SMaag]

here you can find how to list files and folders with recursive methode or with a List()

https://github.com/Maagic7/PureBasicFra ... eSystem.pb

[Fred]

Or use #PB_Any for such things, it's easier.

[Quin]

Here's my method for doing this, written for a basic BreifLZ packer:

Code: Select all

Procedure.i RecurseDirectory(Path$, List Files$())
	Protected ID.i
	ID = ExamineDirectory(#PB_Any, Path$, "*.*")
	If Not ID : ProcedureReturn #False : EndIf
	While NextDirectoryEntry(ID)
		If DirectoryEntryName(ID) = ".." Or DirectoryEntryName(ID) = "." : Continue : EndIf
		If DirectoryEntryType(ID) = #PB_DirectoryEntry_Directory : RecurseDirectory(Path$ + #PS$ + DirectoryEntryName(ID), Files$()) : EndIf
		If DirectoryEntryType(ID) = #PB_DirectoryEntry_File
			AddElement(Files$())
			Files$() = Path$ + #PS$ + DirectoryEntryName(ID)
		EndIf
	Wend
	FinishDirectory(ID)
	ProcedureReturn #True
EndProcedure

[Catdaddy]

I see in my anxiousness that I overlooked re-numbering the directory number when recursively calling the procedure. That makes sense now as it would fall into an endless loop of searching the same folder over and over again. And I always forget about #PB_Any, which is incredibly handy for enumerating things for me. And... I just noticed that I forgot to filter out "." and "..". Well, back to the old drawing board.

Thank you, everyone, for your patience and your help.

[AZJIO]

Not recursive, but also adds folders. The notion of recursiveness is not meant by calling a procedure from the procedure itself, but by going through all the folders. You can also use folder depth and filter-mask for file extension. Here you first get a list of files and then add them to the archive.
Also see here

Code: Select all

EnableExplicit

Procedure FileSearch(List Files.s(), dir.s, mask.s = "", depth = 130, lendir = 0)
	Protected Name.s, c
	Protected Dim SearchPath.s(depth)

	If Right(dir, 1) <> #PS$
		dir + #PS$
	EndIf

	SearchPath(c) = dir
	If Not ExamineDirectory(c, dir, "")
		ProcedureReturn
	EndIf

	Repeat
		While NextDirectoryEntry(c)
			Name = DirectoryEntryName(c)
			If Name = "." Or Name = ".."
				Continue
			EndIf
			If DirectoryEntryType(c) = #PB_DirectoryEntry_Directory
				If c >= depth
					Continue
				EndIf
				dir = SearchPath(c)
				c + 1
				SearchPath(c) = dir + Name + #PS$
				If Not ExamineDirectory(c, SearchPath(c), "")
					c - 1
				EndIf
			Else
				If (Not Asc(mask) Or (Asc(mask) And GetExtensionPart(Name) = mask)) And AddElement(Files())
					If lendir
						Files() = Mid(SearchPath(c) + Name, lendir)
					Else
						Files() = SearchPath(c) + Name
					EndIf
				EndIf
			EndIf
		Wend
		FinishDirectory(c)
		c - 1
	Until c < 0
EndProcedure

UseLZMAPacker()

Define NewList Files.s()
Define Path$ = "C:\PB\Source\Current\"
Define length = Len(Path$) + 1
FileSearch(Files(), Path$)
; FileSearch(Files(), Path$, "png", 130, Len(Path$) + 1)
; Debug "Count = " + Str(ListSize(Files()))

If CreatePack(0, "config-archive.7z", #PB_PackerPlugin_Lzma)
	ForEach Files()
; 		Debug Files()
    	AddPackFile(0, Files(), Mid(Files(), length))
	Next
     ClosePack(0)
EndIf

[Catdaddy]

Not recursive, but also adds folders. The notion of recursiveness is not meant by calling a procedure from the procedure itself, but by going through all the folders. You can also use folder depth and filter-mask for file extension. Here you first get a list of files and then add them to the archive.
Also see here

Wow, thank you, AZJIO. That works brilliantly! It took me a bit to wrap my head around your ExamineDirectory() use but that seems to be a much cleaner approach. I also like the fact that the LZMA packer works seamlessly with 7Zip. The ZIP packer is kind of confusing in the way it stores files and folder paths and does not unzip properly with 7Zip or the Windows zip folder methods. Thank you for your help.

[AZJIO]

You can get the list of files in different ways.
FileDirListCmd
Recursive
RegExp
I shouldn't have redone the current function without #PB_Any. Even though it works fine, it still has a problem if ExamineDirectory() has already been executed, so use my previous use case for this function.