Find all files in one folder (or list folders in subfolder)

[GenRabbit]

The new version is even faster than the old one, nice work!

A bug, though...

Code: Select all

fd\Options = #FD_RECURSE | #FD_FILES | #FD_FULLPATH

Even with #FD_FULLPATH set, the Result.txt file contains only file names, no paths?

Thats not a bug.

Code: Select all

    ; * #FD_FULLPATH *
    ; Files/Folders are stored with fullpath. (will not override #FD_Files)

With the #FD_Files, you should only get files. #FD_FULLPATH is only for use with folders.
Crappy description.. #FD_FILES will make #FD_FULLPATH obsolete.

I'll change it.

It was just set as like that as an example on how to set the flags. #FD_FULLPATH will most likely used with only #FD_RECURSE


If you wan't it even faster(but you need to have HUGE folder structures to notice) do this;

Change Options.w to Options.i in the structure

[oO0XX0Oo]

Thats not a bug

Ah, ok. Maybe the description was just a bit misleading then

Change Options.w to Options.i in the structure

No noticeable difference here. 86GB, 135660 files, 18901 folders
But it doesn't really matter, it's incredible fast (~6500ms) anyway...

Would it be possible to get file / folder attributes with this method
as well (like created date, last modified date)?

[GenRabbit]

Thats not a bug

Ah, ok. Maybe the description was just a bit misleading then

Change Options.w to Options.i in the structure

No noticeable difference here. 86GB, 135660 files, 18901 folders
But it doesn't really matter, it's incredible fast (~6500ms) anyway...

Would it be possible to get file / folder attributes with this method
as well (like created date, last modified date)?

I think so. Give me and hour and I'll see what I can come up with.
But I'll add that as an option, I think there will be performance hit.

[GenRabbit]

Huge Update..
@Davido : You Wanted Modules.. You got it! (I think this is what you mean by modules)
@oO0XX0Oo : You wanted support for Attributes.. You got it!

Ive added all attributes for Windows and MAC/Linux. They are Seperated into (MAC & Linux) and Windows Group.

I do not have a MAC so I can't test that part. More Will be added later and code cleaned more up.

How to add Functions like Attributes..

Insert what you want between these lines

Code: Select all

       ; Insert Macros Begin
       ; Insert Macros end

Example. If you wish filesize and Date Modified. Do it like this

Code: Select all

       ; Insert Macros Begin
       Macro_FileSize_Seperator
       Macro_DateModified_Seperator
       ; Insert Macros end

You can also do it like this

Code: Select all

       ; Insert Macros Begin
       Macro_FileSize
       InsertSeperator #FILESIZESEPERATOR$
       Macro_DateModified
       InsertSeperator #DATESEPERATOR$
       ; Insert Macros end

They both do the same.
You can also do it like this
InsertSeperator "&&&" if you prefer or any other choice. This gets added to the end of the string handling the Attributes

The difference between Macro_FileSize and Macro_FileSize_Seperator is that the later adds the seperator instantly.

Macro_FileSize_Seperator adds #FILESIZESEPERATOR$ as seperator
MACRO_DATEXXXXXX_SEPERATOR adds #DATESEPERATOR$ as seperator
and MACRO_ATTRIBUTEXXXXXX_SEPERATOR adds #ATTRIBUTESEPERATOR$ as seperator

Play with the Macros to your hearts content.


Gotta get some sleep.

Code: Select all

Obselete code. See post further down for upgrades

[oO0XX0Oo]

That's really a huge update

Performance hit wise: To include file size, modified + created date the execution time
nearly doubled (1600ms vs 800ms) for ~55k files in ~8k folders.

Absolutely acceptable performance hit!

[davido]

@GenRabbit,

That's really a huge update
Performance hit wise: To include file size, modified + created date the execution time
nearly doubled (1600ms vs 800ms) for ~55k files in ~8k folders.

Absolutely acceptable performance hit!

I agree. Results on my PC are of the same order.

unfortunately this update does not run on my MacBook, with the following error:

Line 348: Constant not found: #MODULE_READUSER.

I cannot test it on Linux, but it might suffer the same error, there, too.

I must apologise:
For not explaining my reference to 'Module' and also for not taking the opportunity to do so when you requested it. Sorry.

I was actually referring to:
http://www.purebasic.com/documentation/ ... odule.html

[GenRabbit]

Davido : Think this fixed it

While cleaning up the code before posting, I forgot to rename #MODULE_READUSER to #MODULEATTRIBUTEREADUSER, #MODULE_WRITEUSER to #MODULEATTRIBUTEWRITEUSER
and so on. Think its fixed now.

Code: Select all

Obsolete code, see latest below

[GenRabbit]

Davido : I'll look into MODULE, at this moment in time I'm kindof confussed on what it exactly does. Seems like a more beefed version of a macro or something.

[davido]

@GenRabbit,
You have clearly spent a lot of time on this and done a great job with this code; I am very happy with it as it is.

A Module is usually easier to add to existing code, as it can remove name conflicts.
Module was a suggestion not a request.

[GenRabbit]

Davido: I guessed it was a Suggestion. And If I can get it better with modules I wan't to.
Just have to figure out How module works. Might be playing around with it tonight

Anyway, Use of Macro within MACRO reduced program with 144 lines..

Hope this helps with readability.

Code: Select all

Obsolete code. see latest below

[GenRabbit]

Davido : You wished for MODULE.. Christmas comes in January..

I Will post the new code in 2 parts.

Part 1 : What you need to use it.

Part 2 : Just for testing

A deeper description on what the different parts of the API is for and what is required for its use;

Code: Select all

    Structure DirectoryOperationsAPI                
        *addresse
        Options.w
        FileDataBlockSizeAdd.i
        Size.i
        Files.i
        Folders.i
        Total.i
    EndStructure

*Addresse / IN
Pointer/Integer
The addresse of the allocated memory which to write the result into. If not set will return with and error equal to (DirectoryOperations::#ERRMSG_MEMORY_NOT_ALLOCATED)
Advisable : Use 1MB (DirectoryOperations::#MemoryBlockSizeStart)

Options / IN
Word
Set this with the flag(s) needed. See below *1*

FileDataBlockSizeAdd / IN
Integer
Optional: If the allocated buffer runs out of memory, this will be added to the buffers size. If not set, the function will allocate the exact amount of memory it needs. This might slow it down.
Advisable. Use 1MB (DirectoryOperations::#MemoryBlockSizeAdd)

Size / OUT
Integer
Returns the total size of memory used for the files or folders names used. Size is in bytes.
Note: This can be smaller than the amount allocated because if the whole memory allocated aren't used it will remove overhead before returning.

Files / OUT
Integer
Returns the number of files found. These are counted even if not specified

Folders / OUT
Integer
Returns the number of folders found. These are counted even if not specified

Total / OUT
Integer
Returns the number of folders and files found in the allocated memory. Useful if you need to add them afterwards to an array Example: Dim.s Mylines(Total - 1)

A quick howto to get it to work;

Code: Select all

    ;*****************************************
    ; Define variables to use
    ;*****************************************
    Define.i ErrorMessage

    ;*****************************************
    ; Path to start from
    ;*****************************************
    RootFolder = "path to your folder\"                                          ; Must include \ (Windows) or / (Mac/Linux) in the end of the path

    ;*****************************************
    ; Prepare an API structure
    ;*****************************************
    Define.DirectoryOperations::DirectoryOperationsAPI API

    ;*****************************************
    ; Allocate memory, set Allocated memory size increase value, and Flags in the options(Optional)
    ;*****************************************
    API\addresse = AllocateMemory(DirectoryOperations::#MemoryBlockSizeStart)    ; allocate memory for Filenames and folder names.
    API\FileDataBlockSizeAdd = DirectoryOperations::#MemoryBlockSizeAdd             ; Optional.
    API\Options = DirectoryOperations::#FD_RECURSE                                         ; See Flags Below for options

    ;*****************************************
    ; Lets get started
    ;*****************************************
    ErrorMessage = DirectoryOperations::FileDirectory(RootFolder, @fd)
    ; Everything good?
    if ErrorMessage
        ; Check errormsg.
        ; See *1* Below 
    else
        ; Insert your code here
    endif
    ; And free up the memory
    FreeMemory(API\addresse)                                                 ; When memory is no more needed.

Its very important that you free up the memory when you don't need the data in the buffer anymore



*1*
ERROR messages
#ERRMSG_OUT_OF_MEMORY
- Program couldn't allocate memory.
#ERRMSG_MEMORY_NOT_ALLOCATED
- Memory for API aren't allocated
#ERRMSG_NO_PATH_SPECIFIED
- Folderpath aren't specified
#ERRMSG_NO_SUCH_PATH
- Illegal path

Note : To check the error messages use DirectoryOperations:: as prefix for access to errormessage values. Example DirectoryOperations::#ERRMSG_OUT_OF_MEMORY

*2*
FLAGS ALL
#FD_RECURSE
- Also add subfolders
#FD_FOLDERS
- Add folders only
#FD_FILES
- Add files only. Makes #FD_Fullpath obselete as it will ONLY return the names. No path.
#FD_FULLPATH
- Adds the fullpath to file/folder
#FD_FILESIZE
- Adds Filesize to filename/Foldername
#FD_DATEMODIFIED
- Adds Date Modified to filename/Foldername
#FD_DATEACCESSED
- Adds date accessed to filename/Foldername
FLAGS WINDOWS ONLY
#FD_DATECREATED
- Date file was created. NOTE: Created can be newer than modified.
#FD_FileSystemHidden
- Attribute : hidden
#FD_FileSystemArchive
- Attribute : Archive
#FD_FileSystemCompressed
- Attribute : Compressed
#FD_FileSystemNormal
- Attribute : Normal
#FD_FileSystemReadOnly
- Attribute : ReadOnly
#FD_FileSystemSystem
- Attribute : System file
FLAGS MAC & LINUX ONLY
#FD_FileSystemLink
- Attribute : Link
#FD_FileSystemReadUser
- Attribute : ReadUser
#FD_FileSystemWriteUser
- Attribute : WriteUser
#FD_FileSystemExecUser
- Attribute : ExecUser
#FD_FileSystemReadGroup
- Attribute : ReadGroup
#FD_FileSystemWriteGroup
- Attribute : WriteGroup
#FD_FileSystemExecGroup
- Attribute : ExecGroup
#FD_FileSystemReadAll
- Attribute : ReadAll
#FD_FileSystemWriteAll
- Attribute : WriteAll
#FD_FileSystemExecAll
- Attribute : ExecAll

Note: Use of #FD_FILES will make the flag #FD_FULLPATH obsolete
Note 2: To get the attribute values from module use DirectoryOperations:: as prefix. Example DirectoryOperations::#FD_RECURSE

To get data out of buffer
Example 1: Using API\Total as end marker

Code: Select all

    x = 0
    If CreateFile(0,RootFolder + "Result.txt")
        For y = 1 to API\Total
            TempName = PeekS(API\addresse + x, -1, #PB_Unicode)
            TempLength = (Len(TempName) + 1) * DirectoryOperations::#Length_Char
            WriteStringN(0, TempName, #PB_Ascii)
            x + TempLength
        Next y
        CloseFile(0)
    EndIf

Example 2: Using API\Size as end marker

Code: Select all

    x = 0
    If CreateFile(0,RootFolder + "Result.txt")
        While fd\Size > x
            TempName = PeekS(API\addresse + x, -1, #PB_Unicode)
            TempLength = (Len(TempName) + 1) * DirectoryOperations::#Length_Char
            WriteStringN(0, TempName, #PB_Ascii)
            x + TempLength
        Wend
        CloseFile(0)
    EndIf

Code Part 1

Code: Select all

Obsolete code, see below

Code Part 2

Code: Select all

Obsolete code, see below

[davido]

@GenRabbit,
That was quick!
Thank you, very much.

I have only tried it on my MacBook, but it works perfectly, for me.

Just one point it will only recognise

Code: Select all

fd\Options = DirectoryOperations::#FD_RECURSE | DirectoryOperations::#FD_FILESIZE

The remainder product an not found error.

Code: Select all

| DirectoryOperations::#FD_DATECREATED | DirectoryOperations::#FD_DATEACCESSED | DirectoryOperations::#FD_DATEMODIFIED

[GenRabbit]

@GenRabbit,
That was quick!
Thank you, very much.

I have only tried it on my MacBook, but it works perfectly, for me.

Just one point it will only recognise

Code: Select all

fd\Options = DirectoryOperations::#FD_RECURSE | DirectoryOperations::#FD_FILESIZE

The remainder product an not found error.

Code: Select all

| DirectoryOperations::#FD_DATECREATED | DirectoryOperations::#FD_DATEACCESSED | DirectoryOperations::#FD_DATEMODIFIED

if DirectoryOperations::#FD_DATECREATED is what it chokes on, its not a bug. (See my extended post above, under FLAGS(It wasen't done yet when you posted.))
(DirectoryOperations::#FD_DATECREATED) is windows only.
on Mac/Linux use DirectoryOperations::#FD_DATEMODIFIED for created date.

If you think its an good idea, I could make it available for MAC, but you will get back the same you get with DirectoryOperations::#FD_DATEMODIFIED

I hope the deeper explanation above helps.

Oh, and the latest code is above, Pasted in about 05:56 AM.

[GenRabbit]

A deeper description on what the different parts of the API is for and what is required for its use;

Code: Select all

    Structure DirectoryOperationsAPI                
        *addresse
        Options.w
        FileDataBlockSizeAdd.i
        Size.i
        Files.i
        Folders.i
        Total.i
    EndStructure

*Addresse / IN
Pointer/Integer
The addresse of the allocated memory which to write the result into. If not set will return with and error equal to (DirectoryOperations::#ERRMSG_MEMORY_NOT_ALLOCATED)
Advisable : Use 1MB (DirectoryOperations::#MemoryBlockSizeStart)

Options / IN
Word
Set this with the flag(s) needed. See below *1*

FileDataBlockSizeAdd / IN
Integer
Optional: If the allocated buffer runs out of memory, this will be added to the buffers size. If not set, the function will allocate the exact amount of memory it needs. This might slow it down.
Advisable. Use 1MB (DirectoryOperations::#MemoryBlockSizeAdd)

Size / OUT
Integer
Returns the total size of memory used for the files or folders names used. Size is in bytes.
Note: This can be smaller than the amount allocated because if the whole memory allocated aren't used it will remove overhead before returning.

Files / OUT
Integer
Returns the number of files found. These are counted even if not specified

Folders / OUT
Integer
Returns the number of folders found. These are counted even if not specified

Total / OUT
Integer
Returns the number of folders and files found in the allocated memory. Useful if you need to add them afterwards to an array Example: Dim.s Mylines(Total - 1)

A quick howto to get it to work;

Code: Select all

    ;*****************************************
    ; Define variables to use
    ;*****************************************
    Define.i ErrorMessage

    ;*****************************************
    ; Path to start from
    ;*****************************************
    RootFolder = "path to your folder\"                                          ; Must include \ (Windows) or / (Mac/Linux) in the end of the path

    ;*****************************************
    ; Prepare an API structure
    ;*****************************************
    Define.DirectoryOperations::DirectoryOperationsAPI API

    ;*****************************************
    ; Allocate memory, set Allocated memory size increase value, and Flags in the options(Optional)
    ;*****************************************
    API\addresse = AllocateMemory(DirectoryOperations::#MemoryBlockSizeStart)    ; allocate memory for Filenames and folder names.
    API\FileDataBlockSizeAdd = DirectoryOperations::#MemoryBlockSizeAdd             ; Optional.
    API\Options = DirectoryOperations::#FD_RECURSE                                         ; See Flags Below for options

    ;*****************************************
    ; Lets get started
    ;*****************************************
    ErrorMessage = DirectoryOperations::FileDirectory(RootFolder, @fd)
    ; Everything good?
    if ErrorMessage
        ; Check errormsg.
        ; See *1* Below 
    else
        ; Insert your code here
    endif
    ; And free up the memory
    FreeMemory(API\addresse)                                                 ; When memory is no more needed.

Its very important that you free up the memory when you don't need the data in the buffer anymore



*1*
ERROR messages
#ERRMSG_OUT_OF_MEMORY
- Program couldn't allocate memory.
#ERRMSG_MEMORY_NOT_ALLOCATED
- Memory for API aren't allocated
#ERRMSG_NO_PATH_SPECIFIED
- Folderpath aren't specified
#ERRMSG_NO_SUCH_PATH
- Illegal path

Note : To check the error messages use DirectoryOperations:: as prefix for access to errormessage values. Example DirectoryOperations::#ERRMSG_OUT_OF_MEMORY

*2*
FLAGS ALL
#FD_RECURSE
- Also add subfolders
#FD_FOLDERS
- Add folders only
#FD_FILES
- Add files only. Makes #FD_Fullpath obselete as it will ONLY return the names. No path.
#FD_FULLPATH
- Adds the fullpath to file/folder
#FD_FILESIZE
- Adds Filesize to filename/Foldername
#FD_DATEMODIFIED
- Adds Date Modified to filename/Foldername
#FD_DATEACCESSED
- Adds date accessed to filename/Foldername
FLAGS WINDOWS ONLY
#FD_DATECREATED
- Date file was created. NOTE: Created can be newer than modified.
#FD_FileSystemHidden
- Attribute : hidden
#FD_FileSystemArchive
- Attribute : Archive
#FD_FileSystemCompressed
- Attribute : Compressed
#FD_FileSystemNormal
- Attribute : Normal
#FD_FileSystemReadOnly
- Attribute : ReadOnly
#FD_FileSystemSystem
- Attribute : System file
FLAGS MAC & LINUX ONLY
#FD_FileSystemLink
- Attribute : Link
#FD_FileSystemReadUser
- Attribute : ReadUser
#FD_FileSystemWriteUser
- Attribute : WriteUser
#FD_FileSystemExecUser
- Attribute : ExecUser
#FD_FileSystemReadGroup
- Attribute : ReadGroup
#FD_FileSystemWriteGroup
- Attribute : WriteGroup
#FD_FileSystemExecGroup
- Attribute : ExecGroup
#FD_FileSystemReadAll
- Attribute : ReadAll
#FD_FileSystemWriteAll
- Attribute : WriteAll
#FD_FileSystemExecAll
- Attribute : ExecAll

Note: Use of #FD_FILES will make the flag #FD_FULLPATH obsolete
Note 2: To get the attribute values from module use DirectoryOperations:: as prefix. Example DirectoryOperations::#FD_RECURSE

To get data out of buffer
Example 1: Using API\Total as end marker

Code: Select all

    x = 0
    If CreateFile(0,RootFolder + "Result.txt")
        For y = 1 to API\Total
            TempName = PeekS(API\addresse + x, -1, #PB_Unicode)
            TempLength = (Len(TempName) + 1) * DirectoryOperations::#Length_Char
            WriteStringN(0, TempName, #PB_Ascii)
            x + TempLength
        Next y
        CloseFile(0)
    EndIf

Example 2: Using API\Size as end marker

Code: Select all

    x = 0
    If CreateFile(0,RootFolder + "Result.txt")
        While fd\Size > x
            TempName = PeekS(API\addresse + x, -1, #PB_Unicode)
            TempLength = (Len(TempName) + 1) * DirectoryOperations::#Length_Char
            WriteStringN(0, TempName, #PB_Ascii)
            x + TempLength
        Wend
        CloseFile(0)
    EndIf

How to add Functions like Attributes..
All macros as the program stands now is turned of.

How to activate:
For this example, we will need the modifiy attribute. To get the code and Attribute settings we have to set the flags.
Change

Code: Select all

#EXTENDEDFILEDIRECTORY = 0
#MODULE_DATEMODIFIED = 0

to this

Code: Select all

#EXTENDEDFILEDIRECTORY = 1
#MODULE_DATEMODIFIED = 1

This will make these two macros and one flag available available.
Macrosd and Flag;
Macro_DateModified
Macro_DateModified_Seperator
#FD_DATEMODIFIED

If you use Macro_DateModified, you should call InsertSeperator just after to add your own seperator.
example

Code: Select all

Macro_DateModified
InsertSeperator " "

This will insert a space between the Datemodified and files or folder names following.
If a space is what you want to use, these to can be replaced with just Macro_DateModified_Seperator

Insert Functions(Macros) you want between these lines

Code: Select all

       ; Insert Macros Begin
       ; Insert Macros end

Example. for the above

Code: Select all

       ; Insert Macros Begin
       Macro_DateModified_Seperator
       ; Insert Macros end

You can also do it like this

Code: Select all

       ; Insert Macros Begin
       Macro_DateModified
       InsertSeperator #DATESEPERATOR$
       ; Insert Macros end

Main Program Code

Code: Select all

DeclareModule DirectoryOperations
    EnableExplicit
    #EXTENDEDFILEDIRECTORY = 0
    CompilerIf #EXTENDEDFILEDIRECTORY
            #MODULE_DATEACCESSED             = 0
            #MODULE_DATEMODIFIED             = 0
            #MODULE_FILESIZE                 = 0
        CompilerIf #PB_Compiler_OS = #PB_OS_Windows        
            ; Windows Only                
            #MODULE_DATECREATED              = 0
            #MODULE_ATTRIBUTEHIDDEN          = 0
            #MODULE_ATTRIBUTEARCHIVE         = 0
            #MODULE_ATTRIBUTECOMPRESSED      = 0
            #MODULE_ATTRIBUTENORMAL          = 0
            #MODULE_ATTRIBUTEREADONLY        = 0
            #MODULE_ATTRIBUTESYSTEM          = 0
        CompilerElse    
            ; Mac & linux Only
            #MODULE_ATTRIBUTELINK            = 0
            #MODULE_ATTRIBUTEREADUSER        = 0
            #MODULE_ATTRIBUTEWRITEUSER       = 0
            #MODULE_ATTRIBUTEEXECUSER        = 0
            #MODULE_ATTRIBUTEREADGROUP       = 0
            #MODULE_ATTRIBUTEWRITEGROUP      = 0
            #MODULE_ATTRIBUTEEXECGROUP       = 0
            #MODULE_ATTRIBUTEREADALL         = 0
            #MODULE_ATTRIBUTEWRITEALL        = 0
            #MODULE_ATTRIBUTEEXECALL         = 0
        CompilerEndIf
    CompilerEndIf
    CompilerSelect #PB_Compiler_Processor
        CompilerCase #PB_Processor_x86
            #Length_Integer = 4
        CompilerCase #PB_Processor_x64
            #Length_Integer = 8
    CompilerEndSelect
    
    CompilerSelect #PB_Compiler_OS
        CompilerCase #PB_OS_Windows
            #FolderDelimiter$ = "\"
        CompilerDefault
            #FolderDelimiter$ = "/"
    CompilerEndSelect
    
    CompilerIf #PB_Compiler_Unicode
        #Length_Char = 2
    CompilerElse
        #Length_Char = 1
    CompilerEndIf
    
    ;--- Otional Values for memory allocation
    #MemoryBlockSizeStart = 1024 * 1024 ; Allocated memory 1MB
    #MemoryBlockSizeAdd = 1024 * 1024   ; Allocated memory 1MB
    ;--- Error Flag
    Enumeration ERRMSG_ 10 Step 5
        #ERRMSG_OUT_OF_MEMORY
        #ERRMSG_MEMORY_NOT_ALLOCATED
        #ERRMSG_NO_PATH_SPECIFIED
        #ERRMSG_NO_SUCH_PATH
    EndEnumeration
    
    ;--- Flags
        EnumerationBinary FD_
            #FD_RECURSE
            #FD_FOLDERS         ; Only Foldername
            #FD_FILES           ; Only Filename
            #FD_FULLPATH        ; Fullpath?
            #FD_FILESIZE        ; Filesize
            CompilerIf #MODULE_DATEMODIFIED
                #FD_DATEMODIFIED
            CompilerEndIf    
            CompilerIf #MODULE_DATEACCESSED
                #FD_DATEACCESSED
            CompilerEndIf    
            CompilerIf #PB_Compiler_OS = #PB_OS_Windows
                ; Window specific flags
                CompilerIf #MODULE_DATECREATED
                    #FD_DATECREATED
                CompilerEndIf    
                
                CompilerIf #MODULE_ATTRIBUTEHIDDEN
                    #FD_FileSystemHidden
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEARCHIVE
                    #FD_FileSystemArchive
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTECOMPRESSED
                    #FD_FileSystemCompressed
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTENORMAL
                    #FD_FileSystemNormal
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEREADONLY
                    #FD_FileSystemReadOnly
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTESYSTEM
                    #FD_FileSystemSystem
                CompilerEndIf    
            CompilerElse    
                ; Mac/Linux specific flags
                CompilerIf #MODULE_ATTRIBUTELINK
                    #FD_FileSystemLink
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEREADUSER
                    #FD_FileSystemReadUser    ; Access flags For the owning user
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEWRITEUSER
                    #FD_FileSystemWriteUser
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEEXECUSER
                    #FD_FileSystemExecUser
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEREADGROUP
                    #FD_FileSystemReadGroup   ; Access flags for the owning user's group
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEWRITEGROUP
                    #FD_FileSystemWriteGroup
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEEXECGROUP
                    #FD_FileSystemExecGroup
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEREADALL
                    #FD_FileSystemReadAll     ; Access flags for all other users
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEWRITEALL
                    #FD_FileSystemWriteAll
                CompilerEndIf    
                CompilerIf #MODULE_ATTRIBUTEEXECALL
                    #FD_FileSystemExecAll
                CompilerEndIf
            CompilerEndIf
        EndEnumeration
    
    CompilerIf #EXTENDEDFILEDIRECTORY
        #TIMEFORMAT$ = "%DD-%MM-%YYYY %HH:%II"
        #DATESEPERATOR$ = " "
        #ATTRIBUTESEPERATOR$ = " "
        #FILESIZESEPERATOR$ = " "
    CompilerEndIf    
    
    ;--- Macro's begin
    CompilerIf #EXTENDEDFILEDIRECTORY
        Macro InsertSeperator
            eXtended +
        EndMacro
        Macro Macro_Attributes(FDFLAG, PBFLAG, RESULT1, RESULT2, SEPERATOR)
            If (*fd\Options) & FDFLAG
                If DirectoryEntryAttributes(Exadir) & PBFLAG
                    eXtended + RESULT1 + SEPERATOR
                Else
                    eXtended + RESULT2 + SEPERATOR
                EndIf
            EndIf
        EndMacro
        
        CompilerIf #MODULE_FILESIZE
            Macro Macro_FileSize
                If (*fd\Options) & #FD_FILESIZE
                    eXtended + Str(DirectoryEntrySize(ExaDir))
                EndIf
            EndMacro
            Macro Macro_FileSize_Seperator
                If (*fd\Options) & #FD_FILESIZE
                    eXtended + Str(DirectoryEntrySize(ExaDir))+ #FILESIZESEPERATOR$
                EndIf
            EndMacro
        CompilerEndIf
    
        CompilerIf #MODULE_DATEMODIFIED
            Macro Macro_DateModified
                If (*fd\Options) & #FD_DATEMODIFIED
                    eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Modified))
                EndIf
            EndMacro
            Macro Macro_DateModified_Seperator
                If (*fd\Options) & #FD_DATEMODIFIED
                    eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Modified)) + #DATESEPERATOR$
                EndIf
            EndMacro
        CompilerEndIf
    
        CompilerIf #MODULE_DATEACCESSED
            Macro Macro_DateAccessed
                If (*fd\Options) & #FD_DATEACCESSED
                    eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Accessed))
                EndIf
            EndMacro
    
            Macro Macro_DateAccessed_Seperator
                If (*fd\Options) & #FD_DATEACCESSED
                    eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Accessed)) + #DATESEPERATOR$
                EndIf
            EndMacro
        CompilerEndIf
    
        CompilerIf #PB_Compiler_OS = #PB_OS_Windows
            ; Windows Only
            CompilerIf #MODULE_DATECREATED
                Macro Macro_DateCreated
                    If (*fd\Options) & #FD_DATECREATED
                        eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Created))
                    EndIf
                EndMacro
                Macro Macro_DateCreated_Seperator
                    If (*fd\Options) & #FD_DATECREATED
                        eXtended + FormatDate(#TIMEFORMAT$, DirectoryEntryDate(Exadir , #PB_Date_Created)) + #DATESEPERATOR$
                    EndIf
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEHIDDEN
                Macro Macro_Attributehidden
                    Macro_Attributes(#FD_FileSystemHidden, #PB_FileSystem_Hidden, "[Hidden]", "[]", "")
                EndMacro
                Macro Macro_Attributehidden_Seperator
                    Macro_Attributes(#FD_FileSystemHidden, #PB_FileSystem_Hidden, "[Hidden]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEARCHIVE
                Macro Macro_AttributeArchive
                    If (*fd\Options) & #FD_FileSystemArchive
                        If DirectoryEntryAttributes(Exadir) & #PB_FileSystem_Archive
                            eXtended + "[Not archived]"
                        Else
                            eXtended + "[]"
                        EndIf
                    EndIf
                EndMacro
                Macro Macro_AttributeArchive_Seperator
                    If (*fd\Options) & #FD_FileSystemArchive
                      If DirectoryEntryAttributes(Exadir) & #PB_FileSystem_Archive
                          eXtended + "[Not archived]" + #ATTRIBUTESEPERATOR$
                    Else
                        eXtended + "[]" + #ATTRIBUTESEPERATOR$
                      EndIf
                    EndIf
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTECOMPRESSED
                Macro Macro_AttributeCompressed
                    Macro_Attributes(#FD_FileSystemCompressed, #PB_FileSystem_Compressed, "[Compressed]", "[]", "")
                EndMacro
                Macro Macro_AttributeCompressed_Seperator
                    Macro_Attributes(#FD_FileSystemCompressed, #PB_FileSystem_Compressed, "[Compressed]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTENORMAL
                Macro Macro_AttributeNormal
                    Macro_Attributes(#FD_FileSystemNormal, #PB_FileSystem_Normal, "[Normal]", "[]", "")
                EndMacro
                Macro Macro_AttributeNormal_Seperator
                    Macro_Attributes(#FD_FileSystemNormal, #PB_FileSystem_Normal, "[Normal]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEREADONLY
                Macro Macro_AttributeReadOnly
                    Macro_Attributes(#FD_FileSystemReadOnly, #PB_FileSystem_ReadOnly, "[Read Only]", "[]", "")
                EndMacro
                Macro Macro_AttributeReadOnly_Seperator
                    Macro_Attributes(#FD_FileSystemReadOnly, #PB_FileSystem_ReadOnly, "[Read Only]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTESYSTEM
                Macro Macro_AttributeSystem
                    Macro_Attributes(#FD_FileSystemSystem, #PB_FileSystem_System, "[System File]", "[]", "")
                EndMacro
                Macro Macro_AttributeSystem_Seperator
                    Macro_Attributes(#FD_FileSystemSystem, #PB_FileSystem_System, "[System File]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
        CompilerElse
        ; Mac/Linux Only
            CompilerIf #MODULE_ATTRIBUTELINK
                Macro Macro_AttributeLink
                    Macro_Attributes(#FD_FileSystemLink, #PB_FileSystem_Link, "[Link]", "[]", "")
                EndMacro
                Macro Macro_AttributeLink_Seperator
                    Macro_Attributes(#FD_FileSystemLink, #PB_FileSystem_Link, "[Link]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEREADUSER
                Macro Macro_AttributeReadUser
                    Macro_Attributes(#FD_FileSystemReadUser, #PB_FileSystem_ReadUser, "[Owning User/Read]", "[]", "")
                EndMacro
                Macro Macro_AttributeReadUser_Seperator
                    Macro_Attributes(#FD_FileSystemReadUser, #PB_FileSystem_ReadUser, "[Owning User/Read]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEWRITEUSER
                Macro Macro_AttributeWriteUser
                    Macro_Attributes(#FD_FileSystemWriteUser, #PB_FileSystem_WriteUser, "[Owning User/Write]", "[]", "")
                EndMacro
                Macro Macro_AttributeWriteUser_Seperator
                    Macro_Attributes(#FD_FileSystemWriteUser, #PB_FileSystem_WriteUser, "[Owning User/Write]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEEXECUSER
                Macro Macro_AttributeExecUser
                    Macro_Attributes(#FD_FileSystemExecUser, #PB_FileSystem_ExecUser, "[Owning User/Execute]", "[]", "")
                EndMacro
                Macro Macro_AttributeExecUser_Seperator
                    Macro_Attributes(#FD_FileSystemExecUser, #PB_FileSystem_ExecUser, "[Owning User/Execute]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
            
            CompilerIf #MODULE_ATTRIBUTEREADGROUP
                Macro Macro_AttributeReadGroup
                    Macro_Attributes(#FD_FileSystemReadGroup, #PB_FileSystem_ReadGroup, "[Owning User Group/Read]", "[]", "")
                EndMacro
                Macro Macro_AttributeReadGroup_Seperator
                    Macro_Attributes(#FD_FileSystemReadGroup, #PB_FileSystem_ReadGroup, "[Owning User Group/Read]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEWRITEGROUP
                Macro Macro_AttributeWriteGroup
                    Macro_Attributes(#FD_FileSystemWriteGroup, #PB_FileSystem_WriteGroup, "[Owning User Group/Write]", "[]", "")
                EndMacro
                Macro Macro_AttributeWriteGroup_Seperator
                    Macro_Attributes(#FD_FileSystemWriteGroup, #PB_FileSystem_WriteGroup, "[Owning User Group/Write]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEEXECGROUP
                Macro Macro_AttributeExecGroup
                    Macro_Attributes(#FD_FileSystemExecGroup, #PB_FileSystem_ExecGroup, "[Owning User Group/Exec]", "[]", "")
                EndMacro
                Macro Macro_AttributeExecGroup_Seperator
                    Macro_Attributes(#FD_FileSystemExecGroup, #PB_FileSystem_ExecGroup, "[Owning User Group/Exec]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEREADALL
                Macro Macro_AttributeReadAll
                    Macro_Attributes(#FD_FileSystemReadAll, #PB_FileSystem_ReadAll, "[All Other Users/Read]", "[]", "")
                EndMacro
                Macro Macro_AttributeReadAll_Seperator
                    Macro_Attributes(#FD_FileSystemReadAll, #PB_FileSystem_ReadAll, "[All Other Users/Read]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEWRITEALL
                Macro Macro_AttributeWriteAll
                    Macro_Attributes(#FD_FileSystemWriteAll, #PB_FileSystem_WriteAll, "[All Other Users/Write]", "[]", "")
                EndMacro
                Macro Macro_AttributeWriteAll_Seperator
                    Macro_Attributes(#FD_FileSystemWriteAll, #PB_FileSystem_WriteAll, "[All Other Users/Write]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
    
            CompilerIf #MODULE_ATTRIBUTEEXECALL
                Macro Macro_AttributeExecAll
                    Macro_Attributes(#FD_FileSystemExecAll, #PB_FileSystem_ExecAll, "[All Other Users/Execute]", "[]", "")
                EndMacro
                Macro Macro_AttributeExecAll_Seperator
                    Macro_Attributes(#FD_FileSystemExecAll, #PB_FileSystem_ExecAll, "[All Other Users/Execute]", "[]", #ATTRIBUTESEPERATOR$)
                EndMacro
            CompilerEndIf
        CompilerEndIf
    CompilerEndIf
    ;--- Macro's END
    
    ;--- Structure
    Global.i FileDataBlockSize
    Structure DirectoryOperationsAPI                
        *addresse
        Options.w
        FileDataBlockSizeAdd.i
        Size.i
        Files.i
        Folders.i
        Total.i
    EndStructure
    Declare FileDirectory(RootFolder.s, *fd.DirectoryOperationsAPI)
EndDeclareModule

Module DirectoryOperations
    Procedure.i StoreResult(FilePathOrFilename.s, *Fd.DirectoryOperationsAPI)
        Protected *PathMapNew
        Protected.i LengthToAdd
        Protected.i TempLength = (Len(FilePathOrFilename) + 1) * #Length_Char
        If (*Fd\Size + TempLength) >= FileDataBlockSize
            ; To small block. Time to increase
            If *Fd\FileDataBlockSizeAdd > TempLength
                LengthToAdd = *Fd\FileDataBlockSizeAdd
            Else
                LengthToAdd = TempLength
            EndIf
            
            *PathMapNew = ReAllocateMemory(*Fd\addresse, FileDataBlockSize + LengthToAdd)
            If *PathMapNew
                *Fd\addresse = *PathMapNew
                FileDataBlockSize + LengthToAdd
            Else
                ProcedureReturn #ERRMSG_OUT_OF_MEMORY
            EndIf
        EndIf    
        PokeS(*Fd\addresse + *Fd\Size,FilePathOrFilename)
        *Fd\Total + 1
        *Fd\Size + TempLength
        ProcedureReturn 0
    EndProcedure
    
    Procedure.i RecurseDir(RootFolder.s, PresentRoot.s, *Fd.DirectoryOperationsAPI)
        Protected.i ExaDir, TempLength
        Protected.i *PathMapNew
        Protected.s TempName
        Protected.i TempDate
        Protected.i ForcedQuit
        CompilerIf #EXTENDEDFILEDIRECTORY
            Protected.s eXtended
        CompilerEndIf
        ExaDir = ExamineDirectory(#PB_Any, RootFolder,"*.*")
        If ExaDir
            While NextDirectoryEntry(ExaDir)
                CompilerIf #EXTENDEDFILEDIRECTORY
                    ; Macros Begin
                    ;Macro_DateCreated_Seperator
                    ;Macro_DateModified_Seperator
                    ;Macro_DateAccessed_Seperator
                    ; We now add a seperator to the eXtended so Dates gets a seperator from file or folder name
                    Macro_FileSize
                    InsertSeperator #DATESEPERATOR$
                    ; Macros end
                CompilerEndIf            
                Select DirectoryEntryType(ExaDir)
                    ;--- Folder
                    Case #PB_DirectoryEntry_Directory
                        TempName = DirectoryEntryName(ExaDir)
                        If (TempName <> ".") And (TempName <> "..")
                            *Fd\Folders + 1
                            If (*Fd\Options & #FD_RECURSE)
                                ForcedQuit = RecurseDir(RootFolder + TempName + #FolderDelimiter$, PresentRoot + TempName + #FolderDelimiter$, *Fd)
                                If ForcedQuit
                                    Break
                                EndIf
                            EndIf        
                            If *Fd\Options & #FD_FOLDERS
                                ForcedQuit = StoreResult(PresentRoot + TempName + #FolderDelimiter$, *Fd)
                            EndIf
                            If ForcedQuit
                                Break
                            EndIf
                        EndIf                 
                    ;--- File
                    Case #PB_DirectoryEntry_File
                        TempName = DirectoryEntryName(ExaDir)
                        *Fd\Files + 1
                        If *Fd\Options & #FD_FILES
                            CompilerIf #EXTENDEDFILEDIRECTORY
                                ForcedQuit = StoreResult(eXtended + TempName, *Fd)
                            CompilerElse    
                                ForcedQuit = StoreResult(TempName, *Fd)
                            CompilerEndIf
                        ElseIf Not *Fd\Options & #FD_FOLDERS
                            CompilerIf #EXTENDEDFILEDIRECTORY
                                ForcedQuit = StoreResult(eXtended + PresentRoot + TempName, *Fd)
                            CompilerElse    
                                ForcedQuit = StoreResult(PresentRoot + TempName, *Fd)
                            CompilerEndIf
                        EndIf
                EndSelect
                If ForcedQuit
                    Break
                EndIf
                eXtended = ""
            Wend
            FinishDirectory(ExaDir)
        Else    
            ProcedureReturn #ERRMSG_NO_SUCH_PATH
        EndIf
        ProcedureReturn ForcedQuit
    EndProcedure                
    Procedure.i FileDirectory(RootFolder.s, *fd.DirectoryOperationsAPI)
        ; Initialize values 
        Protected.s subfolder
        Protected.i Result
        Protected *OptimizedBlockSize
        If *fd\addresse
            *fd\Files = 0
            *fd\Folders = 0
            *fd\Size = 0
            *fd\Total = 0
            If Right(RootFolder,1) <> #FolderDelimiter$
                ProcedureReturn #ERRMSG_NO_PATH_SPECIFIED
            EndIf        
            FileDataBlockSize = MemorySize(*fd\addresse) ; Sets \FileDataBlocksize to the Size of the Allocated buffer.
            If *fd\Options & #FD_FULLPATH
                SubFolder = RootFolder
            EndIf
            Result = RecurseDir(RootFolder, SubFolder, *fd)
            If Not Result
                If FileDataBlockSize > *fd\Size
                    *OptimizedBlockSize = ReAllocateMemory(*Fd\addresse, *fd\Size)
                    If *OptimizedBlockSize
                        *Fd\addresse = *OptimizedBlockSize
                    Else
                        ProcedureReturn #ERRMSG_OUT_OF_MEMORY
                    EndIf
                EndIf
            EndIf
        Else
            ProcedureReturn #ERRMSG_MEMORY_NOT_ALLOCATED
        EndIf
    EndProcedure
EndModule

Test Code

Code: Select all

EnableExplicit
;--- local Global Strings variables
Define fd.DirectoryOperations::DirectoryOperationsAPI
; Strings
Define.s RootFolder
#OrigFolderRoot$ = "F:\Projects\Modding\ExtractedFiles\Beyond reach"
RootFolder = PathRequester("Mod Root.",#OrigFolderRoot$)
Define dt.i = ElapsedMilliseconds()
If Len(RootFolder)
    ;--- Program variables which must be set
    fd\addresse = AllocateMemory(DirectoryOperations::#MemoryBlockSizeStart)    ; allocate memory for Filenames and folder names.
    ; FileDataBlockSizeAdd:
    ; The amount of memory to allocate to the already allocated memory (Optional, Advisable.)
    fd\FileDataBlockSizeAdd = DirectoryOperations::#MemoryBlockSizeAdd 
    ; Options
    ; Flags for work operation.
    ; * #FD_RECURSE *
    ; Also take subdirectories
    ; * #FD_FOLDERS *
    ; Only stores foldernames
    ; * #FD_FILES *
    ; Only stores Filenames
    ; * #FD_FULLPATH *
    ; Files/Folders are stored with fullpath. (will not override #FD_Files)
    fd\Options = DirectoryOperations::#FD_RECURSE | DirectoryOperations::#FD_DATECREATED | DirectoryOperations::#FD_DATEACCESSED | DirectoryOperations::#FD_DATEMODIFIED | DirectoryOperations::#FD_FILESIZE
    If DirectoryOperations::FileDirectory(RootFolder, @fd) = 0
        ; ************* Add your own Code here - Below
        MessageRequester("Test!","Files: " + Str(fd\files) + " Folders: " + Str(fd\Folders) + " in " + Str(ElapsedMilliseconds() - dt) + "ms.")
        ;--- local Global Strings variables
        Define fd.DirectoryOperations::DirectoryOperationsAPI
        ; Strings
        Define.s TempName, RootFolder
        ; Integers
        Define.i TempLength

        Define.i x
        If CreateFile(0,RootFolder + "Result.txt")
            While fd\Size > x
                TempName = PeekS(fd\addresse + x, -1, #PB_Unicode)
                TempLength = (Len(TempName) + 1) * DirectoryOperations::#Length_Char
                ;WriteStringN(0, "Del " + #DQUOTE$ + TempName + #DQUOTE$, #PB_Ascii)
                WriteStringN(0, TempName, #PB_Ascii)
                x + TempLength
            Wend
            CloseFile(0)
        EndIf
        ; ************* Add your own Code here - Above
    Else    
        MessageRequester("Error.","Something went Shit..")
        
    EndIf
    FreeMemory(fd\addresse)
EndIf

Unless someone finds some bug in it, I will consider the project done. If I find out a way, I will try to add support for filenames & paths above 256 chars.