[6.30 x64 c-backend] Recursive dir scan corrupts variables (Windows too)

[ColeopterusMaximus]

This another problem that has been driving me absolutely crazy, if someone could please confirm I would be very grateful

I've tested it on Windows on version 6.21 and the problem manifests too there.

The following code has to be run with the C compiler from the IDE as it is:

Open the IDE, create a new file, select the C compiler

Code: Select all

EnableExplicit

Procedure listdir(Directory$)
    Protected hnd.i
    Protected Type$
    Protected Size$

    If Right(Directory$,1) <> "/"
        Directory$ + "/"
    EndIf
    hnd = ExamineDirectory(#PB_Any, Directory$, "*.*")
    If Not hnd
        ProcedureReturn #False
    EndIf
    ;CallDebugger
    While NextDirectoryEntry(hnd)
        If DirectoryEntryType(hnd) = #PB_DirectoryEntry_File
            Type$ = "[File] "
            Size$ = " (Size: " + DirectoryEntrySize(hnd) + ")"
        Else
            Type$ = "[Directory] "
            Size$ = "" ; A directory doesn't have a size
            If DirectoryEntryName(hnd) <> "." And DirectoryEntryName(hnd) <> ".."
                listdir(Directory$ + DirectoryEntryName(hnd))
            Else
                Continue
            EndIf
        EndIf
        
        Debug Type$ + DirectoryEntryName(hnd) + Size$
    Wend
    FinishDirectory(hnd)
EndProcedure

listdir("/")

Set a breakpoint over the line containing:

While NextDirectoryEntry(hnd)

Run the program, hit continue so the code runs and stops again at the break point, it is obviously easier to hit whatever shortcut you use for "continue", keep hitting continue until...

Eventually you will see that the variables Directory$ Type$ and Size$ will display gibberish

Funnily if rather than use a break-point you uncomment the "calldebugger" line the corruption doesn't manifest.

It is not only the gibberish, the scan directory will eventually miss files, this is how I noticed something was off.

If you switch back to the regular ASM compiler all works fine.

This is what the gibberish looks like:

[mk-soft]

Can I confirm

Is probably more just a problem of the debugger when it reads out the variable.
The debug output and the result is fine so far.
At the turning points like "While:Wend", "Repeat:Until" I have also noticed that the debugger hangs briefly
And avoid setting stops at these points.

[ColeopterusMaximus]

Can I confirm

Is probably more just a problem of the debugger when it reads out the variable.
The debug output and the result is fine so far.
At the turning points like "While:Wend", "Repeat:Until" I have also noticed that the debugger hangs briefly
And avoid setting stops at these points.

Many thanks for checking it, appreciated it.

With this simple function it doesn't lose many files, however if the function is more complex and inside a module it produces garbage more often, prematurely aborts the scan or misses lots of files, so there is a problem beyond the debugger, the debugger is the only reproducible manifestation I've been able to produce.

It is very random.

[mk-soft]

Except for the debugger output in this case, I had not found any errors in the reading of directories so far.

[ColeopterusMaximus]

Except for the debugger output in this case, I had not found any errors in the reading of directories so far.

Yes, like mentioned, it is very random and wrong outcome depends on program complexity, the reason I noticed something was "loose" is because I wrote a program that has to archive statistic data on a directory tree, the tool should delete automatically files older than 180 days and directories that become empty as a result of the deletion, so when cleaning up it does some file-tree scanning.

On some computers my program wasn't deleting any files at all, on others it was deleting files but not all the files it should and in another computer in particular it deleted files but didn't remove all empty directories.

Other small programs that reuse the same routines work correctly, but these programs only work on very small amount of files compared to the program where I discovered the problem.

Using either ASM (or the C compiler a non-recursive dir-scan function) works 100% is only the function recursive with the C compiler that produces random results.

[mk-soft]

May be due to your search function or the folder/file is currently locked for deletion (folder not empty, file open)

Code: Select all

;-TOP by mk-soft

; ***************************************************************************************

Structure udtFileInfo
  file.s
  size.q
  date.q
EndStructure

Global Stop

; ***************************************************************************************

Procedure InStr(String.s, StringGruppe.s, Separator.s) ; Result BOOL

  Protected gruppe.s, find.s
  
  ; Suchstring vorhanden
  If Not Bool(String)
    ProcedureReturn #False
  EndIf
  ; Gruppenstring vorhanden
  If Not Bool(StringGruppe)
    ProcedureReturn #False
  EndIf
  ; Gruppenstring mit Separator erweitern
  gruppe = StringGruppe
  If Left(StringGruppe, 1) <> Separator
    gruppe = Separator + gruppe
  EndIf
  If Right(StringGruppe, 1) <> Separator
    gruppe + Separator
  EndIf
  ; Suchstring mit Separator erweitern
  find = Separator + String + Separator
  ; Suchen
  If FindString(gruppe, find, 1, #PB_String_NoCase)
    ProcedureReturn #True
  Else
    ProcedureReturn #False
  EndIf
  
EndProcedure

; ***************************************************************************************

Procedure GetFileInfoRecursive(List FileInfo.udtFileInfo(), Directory.s, Recursive = #True, Exclude.s = "")  
  
  Protected ID, name.s, extension.s
  
  ID = ExamineDirectory(#PB_Any, Directory, "*.*")
  If ID
    While NextDirectoryEntry(ID)  
      name = DirectoryEntryName(ID)  
      If DirectoryEntryType(ID) = #PB_DirectoryEntry_Directory
        If Recursive And name <> ".." And name <> "."
          GetFileInfoRecursive(FileInfo(), Directory+name+#PS$, Recursive, exclude)
        EndIf
      Else
        extension = GetExtensionPart(name)
        If Not InStr(extension, exclude, ";")
          If AddElement(FileInfo()) = 0
            ; Out Of Memory
            Break
          EndIf
          With FileInfo()
            \file = Directory + name
            \size = DirectoryEntrySize(ID)
            \date = DirectoryEntryDate(ID, #PB_Date_Modified)
          EndWith
        EndIf
      EndIf
      ; Abbruch
      If Stop
        Break
      EndIf 
    Wend  
    FinishDirectory(ID)
  EndIf  
  
  ProcedureReturn ListSize(FileInfo())
  
EndProcedure 

; ***************************************************************************************

Global NewList FileInfo.udtFileInfo()

OpenWindow(#PB_Any, 0, 0, 0, 0, "DoEvents", #PB_Window_Invisible | #PB_Window_NoGadgets)

Define dir.s = PathRequester("Path", "")
While WindowEvent() : Wend
Define count = GetFileInfoRecursive(FileInfo(), dir, #True, "bak;tmp")
Debug count
ShowVariableViewer() 
CallDebugger