Upper Limit to OpenFileRequester Selections?

[davebar]

Windows 10 Pro x64 - PureBasic 5.73 & 6.0beta 10
AMD FX-8350 Eight-Core Processor - RAM DDR3 32768 MBytes

Some months ago I created a small program for a client which uses the OpenFileRequester function and has been working perfectly, but suddenly they are encountering a failure, when using "Ctrl+A" to select all the files in the directory and the funtion return with some (not all) directories.

Code: Select all

InFile.s = ""
InFile = OpenFileRequester("Choose some files", "", "", 0, #PB_Requester_MultiSelection)

The directory file count is quite high in many cases, up to 10,000 files. The issue seems to manifest itself when the files selected exceeds around 5-6 thousand and OpenFileRequester returns an empty string.

Adapting the code to use ExamineDirectory works as a kluge work-around, but lacks the flexibility and usability of the OpenFileRequester code.

I have searched for MS documentation, but have not found any information about any limit on file selection. Does anyone here have any thoughts or pointers on this subject?

Dave

[Marc56us]

32-bit Windows had a limit of 65535 characters in total for the OpenFileDialog and it is possible that this is not increased in 64-bit (no info found)
This is quite logical because it is not advisable to use this type of dialog to select thousands of files.

[davebar]

Hi Marc56us,
Not sure how the "number of characters" limitation relates to this issue.
OpenFileRequester returns a file name string, which in the case of my code rarely ever exceeds a string length of 16 characters, but that does not necessarily mean that something else within the function call is not impacted by this limitation.
Thank you for taking the time to consider my issue.
Regards
Dave

[skywalk]

The character count includes the filenames' full paths?
You can gobble up characters pretty quickly.
Continue to roll your own requester if you are dealing with multi-thousand file counts.

[davebar]

Hi Skywalk, but where inside the OpenFileRequester function does the return string exceed the max possible string length? (eg. return value = "D:\abc\files"). The thing I am trying to track down is "what is going on inside the OpenFileRequester function?".
Go back to my original post where I said "files selected exceeds around 5-6 thousand and OpenFileRequester returns an empty string."
Thank you for taking the time to consider my issue.
Regards
Dave

[jacdelad]

"D:\ab\files" * 5500 exceeds 65535 characters (also theres a delimiter between each file), which would prove the claim, that you exceed the limit (which indeed didn't change in Win64).

BTW: I tried it myself and the function works until you exceed the chraracter limit. Then an empty string ist returned, because Windows returns an error.

[davebar]

Ah! Thanks jacdelad
Now we are getting somewhere.
So now we have conclusively established that OpenFileRequester is a worthless function, when the file count exceeds some undisclosed limit. Fred (whoever) maybe this should be added to the "help" file?
Kind Regards
Dave

[jacdelad]

Beside the fact that a notice within the help would be indeed helpful there is nothing that Fred can do. The best option for you is, like skywalk said, to create your own FileRequester. If the files all have the same extension you could let the user select the folder and then list all the suitable files into a list and let the user manually delete the ones that are not needed; or somtehing like that.

[davebar]

Thanks jacdelad,
However, that is irrelevant, because the directory already ONLY contains files that the user requires to process , so there in no deletion requirement.
I genuinely thank you for taking the time to consider my issue.
Regards
Dave

[jacdelad]

Then the best and most convenient way would be to use PathRequester() anyway.

[breeze4me]

It's because the internal buffer size is not enough.
If you allocate a buffer with a sufficient size as shown in the code below, there is no problem.

Code: Select all

EnableExplicit

Procedure OpenFileDlg(sTitle.s, sDefaultFile.s, sPattern.s, iPatternPosition, List Files.s())
  
  #MaxCharCount = 800000
  
  Protected Result, ofn.OPENFILENAME
  Protected *Files, *Pattern, *str, *c.Character
  Protected sInitDir.s, sFilename.s
  
  *Files = AllocateMemory((#MaxCharCount + 2) * SizeOf(Character))
  If *Files
    If sDefaultFile
      If FileSize(sDefaultFile) = -2
        sInitDir = sDefaultFile
        sDefaultFile = ""
      Else
        sInitDir = GetPathPart(sDefaultFile)
        sDefaultFile = GetFilePart(sDefaultFile)
      EndIf
      
      If Right(sInitDir, 1) <> "\"
        sInitDir + "\"
      EndIf
      
      If sDefaultFile
        PokeS(*Files, sDefaultFile)
      EndIf
    EndIf
    
    If sPattern
      *Pattern = AllocateMemory(StringByteLength(sPattern) + SizeOf(Character) * 2)
      If *Pattern
        CopyMemory(@sPattern, *Pattern, StringByteLength(sPattern))
        
        *c = *Pattern
        While *c\c
          If *c\c = '|'
            *c\c = 0
          EndIf
          *c + SizeOf(Character)
        Wend
      EndIf
    EndIf
    
    If iPatternPosition < 0 : iPatternPosition = 0 : EndIf
    
    With ofn
      \lStructSize = SizeOf(OPENFILENAME)
      \lpstrFilter = *Pattern
      \nFilterIndex = iPatternPosition + 1
      \lpstrFile = *Files
      \nMaxFile = #MaxCharCount
      If sTitle
        \lpstrTitle = @sTitle
      EndIf
      \lpstrInitialDir = @sInitDir
      \Flags = #OFN_EXPLORER | #OFN_ALLOWMULTISELECT | #OFN_NONETWORKBUTTON | #OFN_FILEMUSTEXIST
    EndWith
    
    ClearList(Files())
    
    If GetOpenFileName_(ofn)
      sInitDir = PeekS(*Files, ofn\nFileOffset)
      If Right(sInitDir, 1) <> "\"
        sInitDir + "\"
      EndIf
      
      *c = *Files + ofn\nFileOffset * SizeOf(Character)
      *str = *c
      Repeat
        While *c\c
          *c + SizeOf(Character)
        Wend
        
        sFilename = PeekS(*str)
        If sFilename
          If AddElement(Files())
            Files() = sInitDir + sFilename
          EndIf
        EndIf
        
        *c + SizeOf(Character)
        *str = *c
      Until *c\c = 0
      
      ;Debug "Total bytes: " + Str(*c - *Files + SizeOf(Character))
      
      Result = 1
    EndIf
    
    FreeMemory(*Files)
    If *Pattern : FreeMemory(*Pattern) : EndIf
  EndIf
  
  ProcedureReturn Result
EndProcedure


NewList Files.s()
Define Pattern.s = "All files (*.*)|*.*|PureBasic (*.pb, *.pbi)|*.pb;*.pbi"

If OpenFileDlg("Choose some files", "C:\", Pattern, 0, Files())
  
  Debug "Selected file count: " + ListSize(Files())
  
  ForEach Files()
    Debug Files()
  Next
  
EndIf

[Marc56us]

... the directory already ONLY contains files that the user requires to process ...

So as jacdelad suggested, for that PathRequester() is the right function. Unless the user needs to see the contents of the directory before selecting.

[davebar]

Many thanks for everyone 's input to this thread. Yes, PathRequester() appears to be the solution to my issue. However, I still believe a small note should be added to the PB help file about the OpenFileRequester limit. Such a note would have saved me some frustration and made it unnecessary for me to pester you good folks.
Regards
Dave

[Lord]

As I understand, this is not a PB limit. It's windows limit.