Bug in long path support?
Bug in long path support?
Suppose LongPath$ is the full path to a folder, using the form "\\?\C:\Folder\....\Folder", where Len(Fullpath$) is somewhere over 260 characters. The "\\?\" prefix is the syntax for long paths as per MS docs here: https://msdn.microsoft.com/en-us/librar ... s.85).aspx
Also, suppose that all parts of that path except the last one exist already, and the part that already exists is less than 255 characters.
Then, CreateDirectory(LongPath$) returns success, but actually creates a folder with the name truncated to a total path length around 257 characters.
Using the native Win32 API call directly creates the folder correctly without truncation, i.e. CreateDirectory_(LongPath$, #NUL) works fine.
It's as if PB's CreateDirectory() function is doing its own truncation of the supplied path, on the expectation that paths that long aren't valid.
Other parts of the FileSystem library do the same. e.g. FileSize(LongPath$) returns -2 to indicate a directory when the truncated version exists, but the full version does not - it's similarly truncating the path.
But, things like ExamineDirectory() work fine, so not everything is doing the truncation.
(PB 5.60)
Also, suppose that all parts of that path except the last one exist already, and the part that already exists is less than 255 characters.
Then, CreateDirectory(LongPath$) returns success, but actually creates a folder with the name truncated to a total path length around 257 characters.
Using the native Win32 API call directly creates the folder correctly without truncation, i.e. CreateDirectory_(LongPath$, #NUL) works fine.
It's as if PB's CreateDirectory() function is doing its own truncation of the supplied path, on the expectation that paths that long aren't valid.
Other parts of the FileSystem library do the same. e.g. FileSize(LongPath$) returns -2 to indicate a directory when the truncated version exists, but the full version does not - it's similarly truncating the path.
But, things like ExamineDirectory() work fine, so not everything is doing the truncation.
(PB 5.60)
Re: Bug in long path support?
PB doesn't supports such syntax for path for now
Re: Bug in long path support?
In the meantime, I've been able to work around it with a small library of functions that use the Win32 API directly, but it would be useful to have native support for it at some point.Fred wrote:PB doesn't supports such syntax for path for now
-
- Addict
- Posts: 4527
- Joined: Thu Jun 07, 2007 3:25 pm
- Location: Berlin, Germany
Re: Bug in long path support?
Absolutely!mikejs wrote:[...] but it would be useful to have native support for it at some point.
+1
Re: Bug in long path support?
There is alot of programs out there not supporting Long filenames. I think this is much to Microsoft's own fault.
if you pass a Path+Filename with a length over 259 chars, only 256 of them will be parsed over as microsoft CMD doesn't support it.
In windows10, it has support for it, but you have to set alot a few settings around for it to work.(Never tested it myself)
if you pass a Path+Filename with a length over 259 chars, only 256 of them will be parsed over as microsoft CMD doesn't support it.
In windows10, it has support for it, but you have to set alot a few settings around for it to work.(Never tested it myself)
Re: Bug in long path support?
@mikejs
Would you be so kind to make this public?I've been able to work around it with a small library of functions that use the Win32 API directly
Re: Bug in long path support?
I read somewhere that the 256 byte lengt api is ANSI, while if you wish to use the NTFS (32768 byte long) api, you have to use unicode
Re: Bug in long path support?
This was put together for a particular application, and so only does the bits needed by that application. I've not done a thorough examination of which bits of the PB filesystem library are long path safe. Based on my own testing, the following are OK:oO0XX0Oo wrote:@mikejsWould you be so kind to make this public?I've been able to work around it with a small library of functions that use the Win32 API directly
- SetFileAttributes() and GetFileAttributes(). Which also work for directory attributes.
- ExamineDirectory(), plus IsDirectory(), NextDirectoryEntry(), DirectoryEntry####(), FinishDirectory()
- FileSize()
- CopyFile()
- DeleteFile()
- DeleteDirectory()
- CreateDirectory()
One complication is that the application that needed these also needed proper error detection and reporting of what went wrong, which means calling GetLastError_() as close as possible to where an error occurred. This means GetLastError_() has to be called within the lib function if an error happens. For my stuff I used a global variable to pass the error number back to the caller in that case. In the code below th_errcode is what is used for that. I've left that in, but removed some bits related to my own logging of whats going on.
Sometimes you can get away with a simple wrapper around the corresponding API call - e.g. CopyFileLP() has nothing else going on. Some of the others are more complicated - e.g. the API call RemoveDirectory_() does not do the recursion for you so we have to handle that ourselves, and dealing with errors gets more complicated (see comments).
Code: Select all
Procedure.q FileSizeLP(file$)
Protected.s parent$, name$
Protected.i dhnd
Protected.q out
; Follow the PB approach of -1 for not found, -2 for a directory.
; Start with it at -1 and update if we find anything.
out=-1
parent$=GetPathPart(file$)
name$=GetFilePart(file$)
dhnd=ExamineDirectory(#PB_Any, parent$, name$)
If IsDirectory(dhnd)
While NextDirectoryEntry(dhnd)
If DirectoryEntryType(dhnd)=#PB_DirectoryEntry_File
If LCase(Trim(name$))=LCase(Trim(DirectoryEntryName(dhnd)))
out=DirectoryEntrySize(dhnd)
EndIf
EndIf
If DirectoryEntryType(dhnd)=#PB_DirectoryEntry_Directory
If LCase(Trim(name$))=LCase(Trim(DirectoryEntryName(dhnd)))
; -2 for a directory.
out=-2
EndIf
EndIf
Wend
FinishDirectory(dhnd)
EndIf
ProcedureReturn out
EndProcedure
Procedure.l DirExistsLP(dir$)
Protected.l out
out=#False
; Using long path aware version of filesize, which follows the PB approach of -1 for not found
; and -2 for a directory.
If FileSizeLP(dir$)=-2
out=#True
EndIf
ProcedureReturn out
EndProcedure
Procedure.l FileExistsLP(file$)
Protected.l out
out=#False
Protected.q tmp
; Using long path aware version of filesize, which follows the PB approach of -1 for not found
; and -2 for a directory.
tmp=FileSizeLP(file$)
If tmp=-2
; dir
Else
If tmp=-1
; not found
Else
; found, not a dir.
out=#True
EndIf
EndIf
ProcedureReturn out
EndProcedure
Procedure.l CreateDirectoryLP(dir$)
Protected.l out
; Just call CreateDirectory_() instead. #NUL gives us default security, which is what the PB function would have done.
out=CreateDirectory_(dir$, #NUL)
; update th_errcode
; 0 indicates error
If out=0
th_errcode=GetLastError_()
EndIf
ProcedureReturn out
EndProcedure
Procedure.l DeleteFileLP(file$, mode.l=0, recursion.b=#False)
Protected.l out
If (mode & #PB_FileSystem_Force)
; explicitly remove attributes first.
SetFileAttributes(file$, #PB_FileSystem_Normal)
EndIf
; Can now call DeleteFile_()
out=DeleteFile_(file$)
; update th_errcode
; 0 indicates error
If out=0
th_errcode=GetLastError_()
EndIf
ProcedureReturn out
EndProcedure
Procedure.l DeleteDirectoryLP(dir$, pattern$, mode.l=0, recursion.b=#False)
Protected.l out
Protected.l result
Protected.i dhnd
Protected.s name$
Protected.l errcode
; Handling the errcode value here is tricky.
; If we have errors during the recursion, we'll leave something behind in the folder,
; and the final RemoveDirectory_() call will always return a "directory not empty" error,
; which is technically correct but hides the underlying problem.
; The errorcode for this is 145.
; So, there are two things we need to do to provide a more meaningful error.
; 1. On initial entry to the function (i.e. not on recursive calls), set th_errcode to 0.
; 2. On getting a 0 returned from RemoveDirectory_(), we only override th_errcode if it's still 0
; or the errcode is something other than 145.
; Firstly, if we're not calling ourselves recursively, set th_errcode to 0.
If recursion=#False
th_errcode=0
EndIf
; also to match PB, we have a default pattern$, but the parameter is not optional.
If pattern$="" : pattern$="*.*" : EndIf
; Need to do our own recursion, before calling RemoveDirectory_()
dhnd=ExamineDirectory(#PB_Any, dir$, pattern$)
If IsDirectory(dhnd)
While NextDirectoryEntry(dhnd)
name$=DirectoryEntryName(dhnd)
If name$="." Or name$=".."
; skip it
Else
; If it's a directory, recurse into it, if the mode says to.
If DirectoryEntryType(dhnd)=#PB_DirectoryEntry_Directory
If (mode & #PB_FileSystem_Recursive)
DeleteDirectoryLP(dir$+"\"+name$, pattern$, mode, #True)
EndIf
EndIf
; If it's a file, delete it with DeleteFileLP()
If DirectoryEntryType(dhnd)=#PB_DirectoryEntry_File
DeleteFileLP(dir$+"\"+name$, mode)
EndIf
EndIf
Wend
FinishDirectory(dhnd)
EndIf
; Finally, call RemoveDirectory_()
out=RemoveDirectory_(dir$)
; 0 indicates error
If out=0
errcode=GetLastError_()
; Use this to overwrite th_errcode if it' still 0.
If th_errcode=0
th_errcode=errcode
EndIf
; or if errcode is something other than 145
If errcode<>145
th_errcode=errcode
EndIf
EndIf
ProcedureReturn out
EndProcedure
Procedure.l CopyFileLP(src$, dest$)
Protected.l out
; Just call CopyFile_ instead. #False means we don't fail if the dest exists.
out=CopyFile_(src$, dest$, #False)
; update th_errcode
; 0 indicates error
If out=0
th_errcode=GetLastError_()
EndIf
ProcedureReturn out
EndProcedure
Re: Bug in long path support?
Windows has had long path support for a long time - you just had to specify the path in such a way that you were telling windows that you were fine with long paths and weren't limited by by the 256-ish character limit. Long paths using the prefix ("\\?\") work fine in CMD, as do utilities that prepend that prefix as needed (RoboCopy; the thing I'm writing here), and they work fine on Win7 and probably earlier.GenRabbit wrote:There is alot of programs out there not supporting Long filenames. I think this is much to Microsoft's own fault.
if you pass a Path+Filename with a length over 259 chars, only 256 of them will be parsed over as microsoft CMD doesn't support it.
In windows10, it has support for it, but you have to set alot a few settings around for it to work.(Never tested it myself)
What's changed in Windows 10 is that MS is starting to have long path support on by default without the application needing to say that it's OK with paths over 256 characters. This means that more things "just work" with long paths without having to do anything special, with the risk that a long path might be passed to an application that genuinely cannot handle it.
Re: Bug in long path support?
Thanks a lot for providing these functions (and explanations), mikejs!
Re: Bug in long path support?
Forgot to mention - when using the LP aware functions, the path you pass has to have already been prepended by the appropriate prefix to indicate to Windows that you're OK with long paths (unless you're on a new enough Win10 build for that to be implicit).
I have a function for ensuring this prefix is present and in the right format:
I have a function for ensuring this prefix is present and in the right format:
Code: Select all
Procedure.s EnsureLongPath(path$)
Protected out$
; Take the supplied path and ensure that it has the necessary prefix to work as a long path
; See this doc: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
; Essentially, drive letter paths need to end up as \\?\C:\whatever
; i.e. a simple prefix of \\?\
; Meanwhile, unc paths need to convert:
; \\server\share
; to
; \\?\UNC\server\share
; This is a bit trickier as it isn't a simple prefix - we need to lop off one of the \ from the original
; before prefixing.
; Either way, if it already starts with \\?\, we have no work to do.
If Left(path$,4)="\\?\"
; path already has long path support prefix
out$=path$
Else
; Is it a UNC?
If Left(path$,2)="\\"
out$="\\?\UNC"+Mid(path$,2)
Else
; Is it a drive letter?
If Mid(path$,2,1)=":"
; Add long path support to the path.
out$="\\?\"+path$
Else
; Dunno what kind of path this is. Leave it alone.
out$=path$
EndIf
EndIf
EndIf
ProcedureReturn out$
EndProcedure
Re: Bug in long path support?
Ok, I always thought longpathandfilename where not supported in CMD.mikejs wrote:Windows has had long path support for a long time - you just had to specify the path in such a way that you were telling windows that you were fine with long paths and weren't limited by by the 256-ish character limit. Long paths using the prefix ("\\?\") work fine in CMD, as do utilities that prepend that prefix as needed (RoboCopy; the thing I'm writing here), and they work fine on Win7 and probably earlier.GenRabbit wrote:There is alot of programs out there not supporting Long filenames. I think this is much to Microsoft's own fault.
if you pass a Path+Filename with a length over 259 chars, only 256 of them will be parsed over as microsoft CMD doesn't support it.
In windows10, it has support for it, but you have to set alot a few settings around for it to work.(Never tested it myself)
What's changed in Windows 10 is that MS is starting to have long path support on by default without the application needing to say that it's OK with paths over 256 characters. This means that more things "just work" with long paths without having to do anything special, with the risk that a long path might be passed to an application that genuinely cannot handle it.
I knew NTFS support it.
Form Win10, version 1607, you no longer need the prefix ("\\?\") as far as I can tell from MSDN
"Tip Starting with Windows 10, version 1607, you can opt-in to remove the MAX_PATH limitation without prepending "\\?\". See the "Maximum Path Length Limitation" section of Naming Files, Paths, and Namespaces for details."