Unable to read same file in parallel

[sepp]

Hello all,

I need to read a logfile in two different applications at the same time. As PB internal functions do not allow to set the Share-Mode I am using direct API-Calls. There I am using #FILE_SHARE_READ, but the second call always fails with error-code 32: ERROR_SHARING_VIOLATION: - The process cannot access the file because it is being used by another process.

Below is an example with 3 threads where one writes a logfile and two try to read them line by line. Does anyone know why the second read-thread fails? It should be possible to read the same file at the same time several times without problems.

When the writer-thread opens the file in modes #FILE_SHARE_WRITE or #FILE_SHARE_READ | #FILE_SHARE_WRITE the file can not be read by any of the two threads only when #FILE_SHARE_READ is used for the writer-thread and then it only works if the reader-threads use #FILE_SHARE_WRITE or #FILE_SHARE_READ | #FILE_SHARE_WRITE, not when using #FILE_SHARE_READ. Strange ...

Thanks

Code: Select all

XIncludeFile "File.pbi" ; https://pb-source-repositery.googlecode.com/svn/trunk/CodeArchiv_v4-Beta/WindowsAPI-Examples/FileHandling_WinAPI.pb

Declare API_FileCreateShared2(*File.API_FileHandle,File$)
Declare API_FileReadShared2(*File.API_FileHandle,File$)
Declare.s ShowError()
  
OpenConsole()

Global file.s = "test.txt"

Procedure WriteMe(number)
  Protected handle.API_FileHandle 
  PrintN("Starting Thread " + Str(number))
  If API_FileCreateShared2(handle,file)
    For i=1 To 40
      API_WriteStringN(handle,Str(i))
      PrintN("Thread " + Str(number) + ": " + Str(i))
      Delay(500)
    Next
  Else
    PrintN("Thread " + Str(number) + ": Can Not write To file: " + ShowError())
  EndIf
EndProcedure  

Procedure ReadMe(number)
  Protected handle.API_FileHandle 
  PrintN("Starting Thread " + Str(number))
  If API_FileReadShared2(handle,file)
    While API_EOF(handle)=#False 
      line.s = API_ReadString(handle)
      PrintN("Thread " + Str(number) + ": " + line.s)
      Delay(1000)
    Wend
    API_CloseFile(handle)
  Else
    PrintN("Thread " + Str(number) + ": Unable to open file: " + ShowError())
  EndIf 
EndProcedure

CreateThread(@WriteMe(), 0)
Delay(1000)
CreateThread(@ReadMe(), 1)
CreateThread(@ReadMe(), 2)
Delay(20000)


Procedure API_FileCreateShared2(*File.API_FileHandle,File$)
  ;*File\FHandle=CreateFile_(File$,#GENERIC_WRITE,#FILE_SHARE_READ Or #FILE_SHARE_WRITE,0,#CREATE_ALWAYS,#FILE_ATTRIBUTE_NORMAL,0)
  *File\FHandle=CreateFile_(File$,#GENERIC_WRITE,#FILE_SHARE_READ,0,#CREATE_ALWAYS,#FILE_ATTRIBUTE_NORMAL,0)
  ;*File\FHandle=CreateFile_(File$,#GENERIC_WRITE,#FILE_SHARE_WRITE,0,#CREATE_ALWAYS,#FILE_ATTRIBUTE_NORMAL,0)
  If *File\FHandle=#INVALID_HANDLE_VALUE
    ProcedureReturn #False
  Else
    ProcedureReturn #True
  EndIf
EndProcedure

Procedure API_FileReadShared2(*File.API_FileHandle,File$)
  *File\FHandle=CreateFile_(File$,#GENERIC_READ,#FILE_SHARE_READ Or #FILE_SHARE_WRITE,0,#OPEN_EXISTING,#FILE_ATTRIBUTE_NORMAL,0)
  ;*File\FHandle=CreateFile_(File$,#GENERIC_READ,#FILE_SHARE_READ,0,#OPEN_EXISTING,#FILE_ATTRIBUTE_NORMAL,0)
  ;*File\FHandle=CreateFile_(File$,#GENERIC_READ,#FILE_SHARE_WRITE,0,#OPEN_EXISTING,#FILE_ATTRIBUTE_NORMAL,0)  
  If *File\FHandle=#INVALID_HANDLE_VALUE
    ProcedureReturn #False
  Else
    *File\Buffer=GlobalAlloc_(#GMEM_FIXED|#GMEM_ZEROINIT,#API_File_BufferLen)
    *File\ReadPos=*File\Buffer
    *File\DataInBuffer=0
    ProcedureReturn #True
  EndIf
EndProcedure

Procedure.s ShowError()
  Protected Error,*MemoryID,e$,Length
  Error = GetLastError_()
  If Error
    *MemoryID = AllocateMemory(255)
    If *MemoryID
      Length = FormatMessage_ (#FORMAT_MESSAGE_FROM_SYSTEM, #Null, Error, 0, *MemoryID, 255, #Null)
      If Length > 1 ; Some error messages are "" + Chr (13) + Chr (10)... stoopid M$... :(
        e$ = PeekS(*MemoryID, Length - 2) + " (" + StrU(Error.i) + ")"
      Else
        e$ = "Unknown error!"
      EndIf
      FreeMemory(*MemoryID)
      ProcedureReturn e$
    Else
      ProcedureReturn "Error while decoding Error("+Str(Error)+")"
    EndIf
  Else
    ProcedureReturn "No error has occurred!"
  EndIf
EndProcedure

[RichAlgeni]

If you are attempting to read and write the same file, you can get rid of all of the Win API code, and replace it with a mutex: Note the following code is 64 bit.

Writing to a file by overwriting the data. The first line must be executed prior to thread creation.

Code: Select all

Global writeLogLock.i = CreateMutex()

Code: Select all

LockMutex(writeLogLock)

logFile = OpenFile(#PB_Any, "logFile.log")

FileSeek(logFile, 0)
TruncateFile(logFile)
WriteStringN(logFile, FormatDate("%yyyy/%mm/%dd %hh:%ii:%ss", Date()) + " " + logText)

FlushFileBuffers(logFile)
CloseFile(logFile)

UnlockMutex(writeLogLock)

Reading the same file from a different thread:

Code: Select all

LockMutex(writeLogLock)

logFile = ReadFile(#PB_Any, "logFile.log")
fileLength = Lof(logFile)

*datLocation = AllocateMemory(fileLength)
*memLocation = *datLocation

totNeed = fileLength
While totNeed > 0
    amtRead = ReadData(logFile, *memLocation, totNeed)
    If  amtRead > 0
        *memLocation = *memLocation + amtRead
        totNeed      = totNeed      - amtRead
    Else
        Break
    EndIf
Wend

CloseFile(logFile)

UnlockMutex(writeLogLock)

FreeMemory(*datLocation)

If you need to read and write to multiple different files, I've come up with a way that seems to be faster than locking each individual file. I create, insert the file name into the map, if it isn't already there. The process uses a Mutex to update the map.

Code: Select all

Global NewMap readFileList.s()
Global readFileLock.i = CreateMutex()

Code: Select all

; lock the map to add our file, so that we do not have file contention

Repeat
    LockMutex(readFileLock)
    If FindMapElement(readFileList(), fileName) > 0
        UnlockMutex(readFileLock)
        Delay(20)
    Else
        AddMapElement(readFileList(), fileName)
        UnlockMutex(readFileLock)
        Break
    EndIf
ForEver

read or write to your file here

Code: Select all

; lock the map again, so that we can remove file we have read in from the list

LockMutex(readFileLock)
If FindMapElement(readFileList(), fileName) > 0
    DeleteMapElement(readFileList(), fileName)
EndIf
UnlockMutex(readFileLock)

[sepp]

Thanks RichAlgeni for your reply.

The manual says that a mutex only works within one programm (but several threads). The code above was just an example, actually I am having 3 seperate applications (1 writer, 2 readers).

You are closing the filehandle before releasing the mutex, that would be something I could use too. But I think that would make things much more complicated, each reader had to open the file, read from it, close it again and again in a very short period. What I wanted to have is an open filehandle in each reader-application and then read from it line by line and only close the filehandle when the application is exited.

[luis]

Code: Select all

#FILE_SHARE_READ | #FILE_SHARE_WRITE

You used logical Or instead of bitwise |

[sepp]

Code: Select all

#FILE_SHARE_READ | #FILE_SHARE_WRITE

You used logical Or instead of bitwise |

BINGO

As #FILE_SHARE_READ=1 and #FILE_SHARE_WRITE=2 the conjunction #FILE_SHARE_READ Or #FILE_SHARE_WRITE = 1 Or 2 = 1 but #FILE_SHARE_READ | #FILE_SHARE_WRITE = 1 | 2 = 3. So it could not work as the Writer-Application needed #FILE_SHARE_WRITE.

Thank you very much!

[luis]

It was my pleasure to help you. You stated your problem clearly and you even wrote a nice little program to show your problem.
Even put the link for the needed include.

Welcome to the forum.

[RichAlgeni]

You're welcome Sepp!

Luis is not only good, he's dang good!

[RichAlgeni]

As is most everyone here!