Question regarding CreateFileMapping_()

[Mistrel]

Would someone explain in what context the parameters dwMaximumSizeHigh and dwMaximumSizeLow should be used?

[PurePWNRER]

What are you, wrapping PB to use in DBP??

[Mistrel]

What are you, wrapping PB to use in DBP??

This has nothing to do with the question.

[srod]

I would use non-zero values only if you wish to limit the size of the mapping objet. That is, if you are dealing with a massive massive file on disc, for example, then you may only want to map the first few megabytes etc. in order to save memory.

Another situation in which you would use these values is if creating a named file mapping object to use, not to map an existing disc file, but to create a chunk of shared memory to use by multiple instances of a dll etc. In these cases you need to specify the size of the object etc.

See the following for an example of using such an object : http://www.purebasic.fr/english/viewtop ... 64&start=0

[blueznl]

two examples, 3.94, I hope they can be usefull to you.

Code: Select all

; purebasic survival guide - pb3.94
; filemapping_1.pb - 06.06.2007 ejn (blueznl)
; http://www.xs4all.nl/~bluez/datatalk/purebasic.htm
;
; - file mapping
; - CreateFile_()
; - CloseHandle_()
; - CreateFileMapping_()
; - MapViewOfFile_()
; - UnMapViewOfFile_()
;
; you DEFINITELY want to have the WIN32.HLP file open and at your side, jump to a page called 'File Mapping' and check on
; the subsequent pages
;
; windows offers a mechanism which maps files to (virtual) memory, whenever you change something in that memory,
; that change will be reflected in the file
;
; a program can have multiple views on a (mapped) file, and their contents will all be consistent when accessed through
; the mapping mechanism, all the (mapped) file data will be in memory thus speeding up access, and finally all data
; in that file can be processed as if it was memory
;
; WARNING! Windows NT and Windows 95 have certain restrictions, that may not be covered below (tested on XP only)
;
;
; *** creating a file using winapi calls... 
;
; (simple winapi stuff, but we need it later, so let's first look at this)
;
; create a file for exclusive use
;
file_name.s = "testfile.dat"                                                            ; name of file to create
file_h = CreateFile_(@file_name,#GENERIC_READ|#GENERIC_WRITE,0,0,#CREATE_ALWAYS,0,0)    ; open the file, empty it if it already existed
If file_h = #INVALID_HANDLE_VALUE Or file_h = 0                                         ; things went bonkers...
  Debug "error creating file"
EndIf
;
; now put some data in it
;
bytes_written.l = 0                                                                     ; placeholder in memory where WriteFile_() can store the number of bytes written
For n = 1 To 10360
  buffer.s = "this is line "+RSet(Str(n),4)+Chr(13)+Chr(10)                             ; create some data that we can write to the file
  result = WriteFile_(file_h,@buffer,Len(buffer),@bytes_written,0)                      ; write the data to the file
Next n
;
; and close the file
;
CloseHandle_(file_h)
;
;
; *** granularity
;
; before we delve deeper into filemapping we have to understand a basic building block: system granularity, or
; allocation granularity: this is the smallest block the memory manager can handle or map, and depends on your
; operating system and hardware
;
; certain calls need this information, which can be obtained this way:
;
systeminfo.SYSTEM_INFO
GetSystemInfo_(@systeminfo)
system_granularity = systeminfo\dwAllocationGranularity
Debug "system granularity "+Str(system_granularity)+" bytes"
;
;
; *** filemapping
;
; mapping a file to memory offers us faster access, windows takes care of all caching
;
; let's start by opening an existing file
;
file_name.s = "testfile.dat"                                                            ; name of file to create
file_h = CreateFile_(@file_name,#GENERIC_READ|#GENERIC_WRITE,0,0,#OPEN_EXISTING,0,0)    ; open an existing file
;
; now we map this file to (a part of) (virtual) memory
; 
; when creating a file mapping object, we can specify the maximal size it can cover, this is NOT the buffer size it
; actually uses, it is the MAXIMAL buffer size it ever uses, by setting it to 0 it will be at max the CURRENT size
; of the specified file, check also WIN32.HLP for limitations of NT and 95...
; 
; notice that CreateFile_(), CreateFileMapping_() and MapViewOfFile_() must have matching parameters
;
filemap_name.s = "testmap.dat"                                                          ; we give the mapping a name, though there isn't a real need here
filemap_h = CreateFileMapping_(file_h,0,#PAGE_READWRITE,0,0,@filemap_name)              ; this creates an object that can map memory to the file
;
; with MapViewOfFile_() we specify a 'floating window' that peeks somewhere into the specified file, we can specify where
; this 'view' looks at, and how large it is
;
;   MapViewOfFile_( <object_handle> , <access type> , <offset hi> , <offset lo> , <view size> )
;
; the pointers is 64 bit value, split up over a low and high part, and must be multiples of the allocation granularity
; (aka system granularity)
;
mapview_p = MapViewOfFile_(filemap_h,#FILE_MAP_WRITE,0,0,system_granularity)            ; creates a map which points to the beginning of the file
;
; the returned value is a pointer, a memory address where we can find the mapped data, let's find out what the first 100 bytes
; of the file contained
;
Debug PeekS(mapview_p,100)                                                              ; show 100 bytes
UnmapViewOfFile_(mapview_p)                                                             ; and be done
;
; unfortunately there is no way to 'move' a view, we first have to delete the old one and create a new one
; we have to be carefull that we won't 'overrun' the file, ie. the map should stay within the boundaries of the file
;
; in other words, we 'carve up' the file in multiple blocks of size 'system_granularity', ie. the specified offset
; must be a mutliple of system_granularity
;
; we must also make sure that the last byte of the view stays within the size specified with CreateFileMapping_(),
; this may mean we have to make the view smaller than system_granularity when reaching the end of the file
;
; let's do some fancy graphics :-) the '-' indicate the file, the 'x' indicate the view
;
;   ----- xxxxx ---
;   ----- ----- xxx
; 
; note: whilst sizes specified with CreateFileMapping_() MAY allocate PHYSICAL memory up to the specified size, Windows
; will not keep the data at any cost in memory, and may write it out to the file itself, the view size specified with
; MapViefOfFile_() WILL most likely take up physical memory and certainly take up virtual memory, and is limited by
; the specified virtual memory settings
;
; super large view sizes may slow down your system and may cause problems, yet sizes too small will decrease the
; efficiency of file mapping... you decide what is an appropriate size :-)
;
position = FileSize(file_name)-200                                                      ; let's move to the end of the file
mapoffset = position - (position % system_granularity)                                  ; make sure we start on a multiple of system granularity
viewoffset = position - mapoffset                                                       ; where can we find the requested data in the view?
viewsize = FileSize(file_name) - mapoffset                                              ; how much file is left?
If viewsize > system_granularity                                                        ; please, fred, give us Min() and Max()
  viewsize = system_granularity                                                         
EndIf
mapview_p = MapViewOfFile_(filemap_h,#FILE_MAP_WRITE,0,mapoffset,viewsize)
Debug "view points to position "+Str(mapoffset)+" in file"
Debug "view size "+Str(viewsize)
Debug "requested data starts in file at "+Str(position)
Debug "and in memory (view) at "+Str(viewoffset)
Debug PeekS(mapview_p+viewoffset,100)
;
; ok, we're done, time to clean up
;
UnmapViewOfFile_(mapview_p)
CloseHandle_(filemap_h)                     
CloseHandle_(file_h)
;
; and to verify the result
;
OpenFile(1,file_name)
FileSeek(position)
buffer.s = Space(100)
ReadData(@buffer,100)
Debug buffer
CloseFile(1)
;

and example 2

Code: Select all

; purebasic survival guide - pb3.94
; filemapping_2.pb - 07.06.2007 ejn (blueznl)
; http://www.xs4all.nl/~bluez/datatalk/purebasic.htm
;
; - file mapping in memory (sharing memory between applications)
; - CloseHandle_()
; - CreateFileMapping_()
; - MapViewOfFile_()
; - UnMapViewOfFile_()
;
; you DEFINITELY want to have the WIN32.HLP file open and at your side, jump to a page called 'File Mapping' and check on
; the subsequent pages
;
; check out filemapping_1.pb first, as it explains some of the API calls made here
;
; WARNING! Windows NT and Windows 95 have certain restrictions, that may not be covered below (tested on XP only)
;
;
; it is possible for different applications to share a mapped file, and access that data simultaneously
; interestingly it doesn't have to be a real file but could be just a chunk of memory!
;
;   CreateFileMapping_( <filehandle> , <security attribs> , <protection> , <hi size> , <lo size> , <name> )
;
; by specifying $FFFFFFFF instead of a real handle a block of (virtual) memory is allocated, but there is no associated
; file for it, a name is optional BUT by specifying a name MULTIPLE PROGRAMS CAN ACCESS THE SAME DATA IN MEMORY...
; ...now that's VERY interesting :-)
;
; each program that wants to access that block of data can try to create a filemap using the filemap name
; GetLastError_() can be used to check if the filemapping object already existed or not
;
; let's create a filemap in memory of 1 kilobyte (actually, it would make more sense to use the system granularity
; as that is what's going to be in use anyway, but who cares...)
;
filemap_name.s = "shared_memory"
filemap_h = CreateFileMapping_($FFFFFFFF,0,#PAGE_READWRITE,0,1024,@filemap_name)
lasterror = GetLastError_()
If filemap_h = #INVALID_HANDLE_VALUE Or filemap_h = 0
  Debug "error creating file"
EndIf
If lasterror = #ERROR_ALREADY_EXISTS
  Debug "object already exists"
EndIf
;
; multiple programs can open up the same mappingfile object by using the name, let's open it up for the second time
; CreateFileMapping_() will still return a valid handle, even if it already existed, so we end up with two handles to the
; same object, as we're just fooling around let's close that second one after we found out it already existed :-)
;
filemap2_name.s = "shared_memory"
filemap2_h = CreateFileMapping_($FFFFFFFF,0,#PAGE_READWRITE,0,1024,@filemap2_name)
lasterror = GetLastError_()
If filemap2_h = #INVALID_HANDLE_VALUE Or filemap2_h = 0
  Debug "error creating file"
EndIf
If lasterror = #ERROR_ALREADY_EXISTS
  Debug "object already exists"
  CloseHandle_(filemap2_h)
EndIf
;
; object is there, let's map a view to it
;
mapview_p = MapViewOfFile_(filemap_h,#FILE_MAP_WRITE,0,0,0)
;
; so now we can access 1024 bytes, starting at mapview_p, which we can share with other programs as long as they open
; the same block of memory using CreateFileMapping_()
;
PokeS(mapview_p,"this is a test")
;
; please note: there is no inherent synchronisation mechanism for accessing filemapping objects, for that you have to use
; a mutex (see the examples in mutex_1.pb) or messages (check RegisterWindowMessage_())
;
Debug PeekS(mapview_p)
;
; we're done, so we unmap the view, all data in the filemapping objects stays in there as long as ANY program has that
; object open, but the moment all clients close it... poof
;
CloseHandle_(filemap_h)
;
; next stop: an example of two communicating processes!
;

[PurePWNRER]

What are you, wrapping PB to use in DBP??

This has nothing to do with the question.

Yet it has to do with the license you agreed upon buying purebasic.

[Mistrel]

What are you, wrapping PB to use in DBP??

This has nothing to do with the question.

Yet it has to do with the license you agreed upon buying purebasic.

I read and understand the the license. Do you?

If you don't have anything constructive to add please don't reply to this topic.

[Hi-Toro]

If you're just reading a file, use 0, 0. This lets you 'view' the whole file (so no need to specify specific portions of the file), and the MapViewOfFile_ call is where you decide how much of the file to read at a time (so you're only ever reading, say, 64K on each call, rather than several gigabytes!).

I did a load of tests, and 64K was the most efficient; which turned out to be exactly the result obtained from info\dwAllocationGranularity (so you should use this rather than hard-coding 64K).

Note that this code is specifically intended for READING files, but hopefully will show how to use the filemapping commands...

Code: Select all

Procedure ReadMappedFile (f$)
    
    ; Used to pass 64-bit number to MapViewOfFile in two parts...
    
    Structure HiLo
        hi.l
        lo.l
    EndStructure

    ; Filemapping REQUIRES use of system's most efficient memory chunk size...
    
    GetSystemInfo_ (info.SYSTEM_INFO)
    chunk = info\dwAllocationGranularity ; Usually 64K / 65536
    
    view.HiLo

    ; How many times to read 'chunk' bytes in For/Next loop...
    
    fsize.q = FileSize (f$)
    loops.q = fsize / chunk
    remainder = fsize % chunk

    If remainder
        loops = loops + 1 ; Extra loop for remainder if less than 'chunk' bytes
    EndIf
    
    ; Get file handle...
    
    file = CreateFile_ (@f$, #GENERIC_READ, 0, #Null, #OPEN_EXISTING, #FILE_ATTRIBUTE_NORMAL | #FILE_FLAG_SEQUENTIAL_SCAN, #Null)
    
    If (file <> #INVALID_HANDLE_VALUE)
    
        ; Create file map, defaulting to the whole file size...
            
        mapped = CreateFileMapping_ (file, #Null, #PAGE_READONLY, 0, 0, #Null)
    
        If mapped

            ; Read 'chunk' bytes (65536 generally) however many times are needed...
            
            For byte = 0 To loops - 1
            
                ; Fill HiLo structure with current filemap offset...
                
                PokeQ (@view, byte * chunk)
                
                ; Hack for remainder on last loop if smaller than 'chunk' bytes...
                
                mapbytes = chunk
                If byte = loops - 1
                    If remainder
                        mapbytes = remainder
                    EndIf
                EndIf
                
                ; Position 'chunk' sized view into filemap...
                
                mapview = MapViewOfFile_ (mapped, #FILE_MAP_READ, view\lo, view\hi, mapbytes)
                
                If mapview

                    ; ------------------------------------------------------
                    ; Process 'mapbytes' bytes from mapview address here, eg...
                    ; ------------------------------------------------------
                    
                            For offset = 0 To mapbytes - 1
                                asc = PeekB (mapview + offset)
                                ; Debug Chr (asc)
                            Next

                    ; ------------------------------------------------------
                    ; End of mapview processing
                    ; ------------------------------------------------------

                    ; Free map view...
                    
                    UnmapViewOfFile_ (mapview)
                    
                EndIf
                            
            Next
    
            ; Close filemapping handle...
            
            CloseHandle_ (mapped)
            
        EndIf
            
        ; Close file handle...
            
        CloseHandle_ (file)
        
    EndIf
    
EndProcedure

; ReadMappedFile ("test.txt")