FileBuffersSize(#File, Size)

[Dr. Dri]

This isn't very handy, i would rather have SetFileBuffersSize(#File, Size) and GetFileBuffersSize(#File). It's not yet possible to know the buffer size of a file. Or maybe i missed something ?

Dri

[Jan Vooijs]

Why do you need the buffer size, most of the time windows and/or linux are more than capable to handle the i/o the buffers are kept in memory so?

Jan V.

[Dr. Dri]

Linux and/or windows is not my problem. As it is possible to set the buffer length, i want to know the buffer length (if any) of a file. If i work with a buffered file, and if i need to change the buffer size, i also need to be able to restore the previous buffer size. Especially for files i didn't create myself...

Dri

[PB]

When you say "for files I didn't create myself" what are you referring to?
And since your app is creating the buffers, how can it not know the size?

[freak]

The buffersize for a file only sets the size of data that PB will read at once from the file and keep in a buffer for you to read.
Changing it changes nothing on the physical file itself.

Lets say you do 100 ReadByte() calls on a file in a row. With a buffersize of 0, that will result
in 100 read-accesses to the file. (though the FileSystem probably has a buffer as well).
Setting the BufferSize to 100 bytes or more will cause PB to read in all at once into memory,
and the ReadByte() calls will just return the data from memory. (which is faster)
The data you get is the same, wether it is buffered or not.

So the command only affects how the file is read/written, not the file itself.
When you open a file, the buffersize is always a fixed value (something like 4096 bytes i think)
When you change it, it changes the behaviour while the file is open. Close it and reopen it and you have the default buffersize again.

Usefull usage of this command are such:

- If you want each write/read to directly reflect what is in the filesystem. (for example so another program can read it),
set the BufferSize to 0.

- If you have many read/write calls of small units (bytes, longs), which you want to be fast,
and don't care about some memory usage: set the BufferSize to a larger value.

- If you have normal reading/writing to do: Something inbetween. The default will mostly do.

You see, the setting that makes sense depends on your program really. It is not a property of the file,
so a command to read the BufferSize is not that usefull really.

[Lebostein]

Usefull usage of this command are such:

- If you want each write/read to directly reflect what is in the filesystem. (for example so another program can read it),
set the BufferSize to 0.

- If you have many read/write calls of small units (bytes, longs), which you want to be fast,
and don't care about some memory usage: set the BufferSize to a larger value.

- If you have normal reading/writing to do: Something inbetween. The default will mostly do.

You see, the setting that makes sense depends on your program really. It is not a property of the file,
so a command to read the BufferSize is not that usefull really.

Add this to the manual, please.

[sverson]

When you open a file, the buffersize is always a fixed value (something like 4096 bytes i think)

Is there a (dirty) way to change this value (like this old workaround for Strings)
...and/or...
Is there a way to avoid getting the data to two places in memory
HDD -> FileBuffer (place1) -> MyDataBuffer (place2)

e.g. When I need to read and check some 1000 files (FileSize 1-3 MB) I would like to set the buffersize to 4MB so ReadData() gets the whole file at once.
But not like ReadData(0,MyDataBuffer,MyFileSize) getting the data from HDD to MyDataBuffer via FileBuffer I would like to avoit the second memory move.

I tried API but it`s not (much) faster. So i think PB could even be better (faster) without the second memory move. ?!?

________________________
...maybe i am just a bloody beginner digging in the mud of impossibilities...

[Rescator]

Is there a way to avoid getting the data to two places in memory
HDD -> FileBuffer (place1) -> MyDataBuffer (place2)

e.g. When I need to read and check some 1000 files (FileSize 1-3 MB) I would like to set the buffersize to 4MB so ReadData() gets the whole file at once.
But not like ReadData(0,MyDataBuffer,MyFileSize) getting the data from HDD to MyDataBuffer via FileBuffer I would like to avoit the second memory move.

Sure! (rough example)

Code: Select all

ReadFile(#File,"blah.bin")
MyDataBuffer=AllocateMemory(Lof(#File))
FileBuffersSize(#File, 0)
ReadData(#File,MyDataBuffer,Lof(#File))

That reads the entire file into your buffer directly in one go,
only limited by the filesystem/diskdrive's hardware/driver buffer.
PB's own file buffer is disabled for #File (hence the 0)

[freak]

> Is there a (dirty) way to change this value

Thats what FileBuffersSize() is good for.

The buffer is there to optimize many smaller accesses. To read a larger buffer (like the whole file at once)
deactivating the buffering by setting the size to 0 of course makes sense.

btw, if you call ReadData() with a larger size than the buffer, the data will also be read directly.
(obviouly using the buffer makes no sense then)

[sverson]

THX Rescator,
THX freak!

@freak:
I was looking for a way to set it once at the beginning of the loop - not file by file...
(one single µsec does not matter but a million of additional µsecs is 1sec which can be a long time...)

btw, if you call ReadData() with a larger size than the buffer, the data will also be read directly.

This may be the reason why there is no time difference between FileBuffersSize(0),FileBuffersSize($100000) and standard. (in my app)

[Rescator]

Hey Freak, any idea if the filebuffer size is tied to the #File id
or if it has to be set (if non default value) each time a new file is opened (but using same file id)

If not then I'd highly advice PB to "remember" the buffer size on per #File id basis instead. As sverson pointed out that would make loops more efficient (one less procedure call

[freak]

The setting is lost when the file is closed.