Performing CRC32 on big files

Justin · Post by **Justin** » Mon Jun 09, 2003 4:27 pm

How can i perform a CRC32 on a file that is bigger than my aviable free disk space? , i can't load it into a buffer. It has to be some way because i have a program that does it, any idea?.

dmoc · Post by **dmoc** » Mon Jun 09, 2003 4:37 pm

I hope you mean "memory"? If so, from memory (mine, not the computers), I think determining the crc involves serial processing the data from start to end, ie, at no time refering to data already processed. Therefore I think you need to check how to combine crc from successive blocks of data. Sorry I don't have time to search the web for more info... but you could

Post by **freak** » Mon Jun 09, 2003 4:38 pm

You should take a look at Rings's FastFile Lib, it has a command calculating
a CRC32 of a file on disk.

http://www.reelmediaproductions.com/pb/ ... stFile.zip

look at the FastFileCRC32Test.pb file.

Timo

Justin · Post by **Justin** » Mon Jun 09, 2003 5:25 pm

The FASTFILECRC32() does it

, i'll make some tests but this appears to be an excellent lib.

Thanks

GPI · Post by **GPI** » Mon Jun 09, 2003 7:33 pm

Justin wrote:How can i perform a CRC32 on a file that is bigger than my aviable free disk space? , i can't load it into a buffer. It has to be some way because i have a program that does it, any idea?.

Make for, example, for every 64Kb a own CRC.

Justin · Post by **Justin** » Mon Jun 09, 2003 9:30 pm

And how do you get the CRC of the whole file from that? , anyways FASTFILECRC32() does it, it can't be easier.

Post by **Rings** » Tue Jun 10, 2003 8:49 am

Justin wrote:And how do you get the CRC of the whole file from that? , anyways FASTFILECRC32() does it, it can't be easier.

but remember that calculating a crc from a 640MB ISO-file can take a lot of seconds....

Justin · Post by **Justin** » Tue Jun 10, 2003 10:36 am

Yes, i made a crc of a cd and took some time, but this is normal. Do you plan to make it linux compatible? , also working with handles to have unlimited files would be great.

hfile1=FastOpenFile(Filename.s)
hfile2=FastOpenFile(Filename.s)
.
.
FastReadData(hfile1,Offset.l,Length.l,AdressofDestination)

etc.

Post by **Rings** » Tue Jun 10, 2003 12:34 pm

Justin wrote:Yes, i made a crc of a cd and took some time, but this is normal. Do you plan to make it linux compatible? , also working with handles to have unlimited files would be great.

hfile1=FastOpenFile(Filename.s)
hfile2=FastOpenFile(Filename.s)
.
.
FastReadData(hfile1,Offset.l,Length.l,AdressofDestination)

etc.

thanx for your interrest in this library.
Yes , multiply files (With Filepointer) are in progress.
No for the linux version, coz i'm not familar enough (or better say Not ) with the linux api.

PureBasic Forums - English

Performing CRC32 on big files

Performing CRC32 on big files

Re: Performing CRC32 on big files