Simple CRC64 Routine

moogle · Post by **moogle** » Mon Aug 15, 2011 1:12 pm

For anyone that would like to use CRC64. It's not optimized so not as fast as it could be. Borrowed SHR64 from netmaestro's thread (think it's Danilo's function so thanks).
Ascii mode compile.

Code: Select all

Procedure.q SHR64(val.q, n)
  If val & (1<<63)  ; sign bit is set, normal shift will fail
    val &~ (1<<63)  ; clear the sign bit
    val>>n          ; shift will work normally now
    val | 1<<(63-n) ; set the original sign bit at its new place
  Else
    val>>n ; nothing needed
  EndIf
	ProcedureReturn val
EndProcedure


Macro CRC64TableFill()
	Dim CRC64Table.q(255)
	CRC64Table(0)  =$0000000000000000:CRC64Table(1)  =$1B00000000000000:CRC64Table(2)  =$3600000000000000:CRC64Table(3)  =$2D00000000000000
	CRC64Table(4)  =$6C00000000000000:CRC64Table(5)  =$7700000000000000:CRC64Table(6)  =$5A00000000000000:CRC64Table(7)  =$4100000000000000
	CRC64Table(8)  =$D800000000000000:CRC64Table(9)  =$C300000000000000:CRC64Table(10) =$EE00000000000000:CRC64Table(11) =$F500000000000000
	CRC64Table(12) =$B400000000000000:CRC64Table(13) =$AF00000000000000:CRC64Table(14) =$8200000000000000:CRC64Table(15) =$9900000000000000
	CRC64Table(16) =$1B00000000000000:CRC64Table(17) =$1AB0000000000000:CRC64Table(18) =$1860000000000000:CRC64Table(19) =$19D0000000000000
	CRC64Table(20) =$1DC0000000000000:CRC64Table(21) =$1C70000000000000:CRC64Table(22) =$1EA0000000000000:CRC64Table(23) =$1F10000000000000
	CRC64Table(24) =$1680000000000000:CRC64Table(25) =$1730000000000000:CRC64Table(26) =$15E0000000000000:CRC64Table(27) =$1450000000000000
	CRC64Table(28) =$1040000000000000:CRC64Table(29) =$11F0000000000000:CRC64Table(30) =$1320000000000000:CRC64Table(31) =$1290000000000000
	CRC64Table(32) =$3600000000000000:CRC64Table(33) =$37B0000000000000:CRC64Table(34) =$3560000000000000:CRC64Table(35) =$34D0000000000000
	CRC64Table(36) =$30C0000000000000:CRC64Table(37) =$3170000000000000:CRC64Table(38) =$33A0000000000000:CRC64Table(39) =$3210000000000000
	CRC64Table(40) =$3B80000000000000:CRC64Table(41) =$3A30000000000000:CRC64Table(42) =$38E0000000000000:CRC64Table(43) =$3950000000000000
	CRC64Table(44) =$3D40000000000000:CRC64Table(45) =$3CF0000000000000:CRC64Table(46) =$3E20000000000000:CRC64Table(47) =$3F90000000000000
	CRC64Table(48) =$2D00000000000000:CRC64Table(49) =$2CB0000000000000:CRC64Table(50) =$2E60000000000000:CRC64Table(51) =$2FD0000000000000
	CRC64Table(52) =$2BC0000000000000:CRC64Table(53) =$2A70000000000000:CRC64Table(54) =$28A0000000000000:CRC64Table(55) =$2910000000000000
	CRC64Table(56) =$2080000000000000:CRC64Table(57) =$2130000000000000:CRC64Table(58) =$23E0000000000000:CRC64Table(59) =$2250000000000000
	CRC64Table(60) =$2640000000000000:CRC64Table(61) =$27F0000000000000:CRC64Table(62) =$2520000000000000:CRC64Table(63) =$2490000000000000
	CRC64Table(64) =$6C00000000000000:CRC64Table(65) =$6DB0000000000000:CRC64Table(66) =$6F60000000000000:CRC64Table(67) =$6ED0000000000000
	CRC64Table(68) =$6AC0000000000000:CRC64Table(69) =$6B70000000000000:CRC64Table(70) =$69A0000000000000:CRC64Table(71) =$6810000000000000
	CRC64Table(72) =$6180000000000000:CRC64Table(73) =$6030000000000000:CRC64Table(74) =$62E0000000000000:CRC64Table(75) =$6350000000000000
	CRC64Table(76) =$6740000000000000:CRC64Table(77) =$66F0000000000000:CRC64Table(78) =$6420000000000000:CRC64Table(79) =$6590000000000000
	CRC64Table(80) =$7700000000000000:CRC64Table(81) =$76B0000000000000:CRC64Table(82) =$7460000000000000:CRC64Table(83) =$75D0000000000000
	CRC64Table(84) =$71C0000000000000:CRC64Table(85) =$7070000000000000:CRC64Table(86) =$72A0000000000000:CRC64Table(87) =$7310000000000000
	CRC64Table(88) =$7A80000000000000:CRC64Table(89) =$7B30000000000000:CRC64Table(90) =$79E0000000000000:CRC64Table(91) =$7850000000000000
	CRC64Table(92) =$7C40000000000000:CRC64Table(93) =$7DF0000000000000:CRC64Table(94) =$7F20000000000000:CRC64Table(95) =$7E90000000000000
	CRC64Table(96) =$5A00000000000000:CRC64Table(97) =$5BB0000000000000:CRC64Table(98) =$5960000000000000:CRC64Table(99) =$58D0000000000000
	CRC64Table(100)=$5CC0000000000000:CRC64Table(101)=$5D70000000000000:CRC64Table(102)=$5FA0000000000000:CRC64Table(103)=$5E10000000000000
	CRC64Table(104)=$5780000000000000:CRC64Table(105)=$5630000000000000:CRC64Table(106)=$54E0000000000000:CRC64Table(107)=$5550000000000000
	CRC64Table(108)=$5140000000000000:CRC64Table(109)=$50F0000000000000:CRC64Table(110)=$5220000000000000:CRC64Table(111)=$5390000000000000
	CRC64Table(112)=$4100000000000000:CRC64Table(113)=$40B0000000000000:CRC64Table(114)=$4260000000000000:CRC64Table(115)=$43D0000000000000
	CRC64Table(116)=$47C0000000000000:CRC64Table(117)=$4670000000000000:CRC64Table(118)=$44A0000000000000:CRC64Table(119)=$4510000000000000
	CRC64Table(120)=$4C80000000000000:CRC64Table(121)=$4D30000000000000:CRC64Table(122)=$4FE0000000000000:CRC64Table(123)=$4E50000000000000
	CRC64Table(124)=$4A40000000000000:CRC64Table(125)=$4BF0000000000000:CRC64Table(126)=$4920000000000000:CRC64Table(127)=$4890000000000000
	CRC64Table(128)=$D800000000000000:CRC64Table(129)=$D9B0000000000000:CRC64Table(130)=$DB60000000000000:CRC64Table(131)=$DAD0000000000000
	CRC64Table(132)=$DEC0000000000000:CRC64Table(133)=$DF70000000000000:CRC64Table(134)=$DDA0000000000000:CRC64Table(135)=$DC10000000000000
	CRC64Table(136)=$D580000000000000:CRC64Table(137)=$D430000000000000:CRC64Table(138)=$D6E0000000000000:CRC64Table(139)=$D750000000000000
	CRC64Table(140)=$D340000000000000:CRC64Table(141)=$D2F0000000000000:CRC64Table(142)=$D020000000000000:CRC64Table(143)=$D190000000000000
	CRC64Table(144)=$C300000000000000:CRC64Table(145)=$C2B0000000000000:CRC64Table(146)=$C060000000000000:CRC64Table(147)=$C1D0000000000000
	CRC64Table(148)=$C5C0000000000000:CRC64Table(149)=$C470000000000000:CRC64Table(150)=$C6A0000000000000:CRC64Table(151)=$C710000000000000
	CRC64Table(152)=$CE80000000000000:CRC64Table(153)=$CF30000000000000:CRC64Table(154)=$CDE0000000000000:CRC64Table(155)=$CC50000000000000
	CRC64Table(156)=$C840000000000000:CRC64Table(157)=$C9F0000000000000:CRC64Table(158)=$CB20000000000000:CRC64Table(159)=$CA90000000000000
	CRC64Table(160)=$EE00000000000000:CRC64Table(161)=$EFB0000000000000:CRC64Table(162)=$ED60000000000000:CRC64Table(163)=$ECD0000000000000
	CRC64Table(164)=$E8C0000000000000:CRC64Table(165)=$E970000000000000:CRC64Table(166)=$EBA0000000000000:CRC64Table(167)=$EA10000000000000
	CRC64Table(168)=$E380000000000000:CRC64Table(169)=$E230000000000000:CRC64Table(170)=$E0E0000000000000:CRC64Table(171)=$E150000000000000
	CRC64Table(172)=$E540000000000000:CRC64Table(173)=$E4F0000000000000:CRC64Table(174)=$E620000000000000:CRC64Table(175)=$E790000000000000
	CRC64Table(176)=$F500000000000000:CRC64Table(177)=$F4B0000000000000:CRC64Table(178)=$F660000000000000:CRC64Table(179)=$F7D0000000000000
	CRC64Table(180)=$F3C0000000000000:CRC64Table(181)=$F270000000000000:CRC64Table(182)=$F0A0000000000000:CRC64Table(183)=$F110000000000000
	CRC64Table(184)=$F880000000000000:CRC64Table(185)=$F930000000000000:CRC64Table(186)=$FBE0000000000000:CRC64Table(187)=$FA50000000000000
	CRC64Table(188)=$FE40000000000000:CRC64Table(189)=$FFF0000000000000:CRC64Table(190)=$FD20000000000000:CRC64Table(191)=$FC90000000000000
	CRC64Table(192)=$B400000000000000:CRC64Table(193)=$B5B0000000000000:CRC64Table(194)=$B760000000000000:CRC64Table(195)=$B6D0000000000000
	CRC64Table(196)=$B2C0000000000000:CRC64Table(197)=$B370000000000000:CRC64Table(198)=$B1A0000000000000:CRC64Table(199)=$B010000000000000
	CRC64Table(200)=$B980000000000000:CRC64Table(201)=$B830000000000000:CRC64Table(202)=$BAE0000000000000:CRC64Table(203)=$BB50000000000000
	CRC64Table(204)=$BF40000000000000:CRC64Table(205)=$BEF0000000000000:CRC64Table(206)=$BC20000000000000:CRC64Table(207)=$BD90000000000000
	CRC64Table(208)=$AF00000000000000:CRC64Table(209)=$AEB0000000000000:CRC64Table(210)=$AC60000000000000:CRC64Table(211)=$ADD0000000000000
	CRC64Table(212)=$A9C0000000000000:CRC64Table(213)=$A870000000000000:CRC64Table(214)=$AAA0000000000000:CRC64Table(215)=$AB10000000000000
	CRC64Table(216)=$A280000000000000:CRC64Table(217)=$A330000000000000:CRC64Table(218)=$A1E0000000000000:CRC64Table(219)=$A050000000000000
	CRC64Table(220)=$A440000000000000:CRC64Table(221)=$A5F0000000000000:CRC64Table(222)=$A720000000000000:CRC64Table(223)=$A690000000000000
	CRC64Table(224)=$8200000000000000:CRC64Table(225)=$83B0000000000000:CRC64Table(226)=$8160000000000000:CRC64Table(227)=$80D0000000000000
	CRC64Table(228)=$84C0000000000000:CRC64Table(229)=$8570000000000000:CRC64Table(230)=$87A0000000000000:CRC64Table(231)=$8610000000000000
	CRC64Table(232)=$8F80000000000000:CRC64Table(233)=$8E30000000000000:CRC64Table(234)=$8CE0000000000000:CRC64Table(235)=$8D50000000000000
	CRC64Table(236)=$8940000000000000:CRC64Table(237)=$88F0000000000000:CRC64Table(238)=$8A20000000000000:CRC64Table(239)=$8B90000000000000
	CRC64Table(240)=$9900000000000000:CRC64Table(241)=$98B0000000000000:CRC64Table(242)=$9A60000000000000:CRC64Table(243)=$9BD0000000000000
	CRC64Table(244)=$9FC0000000000000:CRC64Table(245)=$9E70000000000000:CRC64Table(246)=$9CA0000000000000:CRC64Table(247)=$9D10000000000000
	CRC64Table(248)=$9480000000000000:CRC64Table(249)=$9530000000000000:CRC64Table(250)=$97E0000000000000:CRC64Table(251)=$9650000000000000
	CRC64Table(252)=$9240000000000000:CRC64Table(253)=$93F0000000000000:CRC64Table(254)=$9120000000000000:CRC64Table(255)=$9090000000000000
EndMacro

Macro CRC64StringFingerprint(text)
 CRC64Fingerprint(@text,StringByteLength(text))
EndMacro

Macro CRC32StringFingerprint(text)
 CRC32Fingerprint(@text,StringByteLength(text))
EndMacro

Procedure.q CRC64Fingerprint(*Buffer, Size.l, InitialValue.q=0)
	Protected CRC64TableFill()
	Protected.q crc64=InitialValue, i.l, byte.a, index.a
	For i=1 To Size
		byte=PeekB(*Buffer+(i-1))
		index =  ((crc64 ! byte) & $FF);
		crc64 = CRC64Table(index) ! SHR64(crc64,8);
	Next
	ProcedureReturn crc64;
EndProcedure

string.s = "123456789"; // CRC64 = 46A5A9388A5BEFFE
length=StringByteLength(string)

;1 step calculation
hash0.q=CRC64Fingerprint(@string, length)

;2 step calculation
part1.q=CRC64Fingerprint(@string,length/2)
hash1.q=CRC64Fingerprint(@string+length/2, length/2+1, part1)

Debug Hex(hash0,#PB_Quad)
Debug Hex(hash1,#PB_Quad)

Guimauve · Post by **Guimauve** » Mon Feb 20, 2012 2:43 pm

Hello everyone,

Sorry to re-up this old topic but I have made some change to the code provided by moogle. I have manage to understand why the twin step CRC32 (standard command) and CRC64 return different value in Ascii and Unicode mode. Apparently, we need to calculate the part memory length differently in Ascii and Unicode mode, see the demonstration code.

Best regards
Guimauve

Code: Select all

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Project name : CRC64Fingerprint
; File Name : CRC64Fingerprint.pb
; File version: 1.1.0
; Programming : OK
; Programmed by : moogle (English forum)
; Modified by : Guimauve
; Date : 15-08-2011
; Last Update : 20-02-2012
; PureBasic code : 4.61
; Platform : Windows, Linux, MacOS X
; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Procedure.q SHR64(val.q, n)
  
  If val & (1 << 63)  ; sign bit is set, normal shift will fail
    val &~ (1 << 63)  ; clear the sign bit
    val >> n          ; shift will work normally now
    val | 1 << (63 - n) ; set the original sign bit at its new place
  Else
    val >> n ; nothing needed
  EndIf
  
  ProcedureReturn val
EndProcedure

Structure CRC64Table
  
  Element.q[256]
  
EndStructure 

Macro CRC64Table(CRC64TableA, ElementID)
  
  CRC64TableA\Element[ElementID]
  
EndMacro

Macro InitializeCRC64Table(CRC64TableA)
  
  CRC64Table(CRC64TableA, 000) = $0000000000000000 : CRC64Table(CRC64TableA, 001) = $1B00000000000000 : CRC64Table(CRC64TableA, 002) = $3600000000000000 
  CRC64Table(CRC64TableA, 003) = $2D00000000000000 : CRC64Table(CRC64TableA, 004) = $6C00000000000000 : CRC64Table(CRC64TableA, 005) = $7700000000000000
  CRC64Table(CRC64TableA, 006) = $5A00000000000000 : CRC64Table(CRC64TableA, 007) = $4100000000000000 : CRC64Table(CRC64TableA, 008) = $D800000000000000 
  CRC64Table(CRC64TableA, 009) = $C300000000000000 : CRC64Table(CRC64TableA, 010) = $EE00000000000000 : CRC64Table(CRC64TableA, 011) = $F500000000000000
  CRC64Table(CRC64TableA, 012) = $B400000000000000 : CRC64Table(CRC64TableA, 013) = $AF00000000000000 : CRC64Table(CRC64TableA, 014) = $8200000000000000 
  CRC64Table(CRC64TableA, 015) = $9900000000000000 : CRC64Table(CRC64TableA, 016) = $1B00000000000000 : CRC64Table(CRC64TableA, 017) = $1AB0000000000000
  CRC64Table(CRC64TableA, 018) = $1860000000000000 : CRC64Table(CRC64TableA, 019) = $19D0000000000000 : CRC64Table(CRC64TableA, 020) = $1DC0000000000000 
  CRC64Table(CRC64TableA, 021) = $1C70000000000000 : CRC64Table(CRC64TableA, 022) = $1EA0000000000000 : CRC64Table(CRC64TableA, 023) = $1F10000000000000
  CRC64Table(CRC64TableA, 024) = $1680000000000000 : CRC64Table(CRC64TableA, 025) = $1730000000000000 : CRC64Table(CRC64TableA, 026) = $15E0000000000000
  CRC64Table(CRC64TableA, 027) = $1450000000000000 : CRC64Table(CRC64TableA, 028) = $1040000000000000 : CRC64Table(CRC64TableA, 029) = $11F0000000000000
  CRC64Table(CRC64TableA, 030) = $1320000000000000 : CRC64Table(CRC64TableA, 031) = $1290000000000000 : CRC64Table(CRC64TableA, 032) = $3600000000000000
  CRC64Table(CRC64TableA, 033) = $37B0000000000000 : CRC64Table(CRC64TableA, 034) = $3560000000000000 : CRC64Table(CRC64TableA, 035) = $34D0000000000000
  CRC64Table(CRC64TableA, 036) = $30C0000000000000 : CRC64Table(CRC64TableA, 037) = $3170000000000000 : CRC64Table(CRC64TableA, 038) = $33A0000000000000
  CRC64Table(CRC64TableA, 039) = $3210000000000000 : CRC64Table(CRC64TableA, 040) = $3B80000000000000 : CRC64Table(CRC64TableA, 041) = $3A30000000000000
  CRC64Table(CRC64TableA, 042) = $38E0000000000000 : CRC64Table(CRC64TableA, 043) = $3950000000000000 : CRC64Table(CRC64TableA, 044) = $3D40000000000000
  CRC64Table(CRC64TableA, 045) = $3CF0000000000000 : CRC64Table(CRC64TableA, 046) = $3E20000000000000 : CRC64Table(CRC64TableA, 047) = $3F90000000000000
  CRC64Table(CRC64TableA, 048) = $2D00000000000000 : CRC64Table(CRC64TableA, 049) = $2CB0000000000000 : CRC64Table(CRC64TableA, 050) = $2E60000000000000 
  CRC64Table(CRC64TableA, 051) = $2FD0000000000000 : CRC64Table(CRC64TableA, 052) = $2BC0000000000000 : CRC64Table(CRC64TableA, 053) = $2A70000000000000
  CRC64Table(CRC64TableA, 054) = $28A0000000000000 : CRC64Table(CRC64TableA, 055) = $2910000000000000 : CRC64Table(CRC64TableA, 056) = $2080000000000000 
  CRC64Table(CRC64TableA, 057) = $2130000000000000 : CRC64Table(CRC64TableA, 058) = $23E0000000000000 : CRC64Table(CRC64TableA, 059) = $2250000000000000
  CRC64Table(CRC64TableA, 060) = $2640000000000000 : CRC64Table(CRC64TableA, 061) = $27F0000000000000 : CRC64Table(CRC64TableA, 062) = $2520000000000000
  CRC64Table(CRC64TableA, 063) = $2490000000000000 : CRC64Table(CRC64TableA, 064) = $6C00000000000000 : CRC64Table(CRC64TableA, 065) = $6DB0000000000000
  CRC64Table(CRC64TableA, 066) = $6F60000000000000 : CRC64Table(CRC64TableA, 067) = $6ED0000000000000 : CRC64Table(CRC64TableA, 068) = $6AC0000000000000   
  CRC64Table(CRC64TableA, 069) = $6B70000000000000 : CRC64Table(CRC64TableA, 070) = $69A0000000000000 : CRC64Table(CRC64TableA, 071) = $6810000000000000
  CRC64Table(CRC64TableA, 072) = $6180000000000000 : CRC64Table(CRC64TableA, 073) = $6030000000000000 : CRC64Table(CRC64TableA, 074) = $62E0000000000000
  CRC64Table(CRC64TableA, 075) = $6350000000000000 : CRC64Table(CRC64TableA, 076) = $6740000000000000 : CRC64Table(CRC64TableA, 077) = $66F0000000000000
  CRC64Table(CRC64TableA, 078) = $6420000000000000 : CRC64Table(CRC64TableA, 079) = $6590000000000000 : CRC64Table(CRC64TableA, 080) = $7700000000000000 
  CRC64Table(CRC64TableA, 081) = $76B0000000000000 : CRC64Table(CRC64TableA, 082) = $7460000000000000 : CRC64Table(CRC64TableA, 083) = $75D0000000000000
  CRC64Table(CRC64TableA, 084) = $71C0000000000000 : CRC64Table(CRC64TableA, 085) = $7070000000000000 : CRC64Table(CRC64TableA, 086) = $72A0000000000000 
  CRC64Table(CRC64TableA, 087) = $7310000000000000 : CRC64Table(CRC64TableA, 088) = $7A80000000000000 : CRC64Table(CRC64TableA, 089) = $7B30000000000000
  CRC64Table(CRC64TableA, 090) = $79E0000000000000 : CRC64Table(CRC64TableA, 091) = $7850000000000000 : CRC64Table(CRC64TableA, 092) = $7C40000000000000
  CRC64Table(CRC64TableA, 093) = $7DF0000000000000 : CRC64Table(CRC64TableA, 094) = $7F20000000000000 : CRC64Table(CRC64TableA, 095) = $7E90000000000000
  CRC64Table(CRC64TableA, 096) = $5A00000000000000 : CRC64Table(CRC64TableA, 097) = $5BB0000000000000 : CRC64Table(CRC64TableA, 098) = $5960000000000000
  CRC64Table(CRC64TableA, 099) = $58D0000000000000 : CRC64Table(CRC64TableA, 100) = $5CC0000000000000 : CRC64Table(CRC64TableA, 101) = $5D70000000000000
  CRC64Table(CRC64TableA, 102) = $5FA0000000000000 : CRC64Table(CRC64TableA, 103) = $5E10000000000000 : CRC64Table(CRC64TableA, 104) = $5780000000000000 
  CRC64Table(CRC64TableA, 105) = $5630000000000000 : CRC64Table(CRC64TableA, 106) = $54E0000000000000 : CRC64Table(CRC64TableA, 107) = $5550000000000000
  CRC64Table(CRC64TableA, 108) = $5140000000000000 : CRC64Table(CRC64TableA, 109) = $50F0000000000000 : CRC64Table(CRC64TableA, 110) = $5220000000000000
  CRC64Table(CRC64TableA, 111) = $5390000000000000 : CRC64Table(CRC64TableA, 112) = $4100000000000000 : CRC64Table(CRC64TableA, 113) = $40B0000000000000
  CRC64Table(CRC64TableA, 114) = $4260000000000000 : CRC64Table(CRC64TableA, 115) = $43D0000000000000 : CRC64Table(CRC64TableA, 116) = $47C0000000000000
  CRC64Table(CRC64TableA, 117) = $4670000000000000 : CRC64Table(CRC64TableA, 118) = $44A0000000000000 : CRC64Table(CRC64TableA, 119) = $4510000000000000
  CRC64Table(CRC64TableA, 120) = $4C80000000000000 : CRC64Table(CRC64TableA, 121) = $4D30000000000000 : CRC64Table(CRC64TableA, 122) = $4FE0000000000000 
  CRC64Table(CRC64TableA, 123) = $4E50000000000000 : CRC64Table(CRC64TableA, 124) = $4A40000000000000 : CRC64Table(CRC64TableA, 125) = $4BF0000000000000
  CRC64Table(CRC64TableA, 126) = $4920000000000000 : CRC64Table(CRC64TableA, 127) = $4890000000000000 : CRC64Table(CRC64TableA, 128) = $D800000000000000 
  CRC64Table(CRC64TableA, 129) = $D9B0000000000000 : CRC64Table(CRC64TableA, 130) = $DB60000000000000 : CRC64Table(CRC64TableA, 131) = $DAD0000000000000
  CRC64Table(CRC64TableA, 132) = $DEC0000000000000 : CRC64Table(CRC64TableA, 133) = $DF70000000000000 : CRC64Table(CRC64TableA, 134) = $DDA0000000000000
  CRC64Table(CRC64TableA, 135) = $DC10000000000000 : CRC64Table(CRC64TableA, 136) = $D580000000000000 : CRC64Table(CRC64TableA, 137) = $D430000000000000
  CRC64Table(CRC64TableA, 138) = $D6E0000000000000 : CRC64Table(CRC64TableA, 139) = $D750000000000000 : CRC64Table(CRC64TableA, 140) = $D340000000000000 
  CRC64Table(CRC64TableA, 141) = $D2F0000000000000 : CRC64Table(CRC64TableA, 142) = $D020000000000000 : CRC64Table(CRC64TableA, 143) = $D190000000000000
  CRC64Table(CRC64TableA, 144) = $C300000000000000 : CRC64Table(CRC64TableA, 145) = $C2B0000000000000 : CRC64Table(CRC64TableA, 146) = $C060000000000000
  CRC64Table(CRC64TableA, 147) = $C1D0000000000000 : CRC64Table(CRC64TableA, 148) = $C5C0000000000000 : CRC64Table(CRC64TableA, 149) = $C470000000000000
  CRC64Table(CRC64TableA, 150) = $C6A0000000000000 : CRC64Table(CRC64TableA, 151) = $C710000000000000 : CRC64Table(CRC64TableA, 152) = $CE80000000000000
  CRC64Table(CRC64TableA, 153) = $CF30000000000000 : CRC64Table(CRC64TableA, 154) = $CDE0000000000000 : CRC64Table(CRC64TableA, 155) = $CC50000000000000
  CRC64Table(CRC64TableA, 156) = $C840000000000000 : CRC64Table(CRC64TableA, 157) = $C9F0000000000000 : CRC64Table(CRC64TableA, 158) = $CB20000000000000
  CRC64Table(CRC64TableA, 159) = $CA90000000000000 : CRC64Table(CRC64TableA, 160) = $EE00000000000000 : CRC64Table(CRC64TableA, 161) = $EFB0000000000000
  CRC64Table(CRC64TableA, 162) = $ED60000000000000 : CRC64Table(CRC64TableA, 163) = $ECD0000000000000 : CRC64Table(CRC64TableA, 164) = $E8C0000000000000 
  CRC64Table(CRC64TableA, 165) = $E970000000000000 : CRC64Table(CRC64TableA, 166) = $EBA0000000000000 : CRC64Table(CRC64TableA, 167) = $EA10000000000000
  CRC64Table(CRC64TableA, 168) = $E380000000000000 : CRC64Table(CRC64TableA, 169) = $E230000000000000 : CRC64Table(CRC64TableA, 170) = $E0E0000000000000
  CRC64Table(CRC64TableA, 171) = $E150000000000000 : CRC64Table(CRC64TableA, 172) = $E540000000000000 : CRC64Table(CRC64TableA, 173) = $E4F0000000000000
  CRC64Table(CRC64TableA, 174) = $E620000000000000 : CRC64Table(CRC64TableA, 175) = $E790000000000000 : CRC64Table(CRC64TableA, 176) = $F500000000000000
  CRC64Table(CRC64TableA, 177) = $F4B0000000000000 : CRC64Table(CRC64TableA, 178) = $F660000000000000 : CRC64Table(CRC64TableA, 179) = $F7D0000000000000
  CRC64Table(CRC64TableA, 180) = $F3C0000000000000 : CRC64Table(CRC64TableA, 181) = $F270000000000000 : CRC64Table(CRC64TableA, 182) = $F0A0000000000000
  CRC64Table(CRC64TableA, 183) = $F110000000000000 : CRC64Table(CRC64TableA, 184) = $F880000000000000 : CRC64Table(CRC64TableA, 185) = $F930000000000000
  CRC64Table(CRC64TableA, 186) = $FBE0000000000000 : CRC64Table(CRC64TableA, 187) = $FA50000000000000 : CRC64Table(CRC64TableA, 188) = $FE40000000000000
  CRC64Table(CRC64TableA, 189) = $FFF0000000000000 : CRC64Table(CRC64TableA, 190) = $FD20000000000000 : CRC64Table(CRC64TableA, 191) = $FC90000000000000
  CRC64Table(CRC64TableA, 192) = $B400000000000000 : CRC64Table(CRC64TableA, 193) = $B5B0000000000000 : CRC64Table(CRC64TableA, 194) = $B760000000000000  
  CRC64Table(CRC64TableA, 195) = $B6D0000000000000 : CRC64Table(CRC64TableA, 196) = $B2C0000000000000 : CRC64Table(CRC64TableA, 197) = $B370000000000000
  CRC64Table(CRC64TableA, 198) = $B1A0000000000000 : CRC64Table(CRC64TableA, 199) = $B010000000000000 : CRC64Table(CRC64TableA, 200) = $B980000000000000
  CRC64Table(CRC64TableA, 201) = $B830000000000000 : CRC64Table(CRC64TableA, 202) = $BAE0000000000000 : CRC64Table(CRC64TableA, 203) = $BB50000000000000
  CRC64Table(CRC64TableA, 204) = $BF40000000000000 : CRC64Table(CRC64TableA, 205) = $BEF0000000000000 : CRC64Table(CRC64TableA, 206) = $BC20000000000000
  CRC64Table(CRC64TableA, 207) = $BD90000000000000 : CRC64Table(CRC64TableA, 208) = $AF00000000000000 : CRC64Table(CRC64TableA, 209) = $AEB0000000000000
  CRC64Table(CRC64TableA, 210) = $AC60000000000000 : CRC64Table(CRC64TableA, 211) = $ADD0000000000000 : CRC64Table(CRC64TableA, 212) = $A9C0000000000000
  CRC64Table(CRC64TableA, 213) = $A870000000000000 : CRC64Table(CRC64TableA, 214) = $AAA0000000000000 : CRC64Table(CRC64TableA, 215) = $AB10000000000000
  CRC64Table(CRC64TableA, 216) = $A280000000000000 : CRC64Table(CRC64TableA, 217) = $A330000000000000 : CRC64Table(CRC64TableA, 218) = $A1E0000000000000
  CRC64Table(CRC64TableA, 219) = $A050000000000000 : CRC64Table(CRC64TableA, 220) = $A440000000000000 : CRC64Table(CRC64TableA, 221) = $A5F0000000000000
  CRC64Table(CRC64TableA, 222) = $A720000000000000 : CRC64Table(CRC64TableA, 223) = $A690000000000000 : CRC64Table(CRC64TableA, 224) = $8200000000000000
  CRC64Table(CRC64TableA, 225) = $83B0000000000000 : CRC64Table(CRC64TableA, 226) = $8160000000000000 : CRC64Table(CRC64TableA, 227) = $80D0000000000000
  CRC64Table(CRC64TableA, 228) = $84C0000000000000 : CRC64Table(CRC64TableA, 229) = $8570000000000000 : CRC64Table(CRC64TableA, 230) = $87A0000000000000
  CRC64Table(CRC64TableA, 231) = $8610000000000000 : CRC64Table(CRC64TableA, 232) = $8F80000000000000 : CRC64Table(CRC64TableA, 233) = $8E30000000000000
  CRC64Table(CRC64TableA, 234) = $8CE0000000000000 : CRC64Table(CRC64TableA, 235) = $8D50000000000000 : CRC64Table(CRC64TableA, 236) = $8940000000000000
  CRC64Table(CRC64TableA, 237) = $88F0000000000000 : CRC64Table(CRC64TableA, 238) = $8A20000000000000 : CRC64Table(CRC64TableA, 239) = $8B90000000000000
  CRC64Table(CRC64TableA, 240) = $9900000000000000 : CRC64Table(CRC64TableA, 241) = $98B0000000000000 : CRC64Table(CRC64TableA, 242) = $9A60000000000000
  CRC64Table(CRC64TableA, 243) = $9BD0000000000000 : CRC64Table(CRC64TableA, 244) = $9FC0000000000000 : CRC64Table(CRC64TableA, 245) = $9E70000000000000
  CRC64Table(CRC64TableA, 246) = $9CA0000000000000 : CRC64Table(CRC64TableA, 247) = $9D10000000000000 : CRC64Table(CRC64TableA, 248) = $9480000000000000
  CRC64Table(CRC64TableA, 249) = $9530000000000000 : CRC64Table(CRC64TableA, 250) = $97E0000000000000 : CRC64Table(CRC64TableA, 251) = $9650000000000000
  CRC64Table(CRC64TableA, 252) = $9240000000000000 : CRC64Table(CRC64TableA, 253) = $93F0000000000000 
  CRC64Table(CRC64TableA, 254) = $9120000000000000 : CRC64Table(CRC64TableA, 255) = $9090000000000000
  
EndMacro

Macro CRC64StringFingerprint(text)
  
  CRC64Fingerprint(@text, StringByteLength(text))
  
EndMacro

Macro CRC32StringFingerprint(text)
  
  CRC32Fingerprint(@text, StringByteLength(text))
  
EndMacro

Procedure.q CRC64Fingerprint(*Buffer, Size.l, InitialValue.q = 0)
  
  ; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  ; Modification made by Guimauve : Use of static CRC64Table
  ; We initialize the table on the first call of 
  ; CRC64Fingerprint and we keep the table in memory until 
  ; the program finish. Also I have change some variable names
  
  Static CRC64Table.CRC64Table, OneShot.b
  
  If OneShot = #False
    OneShot = #True
    InitializeCRC64Table(CRC64Table)
  EndIf
  
  CRC64_Fingerprint.q = InitialValue
  
  For ByteID = 0 To Size - 1
    Buffer_UByte.a = PeekA(*Buffer + (ByteID))
    Index.a = ((CRC64_Fingerprint ! Buffer_UByte) & $FF)
    CRC64_Fingerprint = CRC64Table(CRC64Table, Index) ! SHR64(CRC64_Fingerprint, 8)
  Next
  
  ProcedureReturn CRC64_Fingerprint
EndProcedure

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 
; <<<<< !!! WARNING - YOU ARE NOW IN A TESTING ZONE - WARNING !!! <<<<< 
; <<<<< !!! WARNING - THIS CODE SHOULD BE COMMENTED - WARNING !!! <<<<< 
; <<<<< !!! WARNING - BEFORE THE FINAL COMPILATION. - WARNING !!! <<<<< 
; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 

Macro NewLongVector(ElementCount)
  
  AllocateMemory((ElementCount) * SizeOf(Long))
  
EndMacro 

Macro DeleteLongVector(StartPtr)
  
  If StartPtr <> #Null
    FreeMemory(StartPtr)
  EndIf
  
EndMacro 

Macro LongVectorSize(StartPtr)
  
  (MemorySize(StartPtr) / SizeOf(Long) - 1)
  
EndMacro

Macro ReachLongVectorElement(StartPtr, Index)
  
  (StartPtr + (Index) * SizeOf(Long))
  
EndMacro 

Macro PokeLongVectorElement(StartPtr, Index, Value)
  
  PokeL(ReachLongVectorElement(StartPtr, Index), Value)
  
EndMacro 

Macro PeekLongVectorElement(StartPtr, Index)
  
  PeekL(ReachLongVectorElement(StartPtr, Index))
  
EndMacro 

String.s = "123456789"
length = StringByteLength(String)

; 1 step calculation
hash0.q = CRC64StringFingerprint(String)
hash2.l = CRC32StringFingerprint(String)

; 2 step calculation 
part1.q = CRC64Fingerprint(@string, length/2) 
part3.l = CRC32Fingerprint(@string, length/2) 

CompilerIf #PB_Compiler_Unicode
  
  hash1.q = CRC64Fingerprint(@string + length/2, length/2, part1) 
  hash3.l = CRC32Fingerprint(@string + length/2, length/2, part3) 
  
CompilerElse
  
  hash1.q = CRC64Fingerprint(@string + length/2, length/2+1, part1) 
  hash3.l = CRC32Fingerprint(@string + length/2, length/2+1, part3) 
  
CompilerEndIf

; Memory Block test
Vector.i = NewLongVector(25)

For Index = 0 To LongVectorSize(Vector)
  PokeLongVectorElement(Vector, Index, Index * 2)
Next

hash4.q = CRC64Fingerprint(Vector, MemorySize(Vector))
hash5.l = CRC32Fingerprint(Vector, MemorySize(Vector))

Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"

CompilerIf #PB_Compiler_Unicode
  Debug "; Compiler Mode Unicode"
  
CompilerElse
  Debug "; Compiler Mode Ascii"
  
CompilerEndIf

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Single step calculation"
Debug "; "
Debug "; CRC32(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash2, #PB_Long)
Debug "; CRC64(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash0, #PB_Quad) 

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Twin step calculation"
Debug "; "
Debug "; CRC32(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash3, #PB_Long)
Debug "; CRC64(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash1, #PB_Quad)

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Test on memory block"
Debug "; "

Debug "; CRC32(Vector) = " + Hex(hash5, #PB_Long)
Debug "; CRC64(Vector) = " + Hex(hash4, #PB_Quad)

DeleteLongVector(Vector)

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Compiler Mode Ascii

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Single step calculation
; 
; CRC32("123456789") = CBF43926
; CRC64("123456789") = 46A5A9388A5BEFFE

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Twin step calculation
; 
; CRC32("123456789") = CBF43926
; CRC64("123456789") = 46A5A9388A5BEFFE

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Test on memory block
; 
; CRC32(Vector) = 9F8B14FB
; CRC64(Vector) = 2454C5C00DC5820B

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Compiler Mode Unicode

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Single step calculation
; 
; CRC32("123456789") = A290E877
; CRC64("123456789") = A1DE936964162F16

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Twin step calculation
; 
; CRC32("123456789") = A290E877
; CRC64("123456789") = A1DE936964162F16

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Test on memory block
; 
; CRC32(Vector) = 9F8B14FB
; CRC64(Vector) = 2454C5C00DC5820B

; <<<<<<<<<<<<<<<<<<<<<<<
; <<<<< END OF FILE <<<<<
; <<<<<<<<<<<<<<<<<<<<<<<

moogle · Post by **moogle** » Mon Feb 20, 2012 5:03 pm

Guimauve wrote:Hello everyone,

Sorry to re-up this old topic but I have made some change to the code provided by moogle. I have manage to understand why the twin step CRC32 (standard command) and CRC64 return different value in Ascii and Unicode mode. Apparently, we need to calculate the part memory length differently in Ascii and Unicode mode, see the demonstration code.

Best regards
Guimauve

That's okay, you can re-up any topics by me

Yeah I was confused as how to proceed with the different values in Ascii and Unicode since their actual values are different. I saw the MD5 function has a different value for ascii and unicode too so I let it work that way.

BTW I get a invalid memory access (PB 4.61 Beta 1 x86 Windows) on line 331

Code: Select all

CRC64_Fingerprint = CRC64Table(CRC64Table, ((CRC64_Fingerprint ! Buffer_UByte) & $FF)) ! SHR64(CRC64_Fingerprint, 8)

If I change it to

Code: Select all

Index=((CRC64_Fingerprint ! Buffer_UByte) & $FF)
CRC64_Fingerprint = CRC64Table(CRC64Table, Index) ! SHR64(CRC64_Fingerprint, 8)

It works fine now with no errors.

Guimauve · Post by **Guimauve** » Mon Feb 20, 2012 7:16 pm

moogle wrote:BTW I get a invalid memory access (PB 4.61 Beta 1 x86 Windows) on line 331
Code: Select all
CRC64_Fingerprint = CRC64Table(CRC64Table, ((CRC64_Fingerprint ! Buffer_UByte) & $FF)) ! SHR64(CRC64_Fingerprint, 8)
If I change it to
Code: Select all
Index=((CRC64_Fingerprint ! Buffer_UByte) & $FF)
CRC64_Fingerprint = CRC64Table(CRC64Table, Index) ! SHR64(CRC64_Fingerprint, 8)
It works fine now with no errors.

Strange, everything work fine here in LinuxMint 12 x64 with PureBasic 4.61 Beta 1 x64

I have updated the code.

Best regards.
Guimauve

wilbert · Post by **wilbert** » Mon Feb 20, 2012 9:37 pm

The table contains so many zero values that you can shrink it a lot by making the table word based.

Code: Select all

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Project name : CRC64Fingerprint
; File Name : CRC64Fingerprint.pb
; File version: 1.1.0
; Programming : OK
; Programmed by : moogle (English forum)
; Modified by : Guimauve
; Date : 15-08-2011
; Last Update : 20-02-2012
; PureBasic code : 4.61
; Platform : Windows, Linux, MacOS X
; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Procedure.q SHR64(val.q, n)
  
  If val & (1 << 63)  ; sign bit is set, normal shift will fail
    val &~ (1 << 63)  ; clear the sign bit
    val >> n          ; shift will work normally now
    val | 1 << (63 - n) ; set the original sign bit at its new place
  Else
    val >> n ; nothing needed
  EndIf
  
  ProcedureReturn val
EndProcedure

Structure CRC64Table
  
  Element.w[256]
  
EndStructure 

Macro CRC64StringFingerprint(text)
  
  CRC64Fingerprint(@text, StringByteLength(text))
  
EndMacro

Macro CRC32StringFingerprint(text)
  
  CRC32Fingerprint(@text, StringByteLength(text))
  
EndMacro

Procedure.q CRC64Fingerprint(*Buffer.Ascii, Size.l, InitialValue.q = 0)
  
  *CRC64Table.CRC64Table = ?crc64iso_short
  
  CRC64_Fingerprint.q = InitialValue
  
  For i = 1 To Size
    Index.a = CRC64_Fingerprint ! *Buffer\a
    CRC64_Fingerprint = (*CRC64Table\Element[Index] << 48) ! SHR64(CRC64_Fingerprint, 8)
    *Buffer + 1
  Next
  
  ProcedureReturn CRC64_Fingerprint
  
  DataSection
    crc64iso_short:
    Data.q $2D0036001B000000,$41005A0077006C00,$F500EE00C300D800,$99008200AF00B400
    Data.q $19D018601AB01B00,$1F101EA01C701DC0,$145015E017301680,$1290132011F01040
    Data.q $34D0356037B03600,$321033A0317030C0,$395038E03A303B80,$3F903E203CF03D40
    Data.q $2FD02E602CB02D00,$291028A02A702BC0,$225023E021302080,$2490252027F02640
    Data.q $6ED06F606DB06C00,$681069A06B706AC0,$635062E060306180,$6590642066F06740
    Data.q $75D0746076B07700,$731072A0707071C0,$785079E07B307A80,$7E907F207DF07C40
    Data.q $58D059605BB05A00,$5E105FA05D705CC0,$555054E056305780,$5390522050F05140
    Data.q $43D0426040B04100,$451044A0467047C0,$4E504FE04D304C80,$489049204BF04A40
    Data.q $DAD0DB60D9B0D800,$DC10DDA0DF70DEC0,$D750D6E0D430D580,$D190D020D2F0D340
    Data.q $C1D0C060C2B0C300,$C710C6A0C470C5C0,$CC50CDE0CF30CE80,$CA90CB20C9F0C840
    Data.q $ECD0ED60EFB0EE00,$EA10EBA0E970E8C0,$E150E0E0E230E380,$E790E620E4F0E540
    Data.q $F7D0F660F4B0F500,$F110F0A0F270F3C0,$FA50FBE0F930F880,$FC90FD20FFF0FE40
    Data.q $B6D0B760B5B0B400,$B010B1A0B370B2C0,$BB50BAE0B830B980,$BD90BC20BEF0BF40
    Data.q $ADD0AC60AEB0AF00,$AB10AAA0A870A9C0,$A050A1E0A330A280,$A690A720A5F0A440
    Data.q $80D0816083B08200,$861087A0857084C0,$8D508CE08E308F80,$8B908A2088F08940
    Data.q $9BD09A6098B09900,$9D109CA09E709FC0,$965097E095309480,$9090912093F09240
  EndDataSection
  
EndProcedure

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 
; <<<<< !!! WARNING - YOU ARE NOW IN A TESTING ZONE - WARNING !!! <<<<< 
; <<<<< !!! WARNING - THIS CODE SHOULD BE COMMENTED - WARNING !!! <<<<< 
; <<<<< !!! WARNING - BEFORE THE FINAL COMPILATION. - WARNING !!! <<<<< 
; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 

Macro NewLongVector(ElementCount)
  
  AllocateMemory((ElementCount) * SizeOf(Long))
  
EndMacro 

Macro DeleteLongVector(StartPtr)
  
  If StartPtr <> #Null
    FreeMemory(StartPtr)
  EndIf
  
EndMacro 

Macro LongVectorSize(StartPtr)
  
  (MemorySize(StartPtr) / SizeOf(Long) - 1)
  
EndMacro

Macro ReachLongVectorElement(StartPtr, Index)
  
  (StartPtr + (Index) * SizeOf(Long))
  
EndMacro 

Macro PokeLongVectorElement(StartPtr, Index, Value)
  
  PokeL(ReachLongVectorElement(StartPtr, Index), Value)
  
EndMacro 

Macro PeekLongVectorElement(StartPtr, Index)
  
  PeekL(ReachLongVectorElement(StartPtr, Index))
  
EndMacro 

String.s = "123456789"
length = StringByteLength(String)

; 1 step calculation
hash0.q = CRC64StringFingerprint(String)
hash2.l = CRC32StringFingerprint(String)

; 2 step calculation 
part1.q = CRC64Fingerprint(@string, length/2) 
part3.l = CRC32Fingerprint(@string, length/2) 

CompilerIf #PB_Compiler_Unicode
  
  hash1.q = CRC64Fingerprint(@string + length/2, length/2, part1) 
  hash3.l = CRC32Fingerprint(@string + length/2, length/2, part3) 
  
CompilerElse
  
  hash1.q = CRC64Fingerprint(@string + length/2, length/2+1, part1) 
  hash3.l = CRC32Fingerprint(@string + length/2, length/2+1, part3) 
  
CompilerEndIf

; Memory Block test
Vector.i = NewLongVector(25)

For Index = 0 To LongVectorSize(Vector)
  PokeLongVectorElement(Vector, Index, Index * 2)
Next

hash4.q = CRC64Fingerprint(Vector, MemorySize(Vector))
hash5.l = CRC32Fingerprint(Vector, MemorySize(Vector))

Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"

CompilerIf #PB_Compiler_Unicode
  Debug "; Compiler Mode Unicode"
  
CompilerElse
  Debug "; Compiler Mode Ascii"
  
CompilerEndIf

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Single step calculation"
Debug "; "
Debug "; CRC32(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash2, #PB_Long)
Debug "; CRC64(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash0, #PB_Quad) 

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Twin step calculation"
Debug "; "
Debug "; CRC32(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash3, #PB_Long)
Debug "; CRC64(" + Chr(34) + "123456789" + Chr(34) + ") = " + Hex(hash1, #PB_Quad)

Debug ""
Debug "; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<"
Debug "; Test on memory block"
Debug "; "

Debug "; CRC32(Vector) = " + Hex(hash5, #PB_Long)
Debug "; CRC64(Vector) = " + Hex(hash4, #PB_Quad)

DeleteLongVector(Vector)

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Compiler Mode Ascii

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Single step calculation
; 
; CRC32("123456789") = CBF43926
; CRC64("123456789") = 46A5A9388A5BEFFE

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Twin step calculation
; 
; CRC32("123456789") = CBF43926
; CRC64("123456789") = 46A5A9388A5BEFFE

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Test on memory block
; 
; CRC32(Vector) = 9F8B14FB
; CRC64(Vector) = 2454C5C00DC5820B

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Compiler Mode Unicode

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Single step calculation
; 
; CRC32("123456789") = A290E877
; CRC64("123456789") = A1DE936964162F16

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Twin step calculation
; 
; CRC32("123456789") = A290E877
; CRC64("123456789") = A1DE936964162F16

; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
; Test on memory block
; 
; CRC32(Vector) = 9F8B14FB
; CRC64(Vector) = 2454C5C00DC5820B

; <<<<<<<<<<<<<<<<<<<<<<<
; <<<<< END OF FILE <<<<<
; <<<<<<<<<<<<<<<<<<<<<<<

Crusiatus Black · Post by **Crusiatus Black** » Mon Nov 25, 2013 10:20 am

My apologies for the bump, however I've been trying to implement this in
assembly, so I could learn more about ASM. I'm not quite getting the results
I've been expecting, but I'm sure I've made a few mistakes here and there.

If anyone could explain my mistakes to me, I'd be very thankful

Code fixed, thanks to Wilbert's input

Code: Select all

Procedure.q CRC64Checksum(*data, size, init.q)
  !PUSH     EBP
  
  !MOV      EBP, ESP
  
  !PUSH     ESI
  !PUSH     EDI
  
  !MOV      ESI, [EBP + 8]    ; data
  !MOV      ECX, [EBP + 12]   ; size
  !MOV      EAX, [EBP + 16]   ; init.q 00-31
  !MOV      EDI, [EBP + 20]   ; init.q 31-63
  
  !MOV      EBX, crc64_table
  
  !XOR      EDX, EDX
  !JECXZ    crc64_done
  
  !crc64_start:
    !MOV      DL, [ESI]   ; byte
    !INC      ESI
    !XOR      DL, AL      ; index
    
    ; crc64 >> 8
    !SHRD     EAX, EDI, 8 
    !SHR      EDI, 8
    
    ; crc64 xor crc64_table[index] 
    !XOR      EAX, [EBX + 0 + EDX * 8]
    !XOR      EDI, [EBX + 4 + EDX * 8]
    
    ; counter--
    !SUB      ECX, 1
    !JNZ      crc64_start
  !crc64_done:  
    !MOV      EDX, EDI
  
    !POP      EDI
    !POP      ESI
    !POP      EBP
  ProcedureReturn
EndProcedure

test$ = "123456789"
Debug Hex(CRC64Checksum(@test$, Len(test$), 0), #PB_Quad)

; Procedure.q CRC64Fingerprint(*Buffer, Size.l, InitialValue.q=0)
;    Protected CRC64TableFill()
;    Protected.q crc64=InitialValue, i.l, byte.a, index.a
;    For i=1 To Size
;       byte=PeekB(*Buffer+(i-1))
;       index =  ((crc64 ! byte) & $FF);
;       crc64 = CRC64Table(index) ! SHR64(crc64,8);
;    Next
;    ProcedureReturn crc64;
; EndProcedure

End 
!crc64_table:
	!dq 0x0000000000000000, 0x1B00000000000000, 0x3600000000000000, 0x2D00000000000000, 0x6C00000000000000, 0x7700000000000000, 0x5A00000000000000, 0x4100000000000000
	!dq 0xD800000000000000, 0xC300000000000000, 0xEE00000000000000, 0xF500000000000000, 0xB400000000000000, 0xAF00000000000000, 0x8200000000000000, 0x9900000000000000
	!dq 0x1B00000000000000, 0x1AB0000000000000, 0x1860000000000000, 0x19D0000000000000, 0x1DC0000000000000, 0x1C70000000000000, 0x1EA0000000000000, 0x1F10000000000000
	!dq 0x1680000000000000, 0x1730000000000000, 0x15E0000000000000, 0x1450000000000000, 0x1040000000000000, 0x11F0000000000000, 0x1320000000000000, 0x1290000000000000
	!dq 0x3600000000000000, 0x37B0000000000000, 0x3560000000000000, 0x34D0000000000000, 0x30C0000000000000, 0x3170000000000000, 0x33A0000000000000, 0x3210000000000000
	!dq 0x3B80000000000000, 0x3A30000000000000, 0x38E0000000000000, 0x3950000000000000, 0x3D40000000000000, 0x3CF0000000000000, 0x3E20000000000000, 0x3F90000000000000
	!dq 0x2D00000000000000, 0x2CB0000000000000, 0x2E60000000000000, 0x2FD0000000000000, 0x2BC0000000000000, 0x2A70000000000000, 0x28A0000000000000, 0x2910000000000000
	!dq 0x2080000000000000, 0x2130000000000000, 0x23E0000000000000, 0x2250000000000000, 0x2640000000000000, 0x27F0000000000000, 0x2520000000000000, 0x2490000000000000
	!dq 0x6C00000000000000, 0x6DB0000000000000, 0x6F60000000000000, 0x6ED0000000000000, 0x6AC0000000000000, 0x6B70000000000000, 0x69A0000000000000, 0x6810000000000000
	!dq 0x6180000000000000, 0x6030000000000000, 0x62E0000000000000, 0x6350000000000000, 0x6740000000000000, 0x66F0000000000000, 0x6420000000000000, 0x6590000000000000
	!dq 0x7700000000000000, 0x76B0000000000000, 0x7460000000000000, 0x75D0000000000000, 0x71C0000000000000, 0x7070000000000000, 0x72A0000000000000, 0x7310000000000000
	!dq 0x7A80000000000000, 0x7B30000000000000, 0x79E0000000000000, 0x7850000000000000, 0x7C40000000000000, 0x7DF0000000000000, 0x7F20000000000000, 0x7E90000000000000
	!dq 0x5A00000000000000, 0x5BB0000000000000, 0x5960000000000000, 0x58D0000000000000, 0x5CC0000000000000, 0x5D70000000000000, 0x5FA0000000000000, 0x5E10000000000000
	!dq 0x5780000000000000, 0x5630000000000000, 0x54E0000000000000, 0x5550000000000000, 0x5140000000000000, 0x50F0000000000000, 0x5220000000000000, 0x5390000000000000
	!dq 0x4100000000000000, 0x40B0000000000000, 0x4260000000000000, 0x43D0000000000000, 0x47C0000000000000, 0x4670000000000000, 0x44A0000000000000, 0x4510000000000000
	!dq 0x4C80000000000000, 0x4D30000000000000, 0x4FE0000000000000, 0x4E50000000000000, 0x4A40000000000000, 0x4BF0000000000000, 0x4920000000000000, 0x4890000000000000
	!dq 0xD800000000000000, 0xD9B0000000000000, 0xDB60000000000000, 0xDAD0000000000000, 0xDEC0000000000000, 0xDF70000000000000, 0xDDA0000000000000, 0xDC10000000000000
	!dq 0xD580000000000000, 0xD430000000000000, 0xD6E0000000000000, 0xD750000000000000, 0xD340000000000000, 0xD2F0000000000000, 0xD020000000000000, 0xD190000000000000
	!dq 0xC300000000000000, 0xC2B0000000000000, 0xC060000000000000, 0xC1D0000000000000, 0xC5C0000000000000, 0xC470000000000000, 0xC6A0000000000000, 0xC710000000000000
	!dq 0xCE80000000000000, 0xCF30000000000000, 0xCDE0000000000000, 0xCC50000000000000, 0xC840000000000000, 0xC9F0000000000000, 0xCB20000000000000, 0xCA90000000000000
	!dq 0xEE00000000000000, 0xEFB0000000000000, 0xED60000000000000, 0xECD0000000000000, 0xE8C0000000000000, 0xE970000000000000, 0xEBA0000000000000, 0xEA10000000000000
	!dq 0xE380000000000000, 0xE230000000000000, 0xE0E0000000000000, 0xE150000000000000, 0xE540000000000000, 0xE4F0000000000000, 0xE620000000000000, 0xE790000000000000
	!dq 0xF500000000000000, 0xF4B0000000000000, 0xF660000000000000, 0xF7D0000000000000, 0xF3C0000000000000, 0xF270000000000000, 0xF0A0000000000000, 0xF110000000000000
	!dq 0xF880000000000000, 0xF930000000000000, 0xFBE0000000000000, 0xFA50000000000000, 0xFE40000000000000, 0xFFF0000000000000, 0xFD20000000000000, 0xFC90000000000000
	!dq 0xB400000000000000, 0xB5B0000000000000, 0xB760000000000000, 0xB6D0000000000000, 0xB2C0000000000000, 0xB370000000000000, 0xB1A0000000000000, 0xB010000000000000
	!dq 0xB980000000000000, 0xB830000000000000, 0xBAE0000000000000, 0xBB50000000000000, 0xBF40000000000000, 0xBEF0000000000000, 0xBC20000000000000, 0xBD90000000000000
	!dq 0xAF00000000000000, 0xAEB0000000000000, 0xAC60000000000000, 0xADD0000000000000, 0xA9C0000000000000, 0xA870000000000000, 0xAAA0000000000000, 0xAB10000000000000
	!dq 0xA280000000000000, 0xA330000000000000, 0xA1E0000000000000, 0xA050000000000000, 0xA440000000000000, 0xA5F0000000000000, 0xA720000000000000, 0xA690000000000000
	!dq 0x8200000000000000, 0x83B0000000000000, 0x8160000000000000, 0x80D0000000000000, 0x84C0000000000000, 0x8570000000000000, 0x87A0000000000000, 0x8610000000000000
	!dq 0x8F80000000000000, 0x8E30000000000000, 0x8CE0000000000000, 0x8D50000000000000, 0x8940000000000000, 0x88F0000000000000, 0x8A20000000000000, 0x8B90000000000000
	!dq 0x9900000000000000, 0x98B0000000000000, 0x9A60000000000000, 0x9BD0000000000000, 0x9FC0000000000000, 0x9E70000000000000, 0x9CA0000000000000, 0x9D10000000000000
	!dq 0x9480000000000000, 0x9530000000000000, 0x97E0000000000000, 0x9650000000000000, 0x9240000000000000, 0x93F0000000000000, 0x9120000000000000, 0x9090000000000000

wilbert · Post by **wilbert** » Mon Nov 25, 2013 12:02 pm

Maybe something like this ?

Code: Select all

Procedure.q CRC64Fingerprint(*Buffer, Size, InitialValue.q = 0)
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
    
    !mov edx, [p.p_Buffer]
    !mov ecx, [p.v_Size]
    !lea eax, [p.v_InitialValue]
    !push ebx
    !push esi
    !mov esi, edx
    !mov edx, [eax + 4]
    !mov eax, [eax]
    !crc64iso_loop:
    !movzx ebx, al
    !shrd eax, edx, 8
    !shr edx, 8
    !xor bl, [esi]
    !mov bx, [l_crc64iso_short + ebx * 2]
    !shl ebx, 16
    !xor edx, ebx
    !inc esi
    !dec ecx
    !jnz crc64iso_loop
    !pop esi
    !pop ebx
    
  CompilerElse
    
    !lea r9, [l_crc64iso_short]
    !mov r8, [p.p_Buffer]
    !mov rcx, [p.v_Size]
    !mov rax, [p.v_InitialValue]
    !crc64iso_loop:
    !movzx rdx, al
    !shr rax, 8
    !xor dl, [r8]
    !mov dx, [r9 + rdx * 2]
    !shl rdx, 48
    !xor rax, rdx
    !inc r8
    !dec rcx
    !jnz crc64iso_loop
    
  CompilerEndIf
  ProcedureReturn
  
  DataSection
    crc64iso_short:
    Data.q $2D0036001B000000,$41005A0077006C00,$F500EE00C300D800,$99008200AF00B400
    Data.q $19D018601AB01B00,$1F101EA01C701DC0,$145015E017301680,$1290132011F01040
    Data.q $34D0356037B03600,$321033A0317030C0,$395038E03A303B80,$3F903E203CF03D40
    Data.q $2FD02E602CB02D00,$291028A02A702BC0,$225023E021302080,$2490252027F02640
    Data.q $6ED06F606DB06C00,$681069A06B706AC0,$635062E060306180,$6590642066F06740
    Data.q $75D0746076B07700,$731072A0707071C0,$785079E07B307A80,$7E907F207DF07C40
    Data.q $58D059605BB05A00,$5E105FA05D705CC0,$555054E056305780,$5390522050F05140
    Data.q $43D0426040B04100,$451044A0467047C0,$4E504FE04D304C80,$489049204BF04A40
    Data.q $DAD0DB60D9B0D800,$DC10DDA0DF70DEC0,$D750D6E0D430D580,$D190D020D2F0D340
    Data.q $C1D0C060C2B0C300,$C710C6A0C470C5C0,$CC50CDE0CF30CE80,$CA90CB20C9F0C840
    Data.q $ECD0ED60EFB0EE00,$EA10EBA0E970E8C0,$E150E0E0E230E380,$E790E620E4F0E540
    Data.q $F7D0F660F4B0F500,$F110F0A0F270F3C0,$FA50FBE0F930F880,$FC90FD20FFF0FE40
    Data.q $B6D0B760B5B0B400,$B010B1A0B370B2C0,$BB50BAE0B830B980,$BD90BC20BEF0BF40
    Data.q $ADD0AC60AEB0AF00,$AB10AAA0A870A9C0,$A050A1E0A330A280,$A690A720A5F0A440
    Data.q $80D0816083B08200,$861087A0857084C0,$8D508CE08E308F80,$8B908A2088F08940
    Data.q $9BD09A6098B09900,$9D109CA09E709FC0,$965097E095309480,$9090912093F09240
  EndDataSection
  
EndProcedure

Macro CRC64StringFingerprint(text)
  CRC64Fingerprint(@text, StringByteLength(text))
EndMacro

String.s = "123456789"
length = StringByteLength(String)

; 1 step calculation
S.s = Hex(CRC64StringFingerprint(String))
Debug S

Crusiatus Black · Post by **Crusiatus Black** » Mon Nov 25, 2013 2:58 pm

Thanks! That's beautiful, but could you explain to me a little bit what I've done wrong?
Was my code completely rubbish? :p

wilbert · Post by **wilbert** » Mon Nov 25, 2013 3:25 pm

Crusiatus Black wrote:Thanks! That's beautiful, but could you explain to me a little bit what I've done wrong?
Was my code completely rubbish? :p

Your code doesn't look like rubbish

The thing is that I'm usually working on OS X.
The approach you are using to retrieve the procedure arguments (offset from stack pointer) isn't cross platform compatible so several things need to be changed to make it work on OS X.
The code I posted was something I more or less already had on my computer. So I only cleaned up my existing code a bit and didn't test yours.

After looking closer at the code you posted, the problem is with the lookup table.
You are multiplying EDX by 4 instead of 8.

Code: Select all

!XOR      EAX, [EBX + 0 + EDX * 8]
!XOR      EDI, [EBX + 4 + EDX * 8]

Crusiatus Black · Post by **Crusiatus Black** » Mon Nov 25, 2013 3:42 pm

Oooh, I understand! Thanks for the feedback, I should keep in mind that one day my assembly
code should run on Mac and Linux as well (I promise, I usually do that with other code)

Ahhh... the multiplication by 4 is because I had CRC32 in my head (I did a CRC32 before this one,
silly mistake) Thanks!! It works now

I'm going to use your code though, considering it is cross compatible.

coffee · Post by **coffee** » Sat Mar 13, 2021 10:46 pm

Hi,

this doesn't work for the current PB Version.
Really need crc64 - fast for string and file.
Can you help?

thanks

Crusiatus Black · Post by **Crusiatus Black** » Mon Mar 15, 2021 1:31 pm

coffee wrote:Hi,

this doesn't work for the current PB Version.
Really need crc64 - fast for string and file.
Can you help?

thanks

Try it now, I merely moved the datasection outside of the procedure;

Code: Select all

Procedure.q CRC64Fingerprint(*Buffer, Size, InitialValue.q = 0)
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
    !mov edx, [p.p_Buffer]
    !mov ecx, [p.v_Size]
    !lea eax, [p.v_InitialValue]
    !push ebx
    !push esi
    !mov esi, edx
    !mov edx, [eax + 4]
    !mov eax, [eax]
    !crc64iso_loop:
    !movzx ebx, al
    !shrd eax, edx, 8
    !shr edx, 8
    !xor bl, [esi]
    !mov bx, [l_crc64iso_short + ebx * 2]
    !shl ebx, 16
    !xor edx, ebx
    !inc esi
    !dec ecx
    !jnz crc64iso_loop
    !pop esi
    !pop ebx
  CompilerElse
    !lea r9, [l_crc64iso_short]
    !mov r8, [p.p_Buffer]
    !mov rcx, [p.v_Size]
    !mov rax, [p.v_InitialValue]
    !crc64iso_loop:
    !movzx rdx, al
    !shr rax, 8
    !xor dl, [r8]
    !mov dx, [r9 + rdx * 2]
    !shl rdx, 48
    !xor rax, rdx
    !inc r8
    !dec rcx
    !jnz crc64iso_loop 
  CompilerEndIf
  
  ProcedureReturn 
EndProcedure

DataSection
  crc64iso_short:
    Data.q $2D0036001B000000,$41005A0077006C00,$F500EE00C300D800,$99008200AF00B400
    Data.q $19D018601AB01B00,$1F101EA01C701DC0,$145015E017301680,$1290132011F01040
    Data.q $34D0356037B03600,$321033A0317030C0,$395038E03A303B80,$3F903E203CF03D40
    Data.q $2FD02E602CB02D00,$291028A02A702BC0,$225023E021302080,$2490252027F02640
    Data.q $6ED06F606DB06C00,$681069A06B706AC0,$635062E060306180,$6590642066F06740
    Data.q $75D0746076B07700,$731072A0707071C0,$785079E07B307A80,$7E907F207DF07C40
    Data.q $58D059605BB05A00,$5E105FA05D705CC0,$555054E056305780,$5390522050F05140
    Data.q $43D0426040B04100,$451044A0467047C0,$4E504FE04D304C80,$489049204BF04A40
    Data.q $DAD0DB60D9B0D800,$DC10DDA0DF70DEC0,$D750D6E0D430D580,$D190D020D2F0D340
    Data.q $C1D0C060C2B0C300,$C710C6A0C470C5C0,$CC50CDE0CF30CE80,$CA90CB20C9F0C840
    Data.q $ECD0ED60EFB0EE00,$EA10EBA0E970E8C0,$E150E0E0E230E380,$E790E620E4F0E540
    Data.q $F7D0F660F4B0F500,$F110F0A0F270F3C0,$FA50FBE0F930F880,$FC90FD20FFF0FE40
    Data.q $B6D0B760B5B0B400,$B010B1A0B370B2C0,$BB50BAE0B830B980,$BD90BC20BEF0BF40
    Data.q $ADD0AC60AEB0AF00,$AB10AAA0A870A9C0,$A050A1E0A330A280,$A690A720A5F0A440
    Data.q $80D0816083B08200,$861087A0857084C0,$8D508CE08E308F80,$8B908A2088F08940
    Data.q $9BD09A6098B09900,$9D109CA09E709FC0,$965097E095309480,$9090912093F09240
EndDataSection

Macro CRC64StringFingerprint(text)
  CRC64Fingerprint(@text, StringByteLength(text))
EndMacro

String.s = "123456789"
length = StringByteLength(String)

; 1 step calculation
S.s = Hex(CRC64StringFingerprint(String))
Debug S

or change the referenced label name in the inline-assembly:

Code: Select all

Procedure.q CRC64Fingerprint(*Buffer, Size, InitialValue.q = 0)
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
    !mov edx, [p.p_Buffer]
    !mov ecx, [p.v_Size]
    !lea eax, [p.v_InitialValue]
    !push ebx
    !push esi
    !mov esi, edx
    !mov edx, [eax + 4]
    !mov eax, [eax]
    !crc64iso_loop:
    !movzx ebx, al
    !shrd eax, edx, 8
    !shr edx, 8
    !xor bl, [esi]
    !mov bx, [ll_crc64fingerprint_crc64iso_short + ebx * 2]
    !shl ebx, 16
    !xor edx, ebx
    !inc esi
    !dec ecx
    !jnz crc64iso_loop
    !pop esi
    !pop ebx
  CompilerElse
    !lea r9, [ll_crc64fingerprint_crc64iso_short]
    !mov r8, [p.p_Buffer]
    !mov rcx, [p.v_Size]
    !mov rax, [p.v_InitialValue]
    !crc64iso_loop:
    !movzx rdx, al
    !shr rax, 8
    !xor dl, [r8]
    !mov dx, [r9 + rdx * 2]
    !shl rdx, 48
    !xor rax, rdx
    !inc r8
    !dec rcx
    !jnz crc64iso_loop 
  CompilerEndIf
  
  ProcedureReturn 
  
  DataSection
    crc64iso_short:
      Data.q $2D0036001B000000,$41005A0077006C00,$F500EE00C300D800,$99008200AF00B400
      Data.q $19D018601AB01B00,$1F101EA01C701DC0,$145015E017301680,$1290132011F01040
      Data.q $34D0356037B03600,$321033A0317030C0,$395038E03A303B80,$3F903E203CF03D40
      Data.q $2FD02E602CB02D00,$291028A02A702BC0,$225023E021302080,$2490252027F02640
      Data.q $6ED06F606DB06C00,$681069A06B706AC0,$635062E060306180,$6590642066F06740
      Data.q $75D0746076B07700,$731072A0707071C0,$785079E07B307A80,$7E907F207DF07C40
      Data.q $58D059605BB05A00,$5E105FA05D705CC0,$555054E056305780,$5390522050F05140
      Data.q $43D0426040B04100,$451044A0467047C0,$4E504FE04D304C80,$489049204BF04A40
      Data.q $DAD0DB60D9B0D800,$DC10DDA0DF70DEC0,$D750D6E0D430D580,$D190D020D2F0D340
      Data.q $C1D0C060C2B0C300,$C710C6A0C470C5C0,$CC50CDE0CF30CE80,$CA90CB20C9F0C840
      Data.q $ECD0ED60EFB0EE00,$EA10EBA0E970E8C0,$E150E0E0E230E380,$E790E620E4F0E540
      Data.q $F7D0F660F4B0F500,$F110F0A0F270F3C0,$FA50FBE0F930F880,$FC90FD20FFF0FE40
      Data.q $B6D0B760B5B0B400,$B010B1A0B370B2C0,$BB50BAE0B830B980,$BD90BC20BEF0BF40
      Data.q $ADD0AC60AEB0AF00,$AB10AAA0A870A9C0,$A050A1E0A330A280,$A690A720A5F0A440
      Data.q $80D0816083B08200,$861087A0857084C0,$8D508CE08E308F80,$8B908A2088F08940
      Data.q $9BD09A6098B09900,$9D109CA09E709FC0,$965097E095309480,$9090912093F09240
  EndDataSection
EndProcedure

Macro CRC64StringFingerprint(text)
  CRC64Fingerprint(@text, StringByteLength(text))
EndMacro

String.s = "123456789"
length = StringByteLength(String)

; 1 step calculation
S.s = Hex(CRC64StringFingerprint(String))
Debug S

it depends on what you find more neat.

coffee · Post by **coffee** » Mon Mar 15, 2021 7:53 pm

thank you all for your help and suggestions. this works for me:

Code: Select all

EnableExplicit

DeclareModule CRC64
  
  Declare.s CRC64StringFingerprintHex(text.s)
  Declare.s CRC64File(filename.s)
  Declare.q CRC64StringFingerprint(text.s)
  
EndDeclareModule

Module CRC64
  
Procedure.q CRC64Fingerprint(*Buffer, Size, InitialValue.q = 0)
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
    !mov edx, [p.p_Buffer]
    !mov ecx, [p.v_Size]
    !lea eax, [p.v_InitialValue]
    !push ebx
    !push esi
    !mov esi, edx
    !mov edx, [eax + 4]
    !mov eax, [eax]
    !crc64iso_loop:
    !movzx ebx, al
    !shrd eax, edx, 8
    !shr edx, 8
    !xor bl, [esi]
    !mov bx, [crc64iso_short + ebx * 2]
    !shl ebx, 16
    !xor edx, ebx
    !inc esi
    !dec ecx
    !jnz crc64iso_loop
    !pop esi
    !pop ebx
  CompilerElse
    !lea r9, [crc64iso_short]
    !mov r8, [p.p_Buffer]
    !mov rcx, [p.v_Size]
    !mov rax, [p.v_InitialValue]
    !crc64iso_loop:
    !movzx rdx, al
    !shr rax, 8
    !xor dl, [r8]
    !mov dx, [r9 + rdx * 2]
    !shl rdx, 48
    !xor rax, rdx
    !inc r8
    !dec rcx
    !jnz crc64iso_loop
  CompilerEndIf
ProcedureReturn
  DataSection
    !crc64iso_short:
    !dq 0x2D0036001B000000,0x41005A0077006C00,0xF500EE00C300D800,0x99008200AF00B400
    !dq 0x19D018601AB01B00,0x1F101EA01C701DC0,0x145015E017301680,0x1290132011F01040
    !dq 0x34D0356037B03600,0x321033A0317030C0,0x395038E03A303B80,0x3F903E203CF03D40
    !dq 0x2FD02E602CB02D00,0x291028A02A702BC0,0x225023E021302080,0x2490252027F02640
    !dq 0x6ED06F606DB06C00,0x681069A06B706AC0,0x635062E060306180,0x6590642066F06740
    !dq 0x75D0746076B07700,0x731072A0707071C0,0x785079E07B307A80,0x7E907F207DF07C40
    !dq 0x58D059605BB05A00,0x5E105FA05D705CC0,0x555054E056305780,0x5390522050F05140
    !dq 0x43D0426040B04100,0x451044A0467047C0,0x4E504FE04D304C80,0x489049204BF04A40
    !dq 0xDAD0DB60D9B0D800,0xDC10DDA0DF70DEC0,0xD750D6E0D430D580,0xD190D020D2F0D340
    !dq 0xC1D0C060C2B0C300,0xC710C6A0C470C5C0,0xCC50CDE0CF30CE80,0xCA90CB20C9F0C840
    !dq 0xECD0ED60EFB0EE00,0xEA10EBA0E970E8C0,0xE150E0E0E230E380,0xE790E620E4F0E540
    !dq 0xF7D0F660F4B0F500,0xF110F0A0F270F3C0,0xFA50FBE0F930F880,0xFC90FD20FFF0FE40
    !dq 0xB6D0B760B5B0B400,0xB010B1A0B370B2C0,0xBB50BAE0B830B980,0xBD90BC20BEF0BF40
    !dq 0xADD0AC60AEB0AF00,0xAB10AAA0A870A9C0,0xA050A1E0A330A280,0xA690A720A5F0A440
    !dq 0x80D0816083B08200,0x861087A0857084C0,0x8D508CE08E308F80,0x8B908A2088F08940
    !dq 0x9BD09A6098B09900,0x9D109CA09E709FC0,0x965097E095309480,0x9090912093F09240
  EndDataSection
EndProcedure

Procedure.q CRC64StringFingerprint(text.s)
  ProcedureReturn CRC64Fingerprint(@text, StringByteLength(text))
EndProcedure

Procedure.s CRC64StringFingerprintHex(text.s)
  ProcedureReturn Hex(CRC64Fingerprint(@text, StringByteLength(text)))
EndProcedure

Procedure.s CRC64File(filename.s)
  If ReadFile(0, filename)
    Protected length.q = Lof(0)
    Protected *MemoryID = AllocateMemory(length)
    If *MemoryID
      Protected bytes.q = ReadData(0, *MemoryID, length)
    EndIf
    CloseFile(0)
  EndIf
  ProcedureReturn Hex(CRC64Fingerprint(*MemoryID, bytes))
EndProcedure
EndModule

Mijikai · Post by **Mijikai** » Mon Mar 15, 2021 8:26 pm

coffee wrote:

Code: Select all

...
Procedure.s CRC64File(filename.s)
  If ReadFile(0, filename)
    Protected length.q = Lof(0)
    Protected *MemoryID = AllocateMemory(length)
    If *MemoryID
      Protected bytes.q = ReadData(0, *MemoryID, length)
    EndIf
    CloseFile(0)
  EndIf
  ProcedureReturn Hex(CRC64Fingerprint(*MemoryID, bytes))
EndProcedure
...

You forgot to free the memory.
I would also check if ReadData() was successful.
If you want the crc of a file filemapping will be faster than reading the file into a buffer.
The DataSection inside the Procedure looks odd but what do i know the compiler apparently doesnt care.
I personally would put the table inside the code section as it doesnt get modified anyway.

PureBasic Forums - English

Simple CRC64 Routine

Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine

Re: Simple CRC64 Routine