this is bug ?, about g.d of line12

[gurj]

this is bug ?, about g.d of line12

Code: Select all

n=ParseDate("%yyyy.%mm.%dd %hh:%ii:%ss","2016.03.23 17:16:49")
a=ParseDate("%yyyy.%mm.%dd %hh:%ii:%ss","2016.09.21 21:00:00")
b=ParseDate("%yyyy.%mm.%dd %hh:%ii:%ss","2016.09.21 20:50:20")
m=a-n
k=a-b
u.d=k/m
Debug u;=0.000036853028407140316
d.d=u
Debug d;=0.000036853028407140316
g.d=ValD("0.000036853028407140316")
Debug g;=0.000036853028407140316
g.d=0.000036853028407140316
;[19:45:37] [COMPILER] line 12: Numerical overflow: too many digits.

[gurj]

not bug
should:

Code: Select all

d.d=3.6853028407140316e-5
Debug d;=0.000036853028407140316

[gurj]

f:
31-30-23-22-00
1 8 23 2^23=8388608
s exp fraction exp=127+指数值
d:
63-62-52-51-00 exp=1023+指数值
1 11 52 2^52=4503599627370496

if (fraction of f.f)>8388608,best use 'e'
if (fraction of d.d)>4503599627370496,best use 'e'

[gurj]

Code: Select all

;2^52=4503599627370496
;d.d=36853028407140316 best use:
d.d =3685302840714032e1
Debug d
;d.d =0.000036853028407140316 best use:
d.d =3.685302840714032e-5
Debug d

[gurj]

Code: Select all

;***best let ?: 3685302840714032<4503599627370496
;***best use 'e'
d.d =36853028407140320000;***wrong
Debug d;=-40459740278783232.0
d.d =3685302840714032e4;ok
Debug d;=36853028407140319000.0
;ok,but:36853028407140319<>3685302840714032
;so,maybe,*******************
;***for '=', best let:2251799813685248<4503599627370496
;              *******************
; 3685302840714032>2251799813685248
; 368530284071403<2251799813685248
; so,3685302840714032 best use:
d.d=368530284071403e1
Debug d;=3685302840714030.0
;368530284071403=368530284071403
;*********
;maybe, f.f for '=', ***best let:4194304<8388608

[gurj]

or best use ValD()

Code: Select all

g.d=ValD("36853028994071450280000000")
Debug g;=36853028994071449000000000.0
g.d=ValD("3.685302899407145028e-8")
Debug g;=0.000000036853028994071449

[said]

There is certainly a limitation/bug here .... the use of ValD() or scientific form (a.e-n) is just a work around!

[gurj]

maybe has limitation, but no bug:

Code: Select all

;3685302840714032<4503599627370496,so:
d.d=3685302840714032;this is true '='
;but 3685302840714032e4>4503599627370496,so:
d.d =3685302840714032e4;
Debug d;=36853028407140319000.0
;so 36853028407140319<>3685302840714032;
;because:
;3685302840714032e4=64 bit:
;1111111101110000010000100001010010100000111110101000001100000000
;11111111011100000100001000010100101000001111101010000011= 56 bit >52 bit