View Single Post
Old 09-09-2009, 12:25 PM   #22
liteon
Human being with feelings
 
liteon's Avatar
 
Join Date: Apr 2008
Posts: 510
Default precision

for evaluation there are 5 decimal places of precision.
Code:
@sample
a = 5*10^-5; //0.00005
b = 4*10^-5; //0.00004
//(a == b) = 1;
a = 5*10^-4; //0.0005
b = 4*10^-4; //0.0004
//(a == b) = 0;
justin suggests:
Quote:
Originally Posted by Justin
Note (for the docs), if you need more accuracy, you could do a*1000000 == b*1000000
-----
normally the accuracy of mathematical identities is dependent from the implemented functions (if are based on standards such as IEEE) and also very importantly the precision of the format.

evaluating the identity x=exp(log(x)) in c:
Code:
x is 5 (single or double) 
y=exp(log(x))
* for single precision:
x=5.0, y=5.000000000000000000000
(x==y)=true
* for double precision:
x=5.0, y=4.999999999999999100000
(x==y)=false
in js (double precision):
Code:
x=5;
y=exp(log(x));
//x 5.000000...
//y 5.000000... <- js debugger rounds value
x*=10^7;
y*=10^7;
//x 50000000.000000...
//y 49999999.999999...
assuming that the used mathematical functions are standardized this a trick that can be used to determine the precision in programming languages with less strict variable type definitions.

---
lubomir
liteon is offline   Reply With Quote