Xtensity
2010-06-29 14:49:30
How come in any math script I try to make all my extended numbers are rounded to the nearest tenth.
Also how come I can not assign decimals such as .03 or .97 to extended variables.
I'm trying to make a script to calculate extremely small dilutions that can go to the hundredth or thousandths decimal place. I know the math and everything, the only problem is scar rounding my decimals(from calculations such as 3/100 which shows 0 instead of .03), and I can't assign decimals such as x:=.03…. etc
I know it is possible in scar, since functions like ArcTan can return decimal numbers with up to 15-16 places, which it is returning an extended number and not rounding it like it does with my own calculations.
What am I doing wrong/not doing?
Also how come I can not assign decimals such as .03 or .97 to extended variables.
I'm trying to make a script to calculate extremely small dilutions that can go to the hundredth or thousandths decimal place. I know the math and everything, the only problem is scar rounding my decimals(from calculations such as 3/100 which shows 0 instead of .03), and I can't assign decimals such as x:=.03…. etc
I know it is possible in scar, since functions like ArcTan can return decimal numbers with up to 15-16 places, which it is returning an extended number and not rounding it like it does with my own calculations.
What am I doing wrong/not doing?