What happened to the Messenger Plus! forums on msghelp.net?
Shoutbox » MsgHelp Archive » Messenger Plus! for Live Messenger » Scripting » [?] Is my math (a bit) out here?

[?] Is my math (a bit) out here?
Author: Message:
Spunky
Former Super Mod
*****

Avatar

Posts: 3658
Reputation: 61
36 / Male / Flag
Joined: Aug 2006
O.P. [?] Is my math (a bit) out here?
I have to add 0.1 to a number and keep doing it... but I get some wierd results =/

quote:
Script has been stopped
Script is starting
Script is now loaded and ready
(!) 0.1
(!) 0.2
(!) 0.30000000000000004
(!) 0.4
(!) 0.5
(!) 0.6
(!) 0.7
(!) 0.7999999999999999
(!) 0.8999999999999999
(!) 0.9999999999999999
(!) 1.0999999999999998
(!) 1.2


Why is this? :s


EDIT:

A for loop causes this:

code:
    for(var x=0;x<10;x+=y){
        Debug.Trace(x);
    }


quote:
Script has been stopped
Script is starting
Script is now loaded and ready
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
4
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
5.99999999999999
6.09999999999999
6.19999999999999
6.29999999999999
6.39999999999999
6.49999999999999
6.59999999999999
6.69999999999999
6.79999999999999
6.89999999999999
6.99999999999999
7.09999999999999
7.19999999999999
7.29999999999999
7.39999999999999
7.49999999999999
7.59999999999999
7.69999999999999
7.79999999999999
7.89999999999999
7.99999999999999
8.09999999999999
8.19999999999999
8.29999999999999
8.39999999999999
8.49999999999999
8.59999999999999
8.69999999999999
8.79999999999998
8.89999999999998
8.99999999999998
9.09999999999998
9.19999999999998
9.29999999999998
9.39999999999998
9.49999999999998
9.59999999999998
9.69999999999998
9.79999999999998
9.89999999999998
9.99999999999998

This post was edited on 02-26-2008 at 11:35 AM by Spunky.
<Eljay> "Problems encountered: shit blew up" :zippy:
02-26-2008 11:30 AM
Profile PM Find Quote Report
Jesus
Scripting Contest Winner
****

Avatar
Koffie, my cat ;)

Posts: 623
Reputation: 15
38 / Male / Flag
Joined: Jul 2005
RE: [?] Is my math (a bit) out here?
how is y made up?
maybe y != 0.1 but y == 0.0999999999999999999999
try a Debug.Trace(1000000000000 * y)

just an idea, don't know if it will do any good ;)

edit: though that won't explain the
quote:
(!) 0.30000000000000004

This post was edited on 02-26-2008 at 12:02 PM by Jesus.
Man is least himself when he is in his own person. Give him a mask and he will tell you the truth. (Oscar Wilde)
02-26-2008 11:59 AM
Profile PM Find Quote Report
Spunky
Former Super Mod
*****

Avatar

Posts: 3658
Reputation: 61
36 / Male / Flag
Joined: Aug 2006
O.P. RE: [?] Is my math (a bit) out here?
y is definately 0.1

I'm testing the for loop with static variables
<Eljay> "Problems encountered: shit blew up" :zippy:
02-26-2008 12:04 PM
Profile PM Find Quote Report
Adeptus
Senior Member
****


Posts: 732
Reputation: 40
Joined: Oct 2005
RE: [?] Is my math (a bit) out here?
http://en.wikipedia.org/wiki/Floating_point

This is because whatever programming language and data type you use treats your numbers as IEEE floats.  Floats are not exact by their nature; the linked article should explain why not.  If you are going to use floats you should remember to code your program logic to never do exact comparisons -- only "greater than" or "less than". 

If this is not acceptable, there are a few options.  Your programming language may offer a built-in BCD (binary coded decimal) type or there may be a suitable free or commercial class available.  If you are doing fixed point math, such as dealing with money, you can simply use integer types and represent the values using the smallest base unit (e.g. 115 cents instead of $1.15).

This post was edited on 02-26-2008 at 12:10 PM by Adeptus.
02-26-2008 12:10 PM
Profile E-Mail PM Find Quote Report
Spunky
Former Super Mod
*****

Avatar

Posts: 3658
Reputation: 61
36 / Male / Flag
Joined: Aug 2006
O.P. RE: [?] Is my math (a bit) out here?
quote:
Originally posted by Adeptus
If you are doing fixed point math, such as dealing with money, you can simply use integer types and represent the values using the smallest base unit (e.g. 115 cents instead of $1.15).

My best bet I think (Y) ty
<Eljay> "Problems encountered: shit blew up" :zippy:
02-26-2008 12:27 PM
Profile PM Find Quote Report
Patchou
Messenger Plus! Creator
*****

Avatar

Posts: 8607
Reputation: 201
43 / Male / Flag
Joined: Apr 2002
RE: [?] Is my math (a bit) out here?
I confirm what Adeptus said.

This post was edited on 02-26-2008 at 10:24 PM by Patchou.
[Image: signature2.gif]
02-26-2008 10:23 PM
Profile PM Web Find Quote Report
Spunky
Former Super Mod
*****

Avatar

Posts: 3658
Reputation: 61
36 / Male / Flag
Joined: Aug 2006
O.P. RE: [?] Is my math (a bit) out here?
I found away around it now, thanks for everyones help
<Eljay> "Problems encountered: shit blew up" :zippy:
02-26-2008 11:02 PM
Profile PM Find Quote Report
« Next Oldest Return to Top Next Newest »


Threaded Mode | Linear Mode
View a Printable Version
Send this Thread to a Friend
Subscribe | Add to Favorites
Rate This Thread:

Forum Jump:

Forum Rules:
You cannot post new threads
You cannot post replies
You cannot post attachments
You can edit your posts
HTML is Off
myCode is On
Smilies are On
[img] Code is On