http://en.wikipedia.org/wiki/Floating_point
This is because whatever programming language and data type you use treats your numbers as IEEE floats. Floats are not exact by their nature; the linked article should explain why not. If you are going to use floats you should remember to code your program logic to never do exact comparisons -- only "greater than" or "less than".
If this is not acceptable, there are a few options. Your programming language may offer a built-in BCD (binary coded decimal) type or there may be a suitable free or commercial class available. If you are doing fixed point math, such as dealing with money, you can simply use integer types and represent the values using the smallest base unit (e.g. 115 cents instead of $1.15).
