November 5, 2008
@ 02:15 PM
Comments [2]
Exercise Caution When Using Floating Point Numbers

I understand floating point numbers can be somewhat imprecise but I'm a bit bothered today when I see the following evaluates to false:

   1: float f = .16f;
   2: double d = .16d;
   3:  
   4: Assert.That(f, Is.EqualTo(d))

Annoyingly I wanted to see what the two values are:

   1: Console.WriteLine((double)f)
   2: Console.WriteLine(d);

produces the following respectively:

.159999996423721

.16

What bothers me about this, and I'm hoping someone can eloquently explain this, the MSDN docs say this should be implicitly converted.  From the MSDN doc article on implicit numeric conversions:

  • From float to double.

Float is 32-bit and double is 64-bit, so why can't the float fit nicely inside the address space for the double?

Hell, if you're not going to respect the integrity of the number what's the point of the implicit conversion?


 
Categories: Help | Rant

« Test First or Test Driven, Who Cares? | Home | Iowa Code Camp 2 - In Review »
Monday, November 10, 2008 7:54:18 PM (Central Standard Time, UTC-06:00)
"Float is 32-bit and double is 64-bit, so why can't the float fit nicely inside the address space for the double?"
It actually does. The real problem is that .16f doesn't fit nicely inside the address space for a float. To get a feel for what is going on, add a very small number to both f and d. For example:
f += 0.00000001f;
d += 0.00000001d;
Now you will see this:
0.1600000011324883
0.160000001
If you add System.WriteLine(f) to both sets of outputs, you will see 0.16 both times, which clearly illustrates the rounding that takes place when converting the imprecise float representation of 0.16 to a human readable form, while the (double)f conversion retains the exact value of the float.
dave-ilsw
Monday, November 10, 2008 7:56:14 PM (Central Standard Time, UTC-06:00)
P.S. Your comment box does not work in Google's Chrome browser, giving the following error message after clicking on Save Comment:

An error has been encountered while processing the page. We have logged the error condition and are working to correct the problem. We apologize for any inconvenience.
dave-ilsw
Comments are closed.