Why can't I automatically get an integer from this simple function?

I have a simple function defined as follows,

a = 0.4;

b = 120;

c = 120;

index = 2*a*b*(c-2*a*c) + 2*a*c*(b-2*a*b) + 4*a*c*a*b;

The variable index is then used as an index number of elements in a vector, thus should be an integer. However, the above function gives me index = 1.382400000000000e+04 which is double instead of 13824 as an integer, and I cannot used it as index in a vector. If I set a = 0.3 or 0.5 this will not occur. a is actually varied from 0.1 to 0.5 in a step of 0.1. Any idea why this would occur for a = 0.4? Thank you in advance!

[Steven Lord]

In theory, 0.4 is exactly four-tenths.

In practice, you can't exactly represent four-tenths as a double precision number, much like you can't exactly represent 1/3 as a finite string of digits. [ doesn't count. The over the 3 isn't a digit. No, it's not just a 1 on its side.] 0.4 is close to four-tenths.

Answer 1

[Star Strider]

You have encountered floating-point approximation error. See Floating-Point Numbers for an extended discussion.

The difference is vanishingly small:

a = 0.4;
b = 120;
c = 120;
index = 2*a*b*(c-2*a*c) + 2*a*c*(b-2*a*b) + 4*a*c*a*b;

so that:

d = index - fix(index)
d =
     1.818989403545856e-12

The simple solution is to use fix, round, or similare functions to return an integer result.

[Zhelong]

Thank you for the reply! It works!

[Star Strider]

As always, my pleasure!