'fix' function misbehaving

Hi everyone,

I have a table of data.

The first column of the table reads how many heartbeats there have been. It's a number that increases with each row by 1. So I have 9 beats, hence 9 rows, reading 1 through 9.

Outside the table, I have a variable 'Z' calculated from two other variables 'X' and 'Y' which I previously specify.

X and Y are variables from some simulations I have been doing. X is dependant on the value of Y, but Y is held as a constant so that shouldn't matter.

X = 38.2

Y= 25

Z=X*Y, so Z=955.000. Simple enough.

Now, sometimes Z is not a whole number.

For example, one of my simulations had X=19.1000 and Y=25, giving a value of Z = 477.5000

For the next column of my table I want to multiply the heart-beat number by the integer part of Z.

Using the 'fix' function, Z=fix(Z), works perfecly when Z=477.5000. Z returns 477.

However when Z=955.0000, fix(Z) returns 954.

When I type out "Z=955.0000" in place of getting Z from the previous calculation then fix(Z) returns 955.

The same issue is ocuring with the floor funcition, however the round function returns 955.

Any idea what's going wrong? Thanks

Answer 1

[Star Strider]

Temporarily set the format to long, and then also do:

dZ = Z - 955

and check the result. This is most likely the result of rounding in the display, and floating-point approximation error. See Floating-Point Numbers for an extended discussion.

[Charlie Bright]

Hi Star,

Thanks for getting back to me.

The long format helped me see that the X value of 38.2 was actually 38.1999 and therefore Z's true value is 954.9999 hence the rounding.

I'll have to have a think about how that changes my calculations and have a play with the code.

Thanks again!

[Star Strider]

My pleasure!

If my Answer helped you solve your problem, please Accept it!

.