Why is there a value mismatch in my data sets?

I've run across in issue where two values that are supposed to be equivalent are subtracted and their result doesn't yield 0, but rather an extremely small number. I'm not sure if this is a known issue with data typing and precision. Interestingly enough when viewing the numbers in format LONGENG, they also appear to be the same number. This isn't too much of a problem most of the time, but when relational operators are used (<, >, <=, ...) this extremely small number and 0 relate as expected i.e. small # == 0 returns 0.

Edit:

So every time I want to manipulate numbers in MATLAB I have to fix or round them to some precision? Which would also mean changing the units of any values in scientific notation i.e. fix(10.123E-9) = 0

Also

"‘small # == 0 returns 0’ is inaccurate."

"relate as expected i.e. small # == 0 returns 0."

Relational operators don't have a tolerance (at least to my knowledge) so I would say that 'small# == 0' should return 0 because I'm using == rather than ~= and small# is NOT equal to 0. I;ve attached a screen shot.

This becomes problematic when using relational operators after some algebra has been done on the numbers in question and equations that should yield 0 now yield small#

Answer 1

[Star Strider]

First, the floating point approximation error has been around as long as floating point arithmetic has. You have to use a range of values if you want to compare two floating point numbers, or use one of the truncation operators (such as fix, round, etc. with the appropriate multiplication and division if necessary) to truncate them to integers before you compare them.

Second your interpretation of the result of ‘small # == 0 returns 0’ is inaccurate. That operation and result mean that ‘small #’ and 0 are not equal. If they were, the result would be 1.

Answer 2

[Roger Stafford]

It is entirely possible that two matlab 'double' numbers can have the same display using "format longeng" and yet be different. Here is an example using "format long" (which is at least as accurate as "format longeng".)

 x1 = pi;
 x2 = x1*(1+2^(-52));  % Add 1 to the least bit
 format long
 x1 = 3.14159265358979  % They look alike in this display
 x2 = 3.14159265358979
 fprintf('x1 = %20.18f\nx2 = %20.18f\n',x1,x2)
 x1 = 3.141592653589793116  % Their difference shows up using a higher accuracy display
 x2 = 3.141592653589794004
 x1 == x2
 ans = 0

[Bryan]

Thanks for the reply! I wasn't aware that MATLAB had precision beyond that display format.

[Roger Stafford]

Be aware that the precision of matlab's double is determined solely by its stored binary floating point form which has 53 bits in its significand (mantissa). That, together with its 11-bit exponent and 1-bit sign, are the only things that are used in its computations. What you see in the displays are decimal representations of that binary form and are necessarily almost always approximations to its exact value. As you will note above, I had to use %20.18f in 'fprintf' to reveal the difference between two very close values.