Bug in rounding functions? Who can explain? :)

조회 수: 1 (최근 30일)
Pavel
Pavel 2015년 11월 11일
편집: Stephen23 2015년 11월 11일
>> format long
>> 4.1*808e6
ans =
3.312800000000000e+09
>> fix(4.1*808e6)
ans =
3.312799999000000e+09
>> fix(4*808e6 + .1*808e6)
ans =
3.312800000000000e+09

채택된 답변

Walter Roberson
Walter Roberson 2015년 11월 11일
>> 4.1*808e6 - 3312800000
ans =
-4.76837158203125e-07
That is, due to floating point roundoff, 4.1*808e6 falls just short of an integer. fix() is the same as floor() for positive numbers so fix() removes the fraction after 3312799999 that is not quite 1.0, exactly like how floor(1.99999999) is 1.0 not 2.0

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 Logical에 대해 자세히 알아보기

태그

제품

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by