error decimal aproximation, extra decimal

조회 수: 3 (최근 30일)
Fernando Ramirez Huaca
Fernando Ramirez Huaca 2020년 5월 11일
편집: James Tursa 2020년 5월 11일
why in Matlab 0.94-1=-0.0600000000000001? Why is there an extra decimal 1?

답변 (1개)

James Tursa
James Tursa 2020년 5월 11일
편집: James Tursa 2020년 5월 11일
Welcome to the world of floating point arithmetic! See this link:
Bottom line ... 0.94 cannot be represented exactly in IEEE double precision. That trailing 1 is actually part of more digits in an exact floating point binary to decimal conversion:
>> fprintf('%.60f\n',0.94-1)
-0.060000000000000053290705182007513940334320068359375000000000
The 0.94 all by itself converted into decimal is:
>> fprintf('%.60f\n',0.94)
0.939999999999999946709294817992486059665679931640625000000000

카테고리

Help CenterFile Exchange에서 Logical에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by