calculation error or double precision limit?
조회 수: 1 (최근 30일)
이전 댓글 표시
Hello,
I was wondering about the result when calculating following simple formula:
K>> 0.067 * 1e9
ans =
6.7000e+07
Following gives the correct result (integer):
K>> 0.067 * 1e8 * 1e1
ans =
67000000
Where is the difference in the calcualation?
Is it an error or a double precision limit?
댓글 수: 0
답변 (1개)
Stephen23
2015년 9월 17일
편집: Stephen23
2015년 9월 17일
You should download this and give it a try with the same examples as above:
Also have a look at the intermediate values, such as 0.067 * 1e8. This shows that there are a few digits that creep in after the end of the double precision limit (fifteen digits).
댓글 수: 0
참고 항목
카테고리
Help Center 및 File Exchange에서 Logical에 대해 자세히 알아보기
제품
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!