필터 지우기
필터 지우기

calculation error or double precision limit?

조회 수: 1 (최근 30일)
Andreas
Andreas 2015년 9월 17일
편집: Stephen23 2015년 9월 17일
Hello,
I was wondering about the result when calculating following simple formula:
K>> 0.067 * 1e9
ans =
6.7000e+07
Following gives the correct result (integer):
K>> 0.067 * 1e8 * 1e1
ans =
67000000
Where is the difference in the calcualation?
Is it an error or a double precision limit?

답변 (1개)

Stephen23
Stephen23 2015년 9월 17일
편집: Stephen23 2015년 9월 17일
You should download this and give it a try with the same examples as above:
Also have a look at the intermediate values, such as 0.067 * 1e8. This shows that there are a few digits that creep in after the end of the double precision limit (fifteen digits).

카테고리

Help CenterFile Exchange에서 Logical에 대해 자세히 알아보기

제품

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by