필터 지우기
필터 지우기

Very easy issue with division

조회 수: 1 (최근 30일)
Jared
Jared 2011년 11월 10일
I'm sure this is something very simple, but I've been messing around and can't figure it out...
I read in a value and store it as a 1x1 int32. Say the variable is resolution and its value is 100,000.
resolution/1e6
ans=0
No. But it I do:
x=100000;
x/1e6
and=0.1
Which is correct. Something is odd and I can't figure it out.

답변 (1개)

bym
bym 2011년 11월 10일
do a
whos(resolution)
whos(x)
You will find that x is a double and resolution is an int32. Since resolution/1e6 is cast as an int32 is comes out 0

카테고리

Help CenterFile Exchange에서 Graphics Performance에 대해 자세히 알아보기

태그

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by