Fixing biased random number generation
조회 수: 6 (최근 30일)
이전 댓글 표시
Hello. I am currently generating a dataset for a machine learning model, however I am having trouble with one of the variables.
First I generate 3000 values of a variable H between 100 and 1000.
Then I need a variable a that can have any value between 10% and 90% of the corresponding value of H. The problem is that when I look at the histogram of a it is clearly biased and I can´t find out why.
How could I generate an unbiased a?
Here is a piece of the code I am currently using:
N = 3000;
H = 100 + (1000 - 100) * rand(N, 1);
a_min = 0.1 * H;
a_max = 0.9 * H;
a = a_min + (a_max - a_min) .* rand(N, 1);
This is the histogram I am getting:
Some additional context:
The goal is to develop a machine learning (ML) model that is able to predict stress concentration factors (SCF) in a sheet with a hole in it. This has already been solved analitically, but I am doing this as an exercise. The SCF depends on H and a, which are geometrical constraints, they define the width of the plate and position of the hole respectively. So you can see why a is dependant on the values of H. The range 0.1*H to 0.9*H is just to make sure that hole won´t be located outside of the plate and that hole has a reasonable size. Of course there are other geometrical constriants and additional steps before getting to the ML part, but I believe this is enough for this post.
댓글 수: 2
Bruno Luong
2024년 11월 4일
편집: Bruno Luong
2024년 11월 4일
The constraint
% 0.1 H <= a <= 0.9 H
Is conditional probability so a must be "biased". It is simply an innevitable fact.
This is showed in the rejection random method code started from a uniform unconditional probabilty:
N = 3000;
H = 100 + (1000 - 100) * rand(N, 1);
clear a
for k = N:-1:1
a(k) = randone(H(k), 1000);
end
histogram(a)
function a = randone(H, maxa)
L = 0.1*H;
U = 0.9*H;
while true
a10 = maxa*rand(1,10);
k = find(a10 > L & a10 < U, 1, 'first');
if ~isempty(k)
a = a10(k);
return
end
end
end
채택된 답변
埃博拉酱
2024년 11월 3일
편집: 埃博拉酱
2024년 11월 4일
It's a math problem. If you try to take a uniform random variable H as an upper bound on another uniform random variable a, then a is mathematically impossible to still be uniformly distributed. You have to rethink your whole problem.
What on earth are you really trying to do?
Update 20241104
Based on your description, I think it is likely that H should not be uniformly distributed. Since your H seems to correspond to a certain type of item that exists in reality, the probability distribution of H should also refer to that item in reality (usually it should be β or other normal-like distribution).
Secondly, from your question, there is no need to pursue that a must be uniformly distributed. Since a cannot be greater than H, the probability distribution of a should be skewed to the smaller side.
추가 답변 (2개)
Bruno Luong
2024년 11월 3일
% a_max .* rand(N, 1)
is a product of 2 independent uniform variables, it is NOT a uniform random variable as you expect since a_max is not constant.
x1 = rand(1,10000);
x2 = rand(1,10000);
x1Xx2 = x1.*x2;
histogram(x1Xx2)
Ths alone makes your "intuiltion" falling apart.
댓글 수: 3
Bruno Luong
2024년 11월 3일
편집: Bruno Luong
2024년 11월 3일
In which way you need a to be uniform in your ML problem? Does it really matter? Why?
Looks like what you want is not possible (What on earth are you really trying to do? as some of use keep asking)
May be what you call "bias" is simply a wrong expectation.
One thing people always want is that conditional probabimity have the same distribution as unconditional one. This assumption is always wrong.
John D'Errico
2024년 11월 3일
편집: John D'Errico
2024년 11월 3일
You should recognize that the result of this operation will not be uniformly distributed. Does that matter? Or is it just a surprise to you, that starting with uniform random variables, you expect the result to also be uniform? Is there a reason why you need the result to be uniform?
참고 항목
카테고리
Help Center 및 File Exchange에서 Develop Apps Using App Designer에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!