Least-squares method in normfit with censoring?

Can someone explain to me what the normfit function is doing with censoring? What is censoring? I am looking at the code of the function normfit.m and I am not really sure what exactly it is doing. Thanks!

Answer 1

[Tom Lane]

If I test a bunch of lightbulbs for a month and see how long they last, maybe half of them will burn out and I will be able to observe their lifetimes exactly. For the others that don't burn out, I don't know what their lifetime is. I only know that they are all larger than one month. Their lifetimes are said to be censored at one month.

It's possible to compute the likelihood for this scenario. The normfit function estimates the parameters by maximizing the likelihood.

Your heading mentions least squares. The normfit function takes a single sample of data, not x/y data, so it's not typical to refer to what it does as least squares.

[Marmi Afrin]

Hi Tom Thanks for your response. I was looking at normfit.m function and there it says in comments that it estimates parameters from the "least squares" method. I am not quite sure how it's exactly doing that

[Tom Lane]

Ah, I see. This least squares fit is basically a fit to points on a normal probability plot. However:

1. It's only done in the case of censoring.
2. It's only done to get starting values.

The function uses the result of this least squares fit to start off an optimizer that tries to maximize the likelihood function.

So in my example, we can't compute the mean lifetime because we haven't observed all the lifetimes. We can maximize the likelihood function. The least squares trick just gets us started in that direction.

[Marmi Afrin]

I am basically trying to fit a gaussian to my histogram and I need some help in writing a code that would do that using least squares from scratch. I tried looking at normfit but I am still not sure what exactly it is doing. When I do [mu,sigma]=normfit(y) and plot the gaussian from using the mu and sigma in normpdf normalized by the area (no of elements*binwidth) it still doesn't give me the right fit for some of my data. I was told it's because my histogram doesnt have error bars and there error bars are needed to do a correct fit...

[Tom Lane]

Let's back up. This histfit function can fit a normal distribution to data and overlay the density on the histogram. Is that what you need? This function doesn't return the fit parameters, but you could look inside the function and see what it does.

If this "doesn't give the right fit" it would be helpful to have more details on that. Perhaps a normal fit doesn't work well, or it's too sensitive to outliers, or it doesn't capture outliers, or ....?