Multinomial Models for Nominal Responses

The outcome of a response variable might be one of a restricted set of possible values. If there are only two possible outcomes, such as a yes or no answer to a question, these responses are called binary responses. If there are multiple outcomes, then they are called polytomous responses. Some examples include the degree of a disease (mild, medium, severe), preferred districts to live in a city, and so on. When the response variable is nominal, there is no natural order among the response variable categories. Nominal response models explain and predict the probability that an observation is in each category of a categorical response variable.

A nominal response model, also called a multinomial logit model, is one of several natural extensions of the binary logit model. A multinomial logit model explains the relative risk of an observation being in one category versus being in the reference category k, using a linear combination of predictor variables. Consequently, the probability of each outcome is expressed as a nonlinear function of p predictor variables. This multinomial model for nominal responses is sometimes called softmax regression. You can create a MultinomialRegression object by using the fitmnr function. By default, fitmnr specifies separate intercepts and slopes among categories, and uses the logit link function. You cannot specify a different link function for nominal responses.

The multinomial logit model is

$$\begin{array}{l}\ln \left(\frac{{\pi }_1}{{\pi }_k}\right)={\alpha }_1+{\beta }_{11}{X}_1+{\beta }_{12}{X}_2+\cdots +{\beta }_{1p}{X}_p,\\ \ln \left(\frac{{\pi }_2}{{\pi }_k}\right)={\alpha }_2+{\beta }_{21}{X}_1+{\beta }_{22}{X}_2+\cdots +{\beta }_{2p}{X}_p,\\ ⋮\\ \ln \left(\frac{{\pi }_{k-1}}{{\pi }_k}\right)={\alpha }_{(k-1)}+{\beta }_{(k-1)1}{X}_1+{\beta }_{(k-1)2}{X}_2+\cdots +{\beta }_{(k-1)p}{X}_p,\end{array}$$

where π_j = P(y = j) is the probability of an outcome being in category j, k is the number of response categories, and p is the number of predictor variables. Theoretically, any category can be the reference category, but fitmnr selects the final category k as the reference category. Therefore, fitmnr assumes that the coefficients of the kth category are zero. The total of j – 1 equations are solved simultaneously to estimate the coefficients. fitmnr uses the iteratively weighted least-squares algorithm to find the maximum likelihood estimates.

The coefficients in the model express the effects of the predictor variables on the relative risk or the log odds of being in category j versus the reference category, here k. For example, the coefficient β₂₃ indicates that the probability of the response variable being in category 2 compared to the probability of being in category k increases exp(β₂₃) times for each unit increase in X₃, given all else is held constant. Or it indicates that the relative log odds of the response variable being category 2 versus in category k increases β₂₃ times with a one-unit increase in X₃, given all else equal.

Based on the nominal response model, and the assumption that the coefficients for the last category are zero, the probability of being in each category is

$${\pi }_j=P\left(y=j\right)=\frac{e^{{\alpha }_j+{\displaystyle \sum _{l=1}^p{\beta }_{jl}{x}_l}}}{1+{\displaystyle \sum _{j=1}^{k-1}{e}^{{\alpha }_j+{\displaystyle \sum _{l=1}^p{\beta }_{jl}{x}_l}}}},j=1,\cdots ,k-1.$$

The probability of the kth category becomes

$${\pi }_k=P\left(y=k\right)=\frac{1}{1+{\displaystyle \sum _{j=1}^{k-1}{e}^{{\alpha }_j+{\displaystyle \sum _{l=1}^p{\beta }_{jl}{x}_l}}}},$$

which is simply equal to 1 – π₁ – π₂ – ... – π_k–1.

After estimating the model coefficients by using fitmnr to create a MultinomialRegression model object, you can estimate the category probabilities by using predict. This function accepts the MultinomialRegression model object returned by fitmnr, and estimates the category labels, categorical probabilities, and confidence bounds for each categorical probability. You can specify whether predict returns category, cumulative, or conditional probabilities using the ProbabilityType name-value argument.

References

[1] McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.

[2] Long, J. S. Regression Models for Categorical and Limited Dependent Variables. Sage Publications, 1997.

[3] Dobson, A. J., and A. G. Barnett. An Introduction to Generalized Linear Models. Chapman and Hall/CRC. Taylor & Francis Group, 2008.