help mlogitroc

mlogitroc - Multiclass ROC Curves and AUC from Multinomial Logistic Regressio > n


mlogitroc classvar var1 var2 ... varp [if ...]


mlogitroc generates multiclass ROC curves for classification accuracy based on multinomial logistic regression using mlogit. The algorithm begins by running mlogit B=100 times using bootstrapped records for each run while the original class labels are intact. Class prediction is then performed for records not sampled during bootstrapping, and accuracy for the left out records is determined as the fraction of correct class membership predictions. This results in B=100 realizations of the accuracy for the alternative distribution. Next, B=100 mlogit runs are made again, but this time after shuffling class labels of all records prior to modeling, which results in B=100 realizations of null accuracy. Smoothed probability distributions are obtained for the B=100 alternative and null accuracy values using kernel density estimation (KDE, Gaussian kernel) to obtain 100 smoothed realizations for alternative and null accuracy. The false positive rate (FPR), true positive rate (TPR), and area under the curve (AUC) are determined from the smooth pdfs derived from KDE. Twoway scatter plots of the smoothed pdfs are constructed, followed by plotting the ROC curve.

The default number of maximum iterations for reaching convergence is 20, which should be adequate for most runs. A smaller value of maxiter=20 helps terminate an mlogit run that will not converge - which may be common when using shuffled class labels to generate null accuracy values.

Note: This algorithm was not designed for stepwise modeling, and assumes that the user(s) pre-select the informative input variables prior to modeling.

Input variables

classvar must contain positive integer-based class labels such as 1,2,...,C, where C is the total number of classes.

var1 var2 ... varp are the p independent variables.


--------------------------------------------------------------------------- . mlogitroc y x1 x2 x3 . mlogitroc outcome age gender treatment1yes2n0 . mlogitroc class celltype time concentration heattemp . mlogitroc cluster weight bmi gene1exp gene2exp . mlogitroc outcome age treatment1yes2n0 if gender==2 . mlogitroc cluster weight bmi gene1exp gene2exp if age>40 ---------------------------------------------------------------------------


This algorithm for multiclass ROC curve generation was introduced and used for the following papers:

Peterson, L.E., Hoogeveen, R.C., Pownall, H.J., Morrisett, J.D. Classification Analysis of Surface-enhanced Laser Desorption/Ionization Mass Spectral Serum Profiles for Prostate Cancer. International Joint Conference on Neural Networks (IJCNN06), July 2006. IEEE Press, Piscataway(NJ), 2006.

Peterson, L.E., Coleman, M.A. Machine learning-based receiver operating characteristic (ROC) curves for crisp and fuzzy classification of DNA microarrays in cancer research. Int. J. of Approximate Reasoning. 47: 17-36, 2008.

Also see

Help: [R] mlogit, [R] kdensity, [R] summarize, [G] twoway