ROC curve

Understanding ROC curves and how to use them is a must have skill for anyone who practice machine learning in their career/research. And here I am taking my chance to explain ROC Curves from layman perspective.

Receiver operating characteristic aka ROC curve or ROC chart is a graphical representation for evaluating model performances. Generally, ROC curves are used to compare multiple classification models to identify which model can produce the best result over the other models.

Technically ROC Graph is nothing but a plot between TPR and FPR. TPR runs along the y-axis and FPR runs along the x-axis. TPR (True positive rate) aka sensitivity or Recall. FPR (False positive rate) is equivalent to what is known as 1-specificity.

Concept of TPR and FPR can easily be understood by this simple example, say you have a classification model to classify good customers from bad customers, TPR is nothing but a measure that talks about predicting good customer as a good customer and FPR is measure that talks about bad customers predicted as a good customer. In practice, we expect a good model to have high TPR and less FPR value.

Let’s say we have two models, M1 and M2 for the above example to classify good customers from bad customers. Our goal is to find which model M1 or M2 can give the best result – for this we have to draw ROC curves.

Drawing ROC curves starts with computing confusion matrix. Let’s say for model M1 our initial cut off threshold be 0.5, when resultant hypothesis value is less than 0.5, we will consider model predicts the customer as bad customer otherwise as good customers. Now we will repeat the same process with cut off threshold be 0.6, 0.8 etc for the model M1. Outcome of model M1 results is recorded for confusion matrix computation. Similarly, with varying cut off the threshold outcome of model M2 result is also recorded for confusion matrix computation.

Formula for TPR and FPR

TPR = TP / TP + FN

FPR = FP / FP + TN

From the confusion matrix about, we have all information required to calculate TPR and FPR using above formula. Once this is done, computed TPR and FPR values for varying cut off threshold are plotted onto a graph for model M1 and M2.

Now it’s time to use the graph and identify the best model. For this we look for AUC indicator in the graph i.e. we look for a model that has higher area under the cure. In practice a model that has higher area under the curve is considered to be the best model for best results.