Related Links:
Receiver Operator Characteristics (ROC) Explained Page
Introduction
Example
References
This page supports the program from the . The algorithm is derived from publications
listed in the references panel
The Need for Curve Fitting the Receiver Operator Characteristic (ROC) Curve
The underlying assumptions of the ROC curve is that the Outcome is binary, but the Test is a continuous variable. In
establishing the ROC from a set of reference data, it is assumed that the sample size provided is large enough, and the
granularity of the values in the data set is fine enough, to produce a reproducible curve.
The reference data set unfortunately often failed these assumptions. A major problem is that test measurements are often
rounded to clinically meaningful intervals, so that the granularity of the data is coarse, and changes occurred in a
series of steps rather than in a smooth curve.
Furthermore, with rounding of values, often there are multiple cases with the same test value, and to include or exclude
values at the cut off makes a large difference to the subsequent use. Also the changes to the False Positive Rate (FPR) is not
always accompanied by any change to the True Positive Rate (TPR)
An example of this is shown in the plot to the left. If the cut off for decision making is 155 (the black line), data points
can be clearly separated into above or below that value. However, if the cut off value is set at 154, there are 3 data points
in the Outcome Positive group and 2 in the Outcome Negative group that have this exact value. Whether these are considered above
the cut off, below the cut off, or excluded from consideration, will have a large impact on subsequent results produced.
The resulting ROC plot would be as the black line in the plot to the right.
Curve fitting would result in a ROC curve that has the closest fit with the data, yet allows a continuous interpretation along
the whole range of the Test values, as shown in the red line in the plot to the right.
Test Value  FPR  TPR 
162.5  0.96  1 
162  0.92  1 
161.5  0.84  1 
160  0.84  0.96 
159.5  0.76  0.96 
159  0.68  0.96 
158.5  0.48  0.92 
157.5  0.4  0.88 
157  0.36  0.88 
156.5  0.36  0.8 
156  0.24  0.68 
155.5  0.2  0.6 
154  0.12  0.48 
153.5  0.04  0.4 
153  0.04  0.36 
152.5  0  0.28 
151.5  0  0.24 
151  0  0.2 
150.5  0  0.16 
150  0  0.12 
149  0  0.08 
This example uses the default input data from the program in the . This set of the data,
in turn is obtained from producing a Receiver Operator Characteristics (ROC) curve from the first 2 columns of data in the
Receiver Operator Characteristics (ROC) Program Page
. The original data are computer generated and do not represent any real information.
We wished to establish maternal height (cms) as the Test to predict the need for Caesarean Section as the Outcome. Using
the first 2 columns of default example data from Receiver Operator Characteristics (ROC) Program Page
, We produced a table of False and True Positive
Rates. A subset of columns from that table is used as the default example in the for curve
fitting. The data is as shown in the table to the right. Test value is maternal height in cms, FPR and TPR are False and True
Positive Rates for that value.
Curve Fitting
Two curve fitting procedures are carried out by the program, both use the False Positive Rate as the independent variable.
Only data where both TPR and FPR are between 0 and 1 are included the curve fitting, so that the curve is a best fit of where
there are overlap in the test values between the two groups.
 Curve fitting for False Positive Rate.
 The independent variable (x) is the maternal height
 The y variable is the logit of the FPR, so that y = log(FPR(1FPR))
 The standard Regression analysis as in the Correlation and Regression Program Page
is carried out, so that the formula
y=a+bx is produced. In this example, a = 101.6295 and b = 0.6439
 The result is therefore Logit(FPR) = 101.6295 + (0.6439)(maternal height in cm), or
FPR = 1 / (1  exp((101.6295 + (0.6439)(maternal height in cm))))
 Curve fitting for True Positive Rate.
 The logit of FPR and TPR are firstly calculated, so that u = log(FPR(1FPR)) and v = Log(TPR(1TPR)
 The y (dependent) variable is then calculated as the Log Odds Ratio y = uv
 The x (independent variable is calculated as the Standard Error of the Log Odds Ratio x = u+v
 The standard Regression analysis as in the Correlation and Regression Program Page
is carried out, so that the formula
y=a+bx is produced. In this example, a = 2.2319 and b = 0.0445
 The formula to obtain TPR from FPR is TPR = 1 / (1 + exp(a / ( 1  b)) exp(log((1 FPR) / FPR) * ((1 + b) / (1  b))))
 The term (1 + exp(a / ( 1  b)) is a constant, and in this example = 0.118
 The term ((1 + b) / (1  b)) is also a constant, and in this example = 0.9148. Please note that b is changes in Standard
error, and in this curve fitting it contains mostly rounding errors, and is not significantly different from 0. As a
consequence, the second term ((1 + b) / (1  b)) is not significantly different from 1, and can be left out without
materially affecting the end results. However the program in the has included
this term.
 The result is therefore TPR = 1 / (1 + 0.118(exp(log((1FPR)/FPR)(0.9148))))
Results and their Interpretations
The program in the one plot and two tables. Using the default example input data,
the plot produced (black line and circular dots) is as shown to the left.
This plot shows the relationship between original data (black circular dots) and the curve fitted ROC curve (black line).
Please note that curve fitting used overlapping data from the two groups, and excluded the two cases with TPR=1, and the 6 cases
with FPR=0, so these data points do not fit closely to the fitted curve.
Two alternatives were tried but eventually not included in the algorithm.
 Curve fitting using only those data points where TPR>=0.5 and FPR<=0.5, which produced the green curve on the plot.
This was the option suggested by Moses et.al. (see references) for metaanalysis of prediction. Although this produced a
very good fit for data in the top left quadrant, it fitted poorly for the rest of the scale. This was therefore not used, as,
unlike metaanalysis, the current curve fit contains only a single set of data, and the relationship over the whole range
are relevant.
 Curve fitting using all data points, providing 0.0001 to replace 0, and 0.9999 to replace 1 to avoid the divide by zero
crash. This results in the red curve, a very good fit, but dominated by the extreme values, so that the fit is poor in
the range where the two groups overlap. This is therefore also not used as curve fitting for extreme values are not
necessary, and a good fit where the two groups overlap is the main priority of the exercise.
The result of the curve fitting is as shown above and to the right. The original data points are represented by the circular dots, the smooth curve is the curve fitted ROC line. The short straight line joins the coordinate of the original values to the values occupied on the fitted curve for each data point.
Data  Curve Fitted 
Test Value  FPR  TPR  FPR  TPR 
162.5  0.96  1  0.95  0.99 
162  0.92  1  0.94  0.99 
161.5  0.84  1  0.91  0.99 
160  0.84  0.96  0.8  0.97 
159.5  0.76  0.96  0.75  0.96 
159  0.68  0.96  0.68  0.94 
158.5  0.48  0.92  0.61  0.93 
157.5  0.4  0.88  0.45  0.87 
157  0.36  0.88  0.37  0.84 
156.5  0.36  0.8  0.3  0.8 
156  0.24  0.68  0.24  0.74 
155.5  0.2  0.6  0.18  0.68 
154  0.12  0.48  0.08  0.47 
153.5  0.04  0.4  0.06  0.4 
153  0.04  0.36  0.04  0.33 
152.5  0  0.28  0.03  0.27 
151.5  0  0.24  0.02  0.17 
151  0  0.2  0.01  0.13 
150.5  0  0.16  0.01  0.1 
150  0  0.12  0.01  0.08 
149  0  0.08  0  0.04 
Test Value  FPR  TPR  TNR  YI  TPR/TNR  TNR/TPR  LR+  LR 
150.69  0.01  0.11  0.99  0.10  0.11  8.81  11.24  0.90 
151.78  0.02  0.19  0.98  0.17  0.20  5.05  9.71  0.82 
152.43  0.03  0.26  0.97  0.23  0.27  3.72  8.68  0.76 
152.89  0.04  0.32  0.96  0.28  0.33  3.03  7.91  0.71 
153.25  0.05  0.36  0.95  0.31  0.38  2.61  7.29  0.67 
153.55  0.06  0.41  0.94  0.35  0.43  2.31  6.77  0.63 
153.81  0.07  0.44  0.93  0.37  0.48  2.10  6.33  0.60 
154.03  0.08  0.48  0.92  0.40  0.52  1.93  5.95  0.57 
154.23  0.09  0.51  0.91  0.42  0.56  1.80  5.61  0.54 
154.41  0.10  0.53  0.90  0.43  0.59  1.69  5.32  0.52 
154.58  0.11  0.56  0.89  0.45  0.62  1.60  5.05  0.50 
154.73  0.12  0.58  0.88  0.46  0.66  1.52  4.82  0.48 
154.87  0.13  0.60  0.87  0.47  0.69  1.45  4.60  0.46 
155.01  0.14  0.62  0.86  0.48  0.72  1.39  4.41  0.45 
155.13  0.15  0.63  0.85  0.48  0.75  1.34  4.23  0.43 
155.25  0.16  0.65  0.84  0.49  0.77  1.29  4.06  0.42 
155.36  0.17  0.67  0.83  0.50  0.80  1.25  3.91  0.40 
155.47  0.18  0.68  0.82  0.50  0.83  1.21  3.77  0.39 
155.57  0.19  0.69  0.81  0.50  0.85  1.17  3.64  0.38 
155.67  0.20  0.70  0.80  0.50  0.88  1.14  3.52  0.37 
155.77  0.21  0.72  0.79  0.51  0.91  1.10  3.41  0.36 
155.86  0.22  0.73  0.78  0.51  0.93  1.07  3.30  0.35 
155.95  0.23  0.74  0.77  0.51  0.96  1.04  3.21  0.34 
156.04  0.24  0.75  0.76  0.51  0.98  1.02  3.11  0.33 
156.12  0.25  0.76  0.75  0.51  1.01  0.99  3.02  0.33 
156.20  0.26  0.76  0.74  0.50  1.03  0.97  2.94  0.32 
156.28  0.27  0.77  0.73  0.50  1.06  0.94  2.86  0.31 
156.36  0.28  0.78  0.72  0.50  1.09  0.92  2.79  0.30 
156.44  0.29  0.79  0.71  0.50  1.11  0.90  2.72  0.30 
156.51  0.30  0.80  0.70  0.50  1.14  0.88  2.65  0.29 
156.58  0.31  0.80  0.69  0.49  1.16  0.86  2.59  0.29 
156.66  0.32  0.81  0.68  0.49  1.19  0.84  2.53  0.28 
156.73  0.33  0.82  0.67  0.49  1.22  0.82  2.47  0.27 
156.80  0.34  0.82  0.66  0.48  1.25  0.80  2.42  0.27 
156.86  0.35  0.83  0.65  0.48  1.27  0.79  2.37  0.26 
156.93  0.36  0.83  0.64  0.47  1.30  0.77  2.32  0.26 
157.00  0.37  0.84  0.63  0.47  1.33  0.75  2.27  0.26 
157.07  0.38  0.84  0.62  0.46  1.36  0.73  2.22  0.25 
157.13  0.39  0.85  0.61  0.46  1.39  0.72  2.18  0.25 
157.20  0.40  0.85  0.60  0.45  1.42  0.70  2.13  0.24 
157.26  0.41  0.86  0.59  0.45  1.46  0.69  2.09  0.24 
157.32  0.42  0.86  0.58  0.44  1.49  0.67  2.06  0.24 
157.39  0.43  0.87  0.57  0.44  1.52  0.66  2.02  0.23 
157.45  0.44  0.87  0.56  0.43  1.56  0.64  1.98  0.23 
157.51  0.45  0.88  0.55  0.43  1.59  0.63  1.95  0.23 
157.58  0.46  0.88  0.54  0.42  1.63  0.61  1.91  0.22 
157.64  0.47  0.88  0.53  0.41  1.67  0.60  1.88  0.22 
157.70  0.48  0.89  0.52  0.41  1.71  0.59  1.85  0.22 
157.76  0.49  0.89  0.51  0.40  1.75  0.57  1.82  0.21 
157.83  0.50  0.89  0.50  0.39  1.79  0.56  1.79  0.21 
157.89  0.51  0.90  0.49  0.39  1.83  0.55  1.76  0.21 
157.95  0.52  0.90  0.48  0.38  1.88  0.53  1.73  0.21 
158.01  0.53  0.90  0.47  0.37  1.92  0.52  1.71  0.20 
158.08  0.54  0.91  0.46  0.37  1.97  0.51  1.68  0.20 
158.14  0.55  0.91  0.45  0.36  2.02  0.49  1.66  0.20 
158.20  0.56  0.91  0.44  0.35  2.08  0.48  1.63  0.20 
158.26  0.57  0.92  0.43  0.35  2.13  0.47  1.61  0.19 
158.33  0.58  0.92  0.42  0.34  2.19  0.46  1.58  0.19 
158.39  0.59  0.92  0.41  0.33  2.25  0.44  1.56  0.19 
158.46  0.60  0.92  0.40  0.32  2.31  0.43  1.54  0.19 
158.52  0.61  0.93  0.39  0.32  2.38  0.42  1.52  0.19 
158.59  0.62  0.93  0.38  0.31  2.45  0.41  1.50  0.18 
158.65  0.63  0.93  0.37  0.30  2.52  0.40  1.48  0.18 
158.72  0.64  0.93  0.36  0.29  2.60  0.39  1.46  0.18 
158.79  0.65  0.94  0.35  0.29  2.68  0.37  1.44  0.18 
158.86  0.66  0.94  0.34  0.28  2.76  0.36  1.42  0.18 
158.93  0.67  0.94  0.33  0.27  2.85  0.35  1.41  0.18 
159.00  0.68  0.94  0.32  0.26  2.95  0.34  1.39  0.17 
159.07  0.69  0.95  0.31  0.26  3.05  0.33  1.37  0.17 
159.14  0.70  0.95  0.30  0.25  3.16  0.32  1.35  0.17 
159.22  0.71  0.95  0.29  0.24  3.28  0.31  1.34  0.17 
159.29  0.72  0.95  0.28  0.23  3.40  0.29  1.32  0.17 
159.37  0.73  0.95  0.27  0.22  3.54  0.28  1.31  0.17 
159.45  0.74  0.96  0.26  0.22  3.68  0.27  1.29  0.17 
159.53  0.75  0.96  0.25  0.21  3.83  0.26  1.28  0.17 
159.62  0.76  0.96  0.24  0.20  4.00  0.25  1.26  0.16 
159.70  0.77  0.96  0.23  0.19  4.18  0.24  1.25  0.16 
159.79  0.78  0.96  0.22  0.18  4.38  0.23  1.24  0.16 
159.88  0.79  0.97  0.21  0.18  4.60  0.22  1.22  0.16 
159.98  0.80  0.97  0.20  0.17  4.84  0.21  1.21  0.16 
160.08  0.81  0.97  0.19  0.16  5.10  0.20  1.20  0.16 
160.18  0.82  0.97  0.18  0.15  5.40  0.19  1.18  0.16 
160.29  0.83  0.97  0.17  0.14  5.72  0.17  1.17  0.16 
160.40  0.84  0.97  0.16  0.13  6.09  0.16  1.16  0.16 
160.52  0.85  0.98  0.15  0.13  6.51  0.15  1.15  0.16 
160.65  0.86  0.98  0.14  0.12  6.99  0.14  1.14  0.16 
160.78  0.87  0.98  0.13  0.11  7.54  0.13  1.13  0.16 
160.92  0.88  0.98  0.12  0.10  8.18  0.12  1.12  0.16 
161.07  0.89  0.98  0.11  0.09  8.94  0.11  1.10  0.16 
161.24  0.90  0.98  0.10  0.08  9.84  0.10  1.09  0.16 
161.42  0.91  0.99  0.09  0.08  10.96  0.09  1.08  0.16 
161.62  0.92  0.99  0.08  0.07  12.34  0.08  1.07  0.16 
161.84  0.93  0.99  0.07  0.06  14.13  0.07  1.06  0.16 
162.10  0.94  0.99  0.06  0.05  16.51  0.06  1.05  0.16 
162.40  0.95  0.99  0.05  0.04  19.84  0.05  1.04  0.16 
162.76  0.96  0.99  0.04  0.03  24.84  0.04  1.04  0.16 
163.22  0.97  1.00  0.03  0.03  33.17  0.03  1.03  0.16 
163.87  0.98  1.00  0.02  0.02  49.84  0.02  1.02  0.17 
164.96  0.99  1.00  0.01  0.01  99.82  0.01  1.01  0.18 
The program in the also produces two tables. Using the default example input data,
the tables are as shown to the left and right. Please note that the program produces result to 4 decimal point precision,
and these are rounded to 2 for most columns on this page.
Table of Transformed Data is the table on the left. The first 3 columns displays the original data as they were entered,
and the last two column the curve fitted FPR and TPR. This table is for the
user to review, to consider whether the curve fitted results reflect the inputted data, and to decide
whether the parameters of the curve fitted ROC can be generalized for future clinical use.
It can be seen that, above a height of 160cms, all mothers delivered vaginally, and below 153cms all mothers required a
Caesarean Section The data overlapped between these two values were used for curve fitting.
It can also be seen that, where the data overlapped, the curve fitted TPR and maternal height are closed enough to the
original for practical use.
Table of Curve Fitted Values
The table to the right is intended to be downloaded and used as reference for clinical decision making, should the user
decides to accept the curve fitted results.
 Column 1 is the reversed calculated Test value for the FPR, and in this example it is the maternal height (cms).
 Column 2 is the False Positive Rate (FPR), from 0.01 to 0.99 at 0.01 intervals. This is the independent variable (x)
used to calculate the values in the remaining columns
 Column 3 is the curve fitted True Positive Rate, and Columns 2 and 3 define the curve fitted ROC.
 Column 4 is the True Negative Rate TNR = 1FPR. This is used in some of the subsequent calculations
 Column 5 is the Youden Index YI = TPR+TNR1, which represents the overall accuracy of the values. The Test value associated
with the maximum YI is sometimes used as the cut off value for clinical decision making, and in this example, the closest
is a maternal height of 156.04cms, with FPR=0.24, TPR=0.75, TNR=0.76, and Youden Index = 0.51
 Column 6 is the ratio of TPR/TNR. This is commonly used for two purposes
 An alternative to the Youden Index for selecting the best cut off value is Q*, suggested by Moses et.al (see references).
This is a point where TPR=TNR, and represents the intersection between the ROC and a line joining the top left and
bottom right corner of the plot. In this example, the closest is a maternal height of 156.12cms, with FPR=0.25, FNR=0.76,
TNR=0.75, Youden Index=0.51, and TPR/TNR=1.01
 If the test is being used for screening purposes, to create an early warning system, then a High TPR is preferred over a
high TNR. A TPR/TNR ratio of between 2 and 3 is therefore used for this purposes. In our example, the closest is a maternal
height of 158.65cms, where TPR/TNR=2.52. This can be set as an alert trigger, possibly making a rule that all
mothers admitted to the labour ward shorter than 158.65cms should be reviewed by an experienced obstetrician in early labour.
 Column 7 is a ratio of TNR/TPR, the inverse of column 5. This is commonly used for intervention decisions, especially if the
intervention is costly or risky. A high TNR is preferred over high TPR, and a TNR/TPR ratio of 2 to 3 can be used for
this purpose. In our example, the closet is a maternal height of 153.55cms, where TNR/TPR=2.31 . This can be used for
clinical decision, such as a rule that all mothers admitted to the labour ward shorter than 153.55cms be considered
for elective Caesarean Section in early labour.
 Column 8 and 9 are Likelihood Ratios for Test Positive and Test negative. Discussions on Likelihood Ratio are provided in
the Prediction Statistics Explanation Page
and will not be repeated here.
Hanley JA, McNeil BJ (1982) The meaning and use of the Area Under a Receiver
Operating Characteristic (ROC) curve. Radiology 143:2936
Simel D.L., Samsa G.P., Matchar D.B. (1991) Likelihood ratios with confidence:
sample size estimation for diagnostic test studies. J. Clin. Epidemiology
vol 44 No. 8 pp 763770
Moses LE, Shapiro D, and Littenberg B(1993) Combining independent studies of a
diagnostic test into a Summary ROC Curve: Data analysis approaches and some
additional considerations. Statistics in Medicine 12:p12931316.
