What is it about?
Accurate predictive scores from a biomarker or a predictive system can be useful to predict risk of disease (e.g. cancers) or risk of disease recurrence or progression (e.g. time to cancer progression or death). For survival endpoints, the overall predictive accuracy is often assessed using some concordance indexes such as Harrell's c-index. From the view point of precision medicine, it is of great interest to develop valid non-parametric methods (e.g. nonparametric hypothesis tests) to compare predictive accuracy of risk scores which has been a challenge in biostatistics. This paper investigate the validity of a recently proposed nonparametric test to compare predictive accuracy of two risk scores using the difference of two Harrell's c-statistics as the test statistic. The test using the difference of two Harrell's c-statistics was partially justified based on limited simulation on its type 1 error rate, but general theoretical considerations on its validity have been lacking. In this paper, we provide counter-examples to illustrate that the non-parametric test based on the difference of Harrell's c-statistics is not generally valid because it can have large inflated type 1 errors compared with the nominal levels. We also provide necessary and sufficient conditions for the test to be asymptotically valid.
Featured Image
Why is it important?
As biomarkers and predictive procedures are under rapid development in medical and other fields, predictive scores from a biomarker or a predictive system can be useful to predict risk of disease (e.g. cancers) or risk of disease recurrence or progression (e.g. time to cancer progression or death). In particular, accurate predictive scores can be useful to physicians and patients for making informed decisions. Predicting time to cancer recurrence and predicting time to cancer progression or even death are of fundamental importance in medicine and particularly in precision medicine. Therefore, valid statistical methods including valid hypothesis tests for comparison of predictive accuracy of risk scores are of great current interest. For survival endpoints, the overall predictive accuracy or sometimes called discriminatory power are often assessed or compared using some concordance indexes such as Harrell's c-statistic. This paper is of interest because it investigates validity of the existing non-parametric test to compare predictive accuracy of risk scores.
Perspectives
General and valid nonparametric methods for the comparison of predictive accuracy for risk scores or other predictive scores for censored survival outcomes have not emerged yet. In biostatistics, we often assume the censoring distribution is unknown and can be arbitrary. The unknown censoring distribution makes general nonparametric assessment of predictive accuracy very challenging particularly when censoring is heavy. This is partially reflected by the fact that the widely used Harrell's c-statistics is biased under the commonly existing right censoring and its asymptotic distribution depend on the underlying unknown censoring distribution. There are some existing research on reducing the bias by inverse probability weighting, but a completely satisfactory solution has not emerged and likely not exists. Instead of considering pure non-parametric general situation, it can be more productive to consider general semi-parametric survival models that contain the commonly used Cox models, AFT models, proportional odds models as special cases. For example, the general semi-parametric transformations models contain all those widely used survival models as special cases. Thus, without much loss of generality, one can consider such general semi-parametric transformations models instead of seek a completely nonparametric solution. Assessing predictive accuracy in general transformation models is clearly much straightforward and fruitful than in the nonparametric setting.
Yongzhao Shao
New York University
Read the Original
This page is a summary of: On comparing 2 correlated C indices with censored survival data, Statistics in Medicine, July 2017, Wiley,
DOI: 10.1002/sim.7414.
You can read the full text:
Contributors
The following have contributed to this page
