Selecting the most externally predictive covariate of two correlated covariates can be difficult. In this study, we investigated 3 different covariate selection methods with respect to their predictive performance: a modified Stepwise Covariate Modelling (SCM), Full Fixed Effects Model (FFEM) and Prior-Adjusted Covariate Selection (PACS). The selection for SCM is agnostic (i.e. only data driven) between including either of the covariates, whereas the FFEM would assign the most likely covariate without consideration of the information in data. The PCM allows the user to integrate a prior likelihood, representing their strength of belief of the appropriateness of one covariate over another, into the selection.
To do so concentrations were simulated for 25 and 100 subjects, respectively the lower-power (0.88 versus no covariate model) and higher-power (1.00) training data sets; and 10000 subjects, the validation data set. A simple steady state model was used, together with covariates A and B, which were correlated to an extent of 0.5-0.9. The model consisted of only one parameter (CL) with an IIV of 30%, half of which stemmed from the true covariate, which was either A or B according to a set probability P(T). Two models were fitted to each of the training data sets (N=100 replications per design): with either covariate A or B. The selected models of each method (SCM, FFEM and PACS) over each replicate was applied to the corresponding replicate of the validation data. The predictive performance was estimated as the average individual OFV in the validation data for each selection method relative to the PACS method (dOFV).
PACS allows the modeller to penalize the less likely covariate model by adding to the OFV a prior probability-derived constant, calculated as -2LN[(1-Pr(X))/Pr(X)] where X is the most likely covariate. The selected PACS model is then the one with the lowest OFV. The selected FFEM model is the one with the highest Pr(X) and the SCM in this context is equivalent to PACS with a Pr(A)=Pr(B)=0.5.
Some results for simulations with a correlation between A and B of 0.5 are shown in the table below. Differences at higher correlations showed the same pattern, but with lower differences between methods.
| dOFV(SCM-PACS) | dOFV(FFEM-PACS) | ||||||
| Power | Prior(A) | P(A)=0.1 | P(A)=0.5 | P(A)=0.9 | P(A)=0.1 | P(A)=0.5 | P(A)=0.9 |
| 0.88 | 0.9 | -0.892 | -0.462 | 0.023 | 0.214 | 0.085 | 0.015 |
| 0.8 | 0.504 | -0.090 | 0.167 | 0.253 | 0.122 | 0.159 | |
| 0.7 | -0.402 | -0.017 | 0.037 | 0.263 | 0.104 | 0.005 | |
| 0.6 | -0.251 | -0.098 | 0 | 0.278 | 0.121 | -0.000 | |
| 0.5 | 0 | 0 | 0 | 0.304 | 0.131 | -0.000 | |
| 1.00 | 0.9 | 0 | 0 | 0 | 0.333 | 0.167 | 0.034 |
| 0.8 | 0 | 0 | 0 | 0.333 | 0.167 | 0.034 | |
| 0.7 | 0 | 0 | 0 | 0.333 | 0.167 | 0.034 | |
| 0.6 | 0 | 0 | 0 | 0.333 | 0.167 | 0.034 | |
| 0.5 | 0 | 0 | 0 | 0.333 | 0.167 | 0.034 | |
For the lower-power situation, no method is superior to others under all conditions. Whenever the prior used in PACS is far from the true frequency with which the data were generated, either SCM or FFEM perform better. However, PACS is considerably more robust towards errors in the prior compared to FFEM. For the high-power situation PACS and SCM performs the same, both superior to FFEM.