On the whole, model building for data analysis is a well-defined process with relatively few issues relating to a model’s internal consistency. The concept of model internal consistency is defined as a property of the model that is in some way inconsistent within itself rather than an inconsistency resulting from the interaction of the model with some data. The presence of inconsistency will ultimately result in an unstable model (i.e. the model does not behave predictably for similar situations) for which no amount of run retries or parameter tweaks is going to resolve. It is desirable to assess any instability of the model which might arise due to the model itself (e.g. perhaps the study design does not support the proposed model structure) separately to how it behaves when data are added. An example of a model that is not internally consistent might be one in which the model is not identifiable (e.g. a practitioner may attempt to estimate the absolute bioavailability of an orally administered drug in a setting where only data from oral administration is available). While it is generally the case that modellers do not build unidentifiable models to describe their data, when using an existing model for different purposes or re-using a model developed by another research team there are risks of issues with the internal consistency of a model which may require parameters to be fixed or structures to be removed. Consider the case in which you are re-using an insulin-glucose model that you downloaded from GitHub, but when developing the original model the authors had measured both insulin and glucose but in your data you only have glucose measurements.
In this presentation a framework for evaluating the internal consistency of a model and how to resolve any issues is provided. The workflow can be applied to any model building or use setting and can be completed using standard pharmacometric applications, i.e. NONMEM & R (or similar), without the need for specialized software or training.
The workflow consists of 3 diagnostic steps.
- Structural identifiability analysis (performed using $DESIGN)
- Deterministic identifiability analysis based on executed study design (performed using $DESIGN)
- Global sensitivity analysis (Sobol performed using R)
These diagnostic steps are then followed by a series actions depending on the outcomes, for instance these actions might be focused on determining which parameters need to be fixed and what values are reasonable. Importantly, this process provides a formal method to determine and defend the choice of what to fix and to what value to choose and when fixing can be relaxed to using a Bayesian prior.
The workflow is demonstrated for a series of models that illustrate different characteristics of model internal inconsistency.