3A.9 Uncertainty analysis: model inputs and assumptions
Page last updated: September 2016
Information Requests
 Explain the methods used to represent the uncertainty around the model’s input parameters, translations and structure. For each, define the uncertainty or alternatives to be tested in sensitivity or scenario analyses (Subsection 3A.9.1)
 Present and discuss the univariate sensitivity and scenario analyses (Subsection 3A.9.2)
 Present and discuss relevant multivariate analyses and any probabilistic sensitivity analysis (Subsection 3A.9.3)
 Summarise the findings of the uncertainty analysis (Subsection 3A.9.4)
3A.9.1 Identifying and defining uncertainty in the model
Present univariate deterministic sensitivity analyses for all uncertain input parameters, or natural groups of input parameters (eg cost or utility weights for all target clinical outcomes). The following requests are based on goodpractice guidelines for model parameter estimation and uncertainty analysis.^{38}
Parameter uncertainty
Use commonly adopted statistical standards to represent the uncertainty around the true value of each uncertain input parameter. For example, beta distributions are a natural match for transition probabilities; lognormal for relative risks or hazard ratios; logistic distributions to calculate odds ratios; and gamma or lognormal for costs and utility parameters.
Justify using alternative distributions. Use interval estimates (eg 95% CIs) derived from fitted probability distributions to define the ranges of the parameter values tested in the deterministic sensitivity analyses.
Where there is very little information on a parameter, adopt a conservative approach by defining a broad range of possible parameter values. Never exclude parameters from uncertainty analysis on the grounds that there is insufficient information to estimate uncertainty.
Consider correlation between input parameter values. If applicable, represent the joint uncertainty around the true values of two or more input parameters in the uncertainty analyses. In particular, it is preferable to represent the joint uncertainty around transition probabilities in the intervention group and the comparator group through the application of a relative treatment effect parameter. If a relative treatment effect parameter is not applicable, individuallevel data for the comparator and intervention could be bootstrapped to provide more realistic estimates of the joint uncertainty between these.^{38}
The joint estimation of multiple input parameters when using regression analysis produces relevant correlation parameters. Otherwise, model calibration methods may be used to represent joint uncertainty around the true value of model input parameters.
Translational uncertainty
Where clinical data have required translation for applicability issues, transformation or extrapolation for incorporation into the model, systematically consider the assumptions incorporated into the translation and identify any uncertainty in these assumptions. Identify plausible alternatives for testing in scenario analysis.
Examples of analyses that can be used where the data or outcome translations are incorporated into basecase analysis are presented in Table 3A.9.1.
Translations incorporated into basecase analysis 
Suggested uncertainty analysis 

Transformation of continuous outcome data to a dichotomous outcome 
Alternative thresholds (Subsection 2.5.1) 
Treatment effect with adjustment for switching 
Treatment effect without adjustment for switching, and/or using an alternative adjustment technique (Subsection 2.6.4) 
Treatment effect based on translation (eg subgroup analysis) following applicability study 
Treatment effect based on intentiontotreat population (Subsections 2.6.1, 3A.3.2) 
Selected source(s) of data for treatment effect 
Alternative available source(s) of data, and/or metaanalysis of data as source of treatment effect (alternative analyses presented in Subsections 2.5 and 2.6) 
Transformation of a surrogate to a final outcome 
Range of alternative plausible values (as derived establishing STFO relationship; Subsection 3A.4.2) 
Extrapolation of data beyond the trial 
Alternative data truncation point(s), alternative choices of parametric model, or alternative assumptions regarding ongoing treatment effect (Subsection 3A.4.3) 
Pooled withintrial data to estimate utility values (or alternative approach) 
Estimates based on individual arms (or the alternative approach; Subsection 3A.5.1) 
Externally sourced utility values 
Alternative values or sources (Subsection 3A.5.1) 
STFO = surrogate to final outcome
Structural uncertainty
If multiple plausible model structures are defined, assess the potential impact of the alternative structures on the model outputs. If a substantial impact is predicted, use a formal approach to characterise the structural uncertainty. Parameterise structural assumptions where there is sufficient clinical evidence or expert opinion to do so. Alternatively, use scenario analyses to assess the impact of assumptions around the structure of the economic model. Report the results of each set of plausible structural assumptions.
Describe and justify the inclusion and exclusion of potential scenario analyses when making alternative assumptions about data translation and model structure.
Include an analysis of the impact of the time horizon.
Use other scenario analyses to assess the effects of substantial use of the proposed medicine beyond the intended population and circumstances of use defined in the requested restriction. This wider population or circumstances are expected to have demographic and patient characteristics and circumstances that differ from the target population and circumstances.
3A.9.2 Presentation of univariate sensitivity and scenario analyses
Tabulate all parameter values and assumptions included in the model, and present the results of univariate sensitivity and scenario analyses in a similar format to Table 3A.9.2.
Use a tornado diagram to represent the relative effect of the uncertainty around alternative input parameters on the basecase incremental costeffectiveness result.
Identify the input parameters and model assumptions to which the incremental costeffectiveness results are most sensitive.
3A.9.3 Presentation of multivariate and probabilistic sensitivity analyses
Use multivariate sensitivity analyses to test the combined effects of the uncertainty around the true values of input parameters to which the basecase incremental costeffectiveness result was shown to be sensitive in the univariate analyses.
Describe the multivariate sensitivity analyses to be undertaken, and present the results. Justify the inclusion and exclusion of parameters in these analyses.
A probabilistic sensitivity analysis (PSA) may be provided in addition to deterministic sensitivity analysis. Although PSA can usefully characterise parameter uncertainty, it cannot address translational or structural uncertainty.
If undertaking a PSA on a cohortbased state transition model, the number of iterations (sets of randomly sampled input parameter values included in the analysis) should provide stability in the model outputs across multiple analyses using alternative random number seeds. Provide the random seed associated with the presented results to enable replication, and also ensure that the model permits alternative seeds.
If undertaking a PSA on an individuallevel model (eg a discrete event simulation), the number of iterations may be selected to balance stability of model outputs and a reasonable time required to undertake a PSA (eg a few hours, rather than a few days).
Use costeffectiveness planes and acceptability curves to present the results of a PSA, as well as the tabulated presentation of the interval estimates for the ICER or the incremental net benefits of the proposed medicine.
3A.9.4 Summary of the uncertainty analysis
Describe and justify a likely range of values within which the true estimate of the incremental costeffectiveness of the proposed medicine is likely to lie, identifying the key sources of uncertainty. This range may be informed by a formal PSA, or by subjective interpretation of the presented deterministic sensitivity and scenario analyses.
Discuss the implications of the sensitivity and scenario analyses with respect to the certainty of the basecase ICER estimate.
Discuss the likely overall effect of deficiencies in the evidence base on the reported costeffectiveness of the proposed medicine.
Variable or assumption 
Basecase value 
Plausible alternative(s) or range of values 
Incremental outcomes 
Incremental costs 
ICER 
Description of impact on ICER 

Base case 


[base case] 
[base case] 
[base case] 

Discounting rate 
Outcomes and costs = 5% 
Outcomes and costs = 3.5% 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Plausible range of treatment effect, if modelled as a variable (eg hazard ratio or relative risk) 
[add] 
[eg upper and lower 95% confidence intervals around estimate] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Altered patient characteristics, if relevant 
[add] 
[eg different average age, disease or condition severity] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Transition or event probabilities 
[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Outcomerelated assumptions or variables [Recommended examples:

[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Costrelated assumptions or variables 
[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Alternative extrapolation variables or assumptions [Recommended examples:

[eg maximum followup] 
[eg median followup] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Any other translation assumptions [eg use of intentiontotreat/nonadjusted data] 
[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Alternative assumptions regarding model structure 
[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Time horizon 
[add] 
[eg trial based; 5, 10, 20 years, as appropriate] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
Plausible alternatives for other variables or assumptions [eg including leakage beyond the requested restriction] 
[add] 
[add] 
[alternative estimates] 
[alternative estimates] 
[alternative estimates] 
[describe as required] 
ICER = incremental costeffectiveness ratio; STFO = surrogate to final outcome