Appendix 1 Expert opinion
Page last updated: September 2016
Uses of expert opinion
Consider providing expert opinion to supplement or support the observed data from randomised trials or nonrandomised studies (including drug usage evaluations, cross-sectional studies or case studies).
Determining an appropriate body of experts will depend on the nature of the information gap that requires filling. Experts may be panels of medical practitioners, a medical specialty group or consumers. Consumers may provide advice on factors such as the patient relevance of outcomes (particularly if elicited at the time of trial design) or how medicines might be used. Expert opinion can be useful in several aspects of preparing submissions to the PBAC – for example, to help:
- define the clinical need for the proposed medicine and inform the main indication (Subsection 1.4)
- determine how the medicine is most likely to alter the clinical management algorithm (Subsection 1.2) and support the choice of the main comparator (Subsection 1.1), noting that a comparator should not be determined by expert opinion alone
- interpret the clinical importance and patient relevance of the outcome measures reported in the trials (Subsections 2.4 and 2.8)
- modify the patterns of health care resource use measured in randomised trials conducted in different settings, such as in other countries (Subsection 3A.6)
- predict which health care resources would be used and how often each would be used to manage outcomes reported in the randomised trials, but were not followed up (Subsection 3A.6)
- estimate the proportion of patients with the medical condition that would be eligible according to the requested listing, and predict uptake rates (Subsection 4.2)
- predict the extent of increases or decreases of other PBS-listed medicines (Subsection 4.3).
In several examples above, trial data, registry data or analyses of data from other countries, where available, would be used in preference to expert opinion, and it would be expected that the expert opinion supports the applicability of the observed data. An example is to support the representativeness of a drug usage evaluation conducted in another country. In this case, expert opinion reduces uncertainty.
Presenting expert opinion
Justify the use of expert opinion in the introduction of the appropriate section. Include a clear rationale for, and the aims of, eliciting the expert opinion. Where expert opinion is used to fill a gap in information, clearly describe the nature of this gap and indicate the other steps that have been taken to address the gap, such as a literature search.
Describing the collection and collation of expert opinion
Using a well-designed methodology to elicit expert opinion helps to reduce uncertainty. The methods used may vary from large, published questionnaires and surveys with statistical analysis to a summary of interviews with a panel of clinical experts. Present expert opinion as qualitative or quantitative (but not statistically analysed) information.
Include copies of administered surveys or hypothetical scenarios that were presented to experts.
When summarising expert opinions and their variability, interpret the findings, and discuss the limitations and biases of the method chosen. Qualitative studies and interviews should follow best practice for reporting and analysis.61-63 Indicate how the opinions have been used in the main body of the submission.
Where multiple sources of expert opinion are available to address a single assumption or estimate, compare the results, and assess their concordance or lack of it. Present a summary table that compares multiple sources or multiple variables. Table A1.1 provides guidance on the details that should be included. Where multiple estimates (or data) are generated to fill a gap in the information – either from multiple sources of expert opinion or a combination of expert opinion and observed data – compare the estimates (or data) and justify the choice of data used in the submission.
Where expert opinion is used in place of observed data, as may occur when observed data are generated from other health care systems or are historical, present both and clearly justify the use of expert opinion. State if expert opinion (compared with alternative sources of data) is likely to lead to a more favourable clinical, economic or financial assessment of the proposed medicine.
The PBAC is concerned when information used within the clinical, economic or financial analysis of the proposed medicine is uncertain. Where expert opinion is sought for a disease or condition for which the number of prescribers is likely to be large, do not rely on surveys of small numbers of prescribers because this leads to highly uncertain results. In all cases where expert opinion is used to derive estimates for the submission, use the final estimate to minimise the risk to the PBAC of relying on an overestimation of the effectiveness or cost-effectiveness, or underestimation of the financial implications to the Australian Government. To reduce uncertainty associated with expert opinion, provide sensitivity analyses around the derived estimates, or clearly state where the results in the submission are not sensitive to different estimates.
|
Information to be provided |
Notes |
|---|---|
|
Criteria for selecting experts |
Prefer a random or comprehensive set of prescribers likely to prescribe the proposed medicine, or the appropriate medical specialty group. In general, an advisory board created by the sponsor, or for advising on the drug development program or marketing may not be representative of experts in Australian clinical practice. The generalisability of expert opinion derived from such boards is difficult to assess |
|
Number of experts approacheda |
Where the likely number of prescribers is large, it is less acceptable to provide expert opinion derived from a small number of prescribers |
|
Number of experts who participateda |
Assess whether the extent and characteristics of the nonresponders are likely to diminish the representativeness of the opinions provided, compared with the intended sample approached |
|
Declaration of potential conflicts of interest from each expert or medical specialty group whose opinion was sought |
Provide a signed statement from each expert and specialty group specifying any potential conflict of interest and stating the nature of any contractual arrangement, including how much payment was offered and accepted. Where the collection of expert opinion has been contracted out, the contractor should provide this statement, reporting on both the arrangements made between the sponsor and the contractor, and the arrangements made between the contractor and those whose opinions were sought |
|
Background information provided and its consistency with the totality of the evidence provided in the submission |
Include a copy of any background information provided in the technical document or attachment. If background information has been provided, ask the experts to define the comparative clinical place of the proposed medicine and the main comparator based on this background information. Including the experts’ definitions in the technical document or attachment allows an assessment of the consistency of the background information with the evidence provided in the submission |
|
Method used to collect opinions |
For example, were the experts approached individually or was a meeting convened? Was any incentive used to maximise responses? |
|
Medium used to collect opinions |
For example, was information gathered by direct interview, telephone interview or self-administered questionnaire? |
|
Questions askedb |
Explain the design of the tool (quantitative or qualitative). Describe its development. Indicate whether it was pilot tested and, if so, provide the results of that testing and explain how the results were used to improve the questions. On a question-by-question basis, assess the extent to which each question is neutral or biased, and the extent to which each question is open or closed. To allow an independent assessment, include the questionnaire or an outline of the interview questions in the technical document (or attach a copy) |
|
Whether iteration was used in the collation of opinions and, if so, how it was used |
The Delphi technique, for example, uses an iterative approach |
|
Number of responses received for each questiona |
Assess whether the extent of any nonresponse is likely to diminish the representativeness of the opinions provided to particular questions, compared with the intended sample approached |
|
Whether all experts agreed with each response |
If not, specify (i) the approach used to finalise the estimates (eg the majority opinion or a Delphi technique could be applied; for quantitative results, point estimates [such as the mean, median or mode] could be presented), and (ii) the approach used to present the variability in the opinions (eg present the range of opinions expressed, including common and outlying views; for quantitative results, measures of variance [such as confidence intervals, range, centiles] could be presented) |
a Tabulate these information items.
b The way the questions are asked is an important source of potential bias in obtaining expert opinion. A particularly influential extension question extends the respondent beyond ‘what’ the opinion is (eg what would be done, what extent of benefit would be clinically important) to ask ‘why’ (eg explain why would you do this, explain why this is important). Conveying these reasons alongside expert opinion–based estimates might help improve their acceptability. Including these explanations in the technical document or attachment would allow the opinions to be assessed on the basis of the underlying reasoning rather than only depending on the authority of the experts.