Appendix 3 Identify relevant trials

Page last updated: September 2016

Search results

From the literature searches reported in Subsection 2.1, complete a PRISMA flowchart^1,2 (Figure A3.1) to indicate the number of papers screened at each stage of study selection.

Clearly depict the reasons for study exclusion (as discussed in Subsection 2.2.1) in the PRISMA flowchart.

The adapted PRISMA flowchart has a three-step process for study selection, where studies are excluded:

1. on the basis of title and abstract, or when the article cannot be retrieved
2. after retrieving full-text articles
3. on the basis of clearly specified reasons other than the exclusion criteria described in Subsection 2.2.1 (justify each exclusion at this point).

Direct randomised trials

Direct randomised trials that are identified and included will form the basis of the submission.

Indirect comparison of randomised trials

If no direct randomised trials are identified that compare the proposed medicine with the nominated comparator, present PRISMA flowcharts separately for the proposed medicine and for the main comparator (without excluding studies on the basis of comparator) to enable an indirect comparison of randomised trials. Additional searches may be required to populate more complex networks. Describe the searches and justify the approach. Do not exclude trials on the basis of poor exchangeability at this point. An acceptable approach to identifying studies for an indirect comparison of randomised trials is discussed later in this appendix.

Nonrandomised studies

If no randomised trials are identified that would enable an indirect comparison of the proposed medicine and the nominated comparator(s), present a third PRISMA flowchart depicting screening for nonrandomised studies. If the primary reason for not conducting an indirect comparison of randomised trials is the lack of a common reference arm, consider using statistical methods to compare the intervention with the comparator using a matching-adjusted indirect comparison^25,65,66 or a simulated treatment comparison.^26,66 If this approach is taken, still perform a search for nonrandomised studies and include relevant studies.

Published systematic reviews and meta-analyses

Extract individual trials from published meta-analyses and compare each trial against the study selection criteria. Exclude any trials that do not meet the criteria. Justify when this is not possible. Consider presenting the treatment effect from the published meta-analysis in a sensitivity analysis.

Figure A3.1 PRISMA flowchart for presenting initial search results

Master list of relevant trials

Prepare a master list of all the included trials and relevant systematic reviews or meta-analyses that meet the inclusion criteria from Subsection 2.2.1. Ensure that the list represents the trials remaining after step 2 of the PRISMA flowchart, and includes relevant trials identified outside database searches (eg experts, searching reference lists, TGA dossier).

Ensure that the list satisfactorily addresses publication bias, duplication bias and outcomes reporting bias. The Pharmaceutical Evaluation Branch will run an independent literature search, and if this search retrieves relevant trials that were not listed in the submission, processing of the submission will stop until the matter has been resolved.

Table A3.1 provides a suggested format for presenting a master list of all the trials included in the submission.

a Includes eligible full-text publications from the database searches specified in Table 2.1.2 and additional eligible publications and study reports identified from other sources.

Do not remove trials from the master list. Include all trials throughout the submission, and exclude and justify trials using sensitivity analyses (step 3 of PRISMA).

Option to present supplementary evidence

Where data from one or more direct randomised trials are available, justify the inclusion of an indirect comparison of randomised trials or a nonrandomised study. Present the literature search and study selection for supplementary data. Indicate how supplementary evidence is used in the submission in Subsection 2.2.5. Label supplementary evidence throughout the submission when it is presented.

Appendix 6 discusses supplementary evidence that explores nonhealth outcomes to be included in Section 3.

Selecting trials for an indirect comparison

If the proposed medicine and the main comparator can be compared using one or more direct randomised trials, an indirect comparison is not usually required.

When direct randomised trials are not available, conduct an indirect comparison of randomised trials. The approach for performing an indirect comparison is based on the report of the Indirect Comparisons Working Group to the PBAC.

List all indirect comparisons possible using the trials within the master list. If there are two or more common references, or if more than one indirect comparison is possible, present a network diagram with the trials listed against the links in the diagram (see example in Figure A3.2). In the master list of trials, identify the trials that are unable to be used within the network because no common reference is available, or because the number of steps required to include the trials in the network would substantially increase uncertainty (shorter links are preferable). Where a network meta-analysis is to be presented, describe the search strategy required to capture the complete range of trials eligible for the network and describe any limitations of the search.

Figure A3.2 Example network diagram of the trials included to inform an indirect comparison of the proposed medicine with the main comparator

k = number of trials; N = number of patients enrolled

In an attachment, present the results (relative and absolute comparative treatment effect) for the outcome(s) on which the submission’s therapeutic claim is based, for all trials listed in the master list of relevant trials. If there are multiple trials with the same treatment comparison, use forest plots.

Excluding trials on the basis of heterogeneity

Justify the exclusion of any trials or pathways. Do not exclude trials or pathways on the basis of heterogeneous characteristics when these are unlikely to influence the treatment effect in the trial (ie unlikely to affect the assumption of transitivity).

Although trials are excluded to improve the transitivity of the trials remaining in the comparison, the possible reasons for exclusion are many, and there is a risk that this process will introduce bias. Therefore, unless there is concern about heterogeneity, and this has a demonstrable effect on the results of the indirect comparison, retain the trials and examine the effect of removing them in sensitivity analyses.

Only one intervention trial and one comparator trial that are not transitive

Where there is one trial of the proposed medicine and one trial of the main comparator with a common reference, do not exclude them because of poor transitivity. Discuss the implications of differences in trial and patient characteristics when presenting the results of the indirect comparison (Subsection 2.6.3). If patient characteristics are heterogeneous in trials, adjust for these differences with, for example, a matching-adjusted indirect comparison or simulated treatment comparison.

Two or more indirect comparison pathways

If there are multiple comparison pathways in an indirect network of trials, present longer pathways or those that do not meet the transitivity assumption as supplementary analyses, if required. To justify the choice of the pathway as the base case, reference trial characteristics that may cause heterogeneity (Appendix 4) and discuss why such characteristics affect the transitivity of trials.

Two or more intervention trials or comparator trials

If there are multiple trials of the same comparison, justify the exclusion of trials on the basis that differences in trial characteristics may affect the transitivity of the trials in an indirect comparison. Adapt the table in Appendix 4 to permit a comparison of trial and patient characteristics within and across trial sets, and a comparison of differences identified that may affect the treatment effect across the trials.

Excluding trials on the basis of differences in treatment effect in the common reference arm

Compare the event rates in the common reference groups within trial sets (Table A3.2).

Justify the exclusion of trials with markedly different event rates or treatment effects in the common reference arms, both within the trial sets and across the indirect comparison. Take care when excluding trials because of differences in event rates in the common reference arms when there is evidence of a constant relative (or absolute) treatment effect across the range of these event rates. Include any excluded trials in a sensitivity analysis in Subsection 2.6.3.

Presenting trials included in the indirect comparison

Present a summary list of trials using the unique name from the master list to identify trials that are included in the indirect comparison, included in sensitivity analyses or excluded from the remainder of the submission (Table A3.3).