Oncotarget

Meta-Analysis:

Comparison of direct sequencing and amplification refractory mutation system for detecting epidermal growth factor receptor mutation in non-small-cell lung cancer patients: a systematic review and meta-analysis

PDF |  HTML  |  Supplementary Files  |  How to cite  |  Order a Reprint

Oncotarget. 2017; 8:59552-59562. https://doi.org/10.18632/oncotarget.19110

Metrics: PDF 1122 views  |   HTML 1806 views  |   ?  

Qi Feng, Zu-Yao Yang, Jia-Tong Zhang and Jin-Ling Tang _

Abstract

Qi Feng1,*, Zu-Yao Yang1,*, Jia-Tong Zhang1 and Jin-Ling Tang1

1 Division of Epidemiology, JC School of Public Health and Primary Care, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong

* These authors have contributed equally to this work

Correspondence to:

Jin-Ling Tang, email:

Keywords: epidermal growth factor receptor, direct sequencing, amplification refractory mutation system, non-small cell lung cancer, tyrosine kinase inhibitors

Received: January 17, 2017 Accepted: June 27, 2017 Published: July 08, 2017

Abstract

Background: Direct sequencing and amplification refractory mutation system (ARMS) are commonly used to detect epidermal growth factor receptor (EGFR) mutation status in patients with non-small-cell lung cancer to inform the decision-making on tyrosine kinase inhibitors treatment. This study aimed to systematically compare the two methods in terms of the rate of detected mutations and the association of detected mutations with clinical outcomes.

Material and methods: PubMed, EMBASE, China National Knowledge Infrastructure (in Chinese) and Wanfang database (in Chinese) were searched to identify relevant studies. Meta-analyses of EGFR mutation rates, rate differences, and the associations of EGFR mutations with clinical outcomes of tyrosine kinase inhibitors treatment were conducted.

Results: Eight hundred and sixty-six records were retrieved and 26 studies with 3282 patients were included. The pooled rate of mutations detected by ARMS (41%, 95% confidence interval (CI) 35% to 47%) was significantly higher than that by direct sequencing (28%, 95%CI 22% to 34%), with a weighted rate difference of 11% (95%CI 8% to 13%). There was a consistent trend that the associations between ARMS-detected mutations and clinical outcomes were stronger than those between direct-sequencing-detected mutations and clinical outcomes (pooled risk ratio for objective response: 5.18 vs. 2.25; hazard ratio for progression-free survival: 0.30 vs. 0.42; hazard ratio for overall survival: 0.46 vs. 0.54).

Conclusions: More patients with EGFR mutations can be identified by ARMS than by direct sequencing, and those identified by ARMS seems to be able to benefit more from tyrosine kinase inhibitors than those identified by direct sequencing.


Introduction

Epidermal growth factor receptor tyrosine kinase inhibitors (EGFR TKIs) are now the standard treatment for patients with advanced non-small cell lung cancer (NSCLC) harboring activating mutations in the EGFR gene [1-3]. Testing EGFR mutations is therefore very important for the decision-making with regard to this treatment. Two broad categories of methods, i.e. screening methods and targeted methods, are available for EGFR mutation testing [4]. Screening methods, such as direct sequencing, denaturing high-performance liquid chromatography and high-resolution melting analysis, detect all mutations, including novel unknown variants. Targeted methods, such as amplification refractory mutation system (ARMS), fragment length analysis and pyrosequencing, detect specific known mutations, including exon 19 deletions and exon 21 L858R point mutation that represent the majority of activating mutations of EGFR [4].

Direct sequencing of DNA extracted from fresh or formalin-fixed, paraffin-embedded tumor tissue is the historical standard for EGFR mutation testing. This method is relatively cost-effective compared with targeted methods, but it requires a mutation to be present in at least 20% of all DNA in a sample to be reliably detected [5, 6]. As the proportion of tumor cells in lung tissue samples can vary from 5% to 100% [4], and tumor tissues are often not available from advanced NSCLC for various reasons [7], in which case cytological samples such as plural effusion with low proportion of tumor cells may have to be used as alternatives, more sensitive methods for EGFR mutation testing are warranted. In addition, direct sequencing needs experienced operators and tends to be time-consuming and labor-intensive.

Several methods have been developed as potential alternatives to direct sequencing, each with its own limitations [4]. For example, fragment length analysis can detect insertions and deletions but not point mutations in EGFR; pyrosequencing requires the proportion of tumor cells in a sample to be 20% or more to maintain its accuracy [8]. Among them, the allele specific polymerase chain reaction-based method ARMS is heavily investigated, with validated, quality controlled testing kits available for use. It was employed by the landmark trial IPASS [9] to establish the predictive value of EGFR mutations in the EGFR TKIs treatment of advanced NSCLC, and is currently widely adopted in practice. By incorporating fluorescent probes such as TaqMan and Scorpions, ARMS can be further enhanced to analyze the results in a real-time, closed-tube format [10].

A number of studies have compared ARMS and direct sequencing in terms of the rate of EGFR mutations detected. While some found that the two methods yielded similar mutation rates [1, 10] , others reported significantly higher mutation rate detected by ARMS than by direct sequencing [11, 12]. Whether the discrepancy across studies was caused by pure chances, different specimen types used or true difference in detecting ability between the two methods remains to be clarified. More importantly, existing studies were mainly focused on the comparison of mutation rates. Attention was rarely paid to the difference in strength of association between the EGFR mutations detected by different methods and clinical outcomes of EGFR TKIs treatment, which is crucial in determining the superiority of one method over the other.

We therefore conducted a systematic review with meta-analysis to address these issues. For simplicity, the comparison of ARMS with direct sequencing in this paper was focused on: (1) the ability of different methods in detecting EGFR mutations, as measured by mutation rate, and (2) the association of detected EGFR mutations with clinical outcomes of EGFR TKIs treatment. Other issues, such as logistics, requirement of expertise and costs of the methods, were not the interest of this study. The superiority of one method over the other will depend on the results of comparison in the two aspects mentioned above. Specifically, if the rate of mutations detected by ARMS is higher than by direct sequencing and the association of mutations detected by ARMS with clinical outcomes is stronger than that of mutations detected by direct sequencing, ARMS will be considered superior to direct sequencing, and vice versa. If the two methods are comparable in both aspects, they will be considered equivalent. If one of them yields higher mutation rate but the mutations detected by the other have stronger associations with clinical outcomes, then it would be hard to tell which one is superior.

Results

The results of literature search and flow of study selection are shown in Figure 1. Briefly, 26 eligible studies with 3282 NSCLC patients [1, 10-34] were identified from the 866 records initially retrieved.

Diagram of study selection.

Figure 1: Diagram of study selection. CNKI: China National Knowledge Infrastructure. ASCO: American Society of Clinical Oncology. ESMO: European Society for Medical Oncology. DS: direct sequencing. ARMS: amplification refractory mutation system. EGFR: epidermal growth factor receptor.

Study characteristics

Most of the included studies were from China (n = 20), followed by Japan (n = 4), South Korea (n = 1) and UK (n = 1). The sample sizes of studies ranged from 15 to 451, with a mean of 126. For EGFR mutation testing, 19 studies used tumor tissue samples [1, 4, 12, 14-16, 19, 20, 23, 24, 26-34], six studies cytological materials [1, 11, 13, 16, 18, 25] and three studies blood samples [18, 21, 34], with four studies using more than one type of samples [1, 16, 18, 34] and two not specifying the specimen types [17, 22]. All studies reported EGFR mutation rates detected by the two methods. According to the modified AHRQ quality assessment tool, the mean score was 6.2 out of 9, and eight studies (30.7%) got a score of 7 or above (Supplementary Table 1), which were considered as with high quality. The association of EGFR mutations with objective response, progression-free survival and overall survival was investigated by five [11, 18, 30, 32, 33] five [25, 29, 30, 32, 33] and two [25, 32] studies, respectively. All of the studies got 7 or above out of 9 scores on the Newcastle-Ottawa Scale and thus were considered as with high quality (Supplementary Table 2). The characteristics of included studies are shown in detail in Table 1.

Table 1: Baseline characteristics of included studies

study

Country

Test method

Sample size

Specimen type

Mutation type

Sex, male/total

Smoking status, never smoker/total

Histology, adenocarcinoma/total

Outcomes

Study quality assessment

AHRQ score(out of 9)

Newcastle-Ottawa score(out of 9)

Chu H 2013

China

DS vs ARMS

24

cytological

exon 19-21

16/24

8/24

20/24

mutation rate

8

NA3

Dou Y 2013

China

DS vs ARMS

199

tumor tissue

exon 19,21

146/199

NA

103/199

mutation rate

6

NA3

Ellison G 2010

UK

DS vs ARMS

197

tumor tissue

exon 18-21

NA

NA

NA

mutation rate

6

NA3

Goto K 2012

Japan

DS vs ARMS

135

tumor tissue, cytological

exon 18-21

NA

NA

NA

mutation rate

5

NA3

Horiike A 2007

Japan

DS vs ARMS

91

tumor tissue

exon 19, 21

63/94

34/94

58/94

mutation rate

6

NA3

Kimura H 2006

Japan

DS vs ARMS

24

cutological

exon 18-21

13/24

14/24

23/24

mutation rate, ORR

6

8

Lee D 2010

South Korea

DS vs ARMS

21

tumor tissue

exon 18, 19, 21

8/21

13/21

20/21

mutation rate

NA2

NA2

Li C 2014

China

DS vs ARMS

451

tumor tissue, cytological

exon 18-21

204/451

NA

329/406

mutation rate

7

NA3

Li H 2011

China

DS vs ARMS

15

NA1

exon 18-21

NA

NA

NA

mutation rate

NA2

NA2

Liu Y 2011

China

DS vs ARMS

50

blood, cytological

exon 19, 21

32/50

NA

50/50

mutation rate, ORR

5

8

Morinaga R 2008

Japan

DS vs ARMS

100

tumor tissue

exon 18-21

64/100

33/100

61/100

mutation rate

7

NA3

Qian X 2015

China

DS vs ARMS

131

tunor

exon 18-21

NA

NA

NA

mutation rate

7

NA3

Qin L 2011

China

DS vs ARMS

73

blood

exon 19, 21

33/73

48/73

73/73

mutation rate

6

NA3

Shujie A 2014

China

DS vs ARMS

154

NA1

exon 18-21

84/154

NA

121/154

mutation rate

6

NA3

Wang J 2012

China

DS vs ARMS

45

tumor tissue

exon 18-21

21/45

NA

28/45

mutation rate

7

NA3

Wang S 2012

China

DS vs ARMS

37

tumor tissue

exon 18-21

127/241

126/241

213/241

mutation rate

7

NA3

Wang Z 2014

China

DS vs ARMS

180

cytological

exon 18-21

109/180

NA

177/180

mutation rate, PFS,OS

7

8

Xu H 2014

China

DS vs ARMS

182

tumor tissue

exon 18-21

126/220

NA

183/220

mutation rate

8

NA3

Zhang J 2008

China

DS vs ARMS

82

tumor tissue

exon 18-21

48/82

NA

39/82

mutation rate

6

NA3

Zhang X 2013

China

DS vs ARMS

420

tumor tissue

NA

NA

NA

420/420

mutation rate

NA2

NA2

Zhao J 2011

China

DS vs ARMS

31

tumor tissue,

exon 19, 21

NA

NA

NA

mutation rate

5

NA3

Zhao J 2013

China

DS vs ARMS

168

tumor tissue

exon 18-21

96/168

109/168

125/168

mutation rate, ORR, PFS

6

9

Zhao J 2014

China

DS vs ARMS

124

tumor tissue

exon 18-21

43/124

97/124

105/124

mutation rate, PFS

4

8

Zhou Q 2011

China

DS vs ARMS

100

tumor tissue

exon 18-21

49/100

77/100

93/100

mutation rate,ORR,PFS,OS

6

8

Zhou S 2014

China

DS vs ARMS

158

tumor tissue

exon 19, 21

86/158

88/158

101/158

mutation rate, ORR,PFS

5

8

Zou M 2013

China

DS vs ARMS

90

tumor tissue, blood

exon 19, 21

55/90

59/90

57/90

mutation rate

6

NA3

DS: direct sequencing; ARMS: amplification refractory mutation system; ORR: objective response rate; OS: overall survival; PFS: progression-free survival; NA1: the study did not provide sufficient information on specimen types; NA2: conference abstract without sufficient information for quality assessment; NA3: cross-sectional studies, inapplicable for cohort study quality assessment.

EGFR mutation rate

The rates of EGFR mutations detected by ARMS ranged from 9% to 69%, with a pooled mutation rate of 41% (95% confidence interval (CI) 35%-48%; heterogeneity test: P < 0.001, I2 = 92%). The mutation rates detected by direct sequencing ranged from 4% to 60%, with a pooled mutation rate of 28% (95%CI 23%-35%; heterogeneity test: P < 0.001, I2 = 93%). The within-study difference in mutation rate between ARMS and direct sequencing ranged from -1% to 38%, with a pooled rate difference of 11% (95%CI 8% to 13%; heterogeneity test: p < 0.001, I2 = 92%; Figure 2), which indicated that the mutation rate by ARMS was significantly higher than by direct sequencing. Subgroup analyses were conducted to investigate the significant heterogeneity among studies. The results (Table 2) showed that mutation rate differences were significantly larger in Asian than non-Asian population, in the studies with big sample sizes than those with small, and in cytological and blood samples than in tumor tissue, while not influenced much by study quality. Meta-regression analysis showed that the four factors contributed 27.6% to the overall heterogeneity, and sample size was the only factor that reached statistical significance (p = 0.008).

Funnel plots for detecting potential publication bias were not constructed because of the substantial heterogeneity between included studies [35, 36].

Table 2: Meta-analyses of mutation rates detected by ARMS and direct sequencing and their difference

Group/Subgroup

(number of studies)

ARMS mutation rate(95%CI) (a)

DS mutation rate(95%CI) (b)

Mutation rate difference(95%CI) (a-b)

Test for difference (a-b) across subgroups

Overall (26)

0.41(0.35,0.48)

0.28(0.23,0.35)

0.11(0.08,0.13)

Ethnicity

p < 0.001

UK (1)

0.09(0.06,0.14)

0.08(0.05,0.13)

0.01(0.00,0.02)

Asian (25)

0.43(0.38,0.49)

0.3(0.24,0.36)

0.12(0.09,014)

Sample size

p < 0.001

Big (12)

0.36(0.27,0.44)

0.27(0.19,0.37)

0.07(0.04,0.10)

Small (14)

0.48(0.39,0.57)

0.3(0.21,0.40)

0.17(0.12,0.22)

Study quality

p = 0.93

High (8)

0.45(0.38, 0.53)

0.33(0.26,0.40)

0.11(0.07,0.15)

Low (18)

0.39(0.31, 0.49)

0.26(0.19,0.36)

0.11(0.08,0.14)

Specimen type

p = 0.02

Tumor tissue (19)

0.42(0.34,0.50)

0.31(0.24,0.39)

0.08(0.05,0.10)

Cytological sample (6)

0.5(0.38,0.61)

0.36(0.24,0.50)

0.16(0.07,0.24)

Blood (3)

0.28(0.16,0.44)

0.05(0.03,0.10)

0.24(0.11,0.36)

ARMS: amplification refractory mutation system.

DS: direct sequencing.

Meta-analysis of mutation rate differences between ARMS and direct sequencing.

Figure 2: Meta-analysis of mutation rate differences between ARMS and direct sequencing. DS: direct sequencing. ARMS: amplification refractory mutation system.

Association of EGFR mutation status and clinical outcomes

As expected, the EGFR mutations detected were significantly associated with all the three clinical outcomes of EGFR TKIs treatment, regardless of the testing method used (Figure 3). The association of mutations by ARMS with clinical outcomes was consistently stronger than that of mutations by direct sequencing with clinical outcomes (RR for objective response rate: 4.01 vs 1.89, test for difference p = 0.13; HR for progression-free survival: 0.27 vs 0.42, test for difference p = 0.11; HR for overall survival: 0.46 vs 0.54, test for difference p = 0.61), although the difference between the two methods were not always statistically significant, possibly due to limited number of studies. For the same reason, subgroup analyses and funnel plots were not performed [37]. Sensitivity analyses according to study quality were also not conducted, because all studies in the meta-analyses were of relatively high quality.

Associations between clinical outcomes and

Figure 3: Associations between clinical outcomes and EGFR mutation status detected by ARMS and direct sequencing. A. Associations between objective response to TKIs treatment and EGFR mutation status detected by ARMS and direct sequencing. B. Associations between PFS and EGFR mutation status detected by ARMS and direct sequencing. C. Association between OS and EGFR mutation status detected by ARMS and direct sequencing. ARMS: amplification refractory mutation system. DS: direct sequencing. EGFR: epidermal growth factor receptor. PFS: progression-free survival. OS: overall survival.

Discussion

This systematic review included 26 studies with 3282 NSCLC patients and compared direct sequencing and ARMS in terms of the rate of mutations detected and the associations of detected mutations with clinical outcomes of EGFR TKIs treatment.

The pooled rate of mutations detected by ARMS was higher than that by direct sequencing and the difference was statistically significant. This finding could be a result of the differential sensitivity of the two methods [18]. Generally speaking, direct sequencing requires the mutant tumor cells to take up 20% or more of total tumor cells for detection [5, 6], while ARMS can detect mutations with much lower concentration (as low as 1%) because it selectively amplifies mutation sequences that are identified with specific probes [10, 33]. Thus, patients with EGFR mutations are more likely to be identified by ARMS than by direct sequencing. For example, in Hideharu’s [11] and Liu’s [18] studies, ARMS identified EGFR mutant patients that were deemed “negative” by direct sequencing, while no patients “negative” on ARMS was identified as mutants by direct sequencing. People may argue that ARMS is disadvantaged by the fact that it can detect only known mutants while direct sequencing can detect both known and unknown. But actually, common mutations (such as the exon 19 deletion and the exon 21 L858R point mutation) and many uncommon subtypes (such as exon 18 G719X, exon 20 Ins, exon 20 T790M), which comprise over 95% of all EGFR mutations [38], can be detected by both methods. Thus, this issue is unlikely to constitute a major limitation of ARMS.

The ARMS-detected EGFR mutations seemed to have stronger association with clinical outcomes of EGFR TKIs treatment than did direct-sequencing-detected mutations, although the differences between the two were not always statistically significant possibly due to the limited number of studies available.

One potential explanation for this finding might have to do with the proportion of key mutation subtypes that are highly related to the efficacy of TKIs. EGFR mutations contain multiple subtypes occurring within the exon 18/19/20/21, and they respond to TKIs differentially [38, 39]. Previous studies indicated that mutations in exon 18/19/21 are sensitive but those in exon 20 are resistant to TKIs [38]. Since direct sequencing could identify more other unknown subtypes than ARMS, the proportion of key common mutations out of all detected mutations was supposed to be lower for direct sequencing than for ARMS, leading to “dilution” of the association between direct-sequencing-detected mutations as a whole and the efficacy of TKIs, in contrast to the situation of ARMS. However, after revisiting all included studies, we found that the proportion of sensitive mutations (exon 18/19/21 mutations) out of all were around 98% for both ARMS and direct sequencing. Thus, this explanation is not supported by data from empirical studies.

Another potential mechanism is that direct sequencing has a higher false negative misclassification rate than does ARMS [33]. False negative patients are categorized as non-mutant ones, but actually they benefit from EGFR TKIs. Thus, a higher false negative rate means that the association between mutation status and treatment efficacy is diluted to a larger extent. This seems to be a more plausible explanation for our findings.

So far, we can see that ARMS is better than direct sequencing in terms of both the EGFR mutation rate and the ability of detected mutations in predicting efficacy of EGFR TKIs. However, based on this comparison alone, it is still not straightforward whether ARMS is more favorable than direct sequencing in clinical practice, as other factors should be taken into consideration as well when deciding which method to use. Gillian et al [4] proposed a framework for choosing an appropriate detection method, which consists of three main components, i.e. sample type, relevant expertise and equipment, and whether detection of known mutations only is sufficient.

The results of our subgroup analyses showed that the mutation rates detected by ARMS (range: 28% to 50%) varied less than those by direct sequencing (range: 5% to 36%) when different types of biological samples were used. This finding has important implications for clinical practice. Since tumor tissues are often difficult to obtain and frequently not available from advanced patients, cytological materials and blood samples have been suggested to be used as the substitutes. For example, the College of American Pathologists, the International Association for the Study of Lung Cancer and the Association for Molecular Pathology have recommended the use of cytological materials in EGFR mutation testing since October 2013 [40]. Given its better performance in these types of samples, ARMS seems to have more advantages than direct sequencing in clinical practice.

This study has several limitations. Firstly, the small number of included studies that evaluated the association of detected mutation status with clinical outcomes prevented us from drawing a firm conclusion on the superiority of predictive value of mutations detected by different methods. Secondly, for difference in mutation rates between the two methods, only about one third of the studies were of high quality. However, subgroup analysis showed that the results from high quality studies were consistent with those form low quality ones. Thus, we argue that this issue is unlikely to be a major problem. Thirdly, substantial heterogeneity was observed in the meta-analysis of mutation rate differences, but pre-planned subgroup and meta-regression analyses failed to identify the major source of the heterogeneity, which undermined the validity of our findings. Fourthly, test for publication bias was not performed, due to either the significant heterogeneity or the small number of included studies. Lastly, as most included studies were conducted in Asian population, caution should be taken in generalizing the results to Western population.

In conclusion, more NSCLC patients with EGFR mutations can be identified by ARMS than by direct sequencing, and those identified by ARMS seems to be able to benefit more from tyrosine kinase inhibitors than do those identified by direct sequencing. In terms of the technical performance alone, ARMS represents a valid alternative to direct sequencing for testing EGFR mutations.

Materials and methods

Search strategy

We systematically searched PubMed, EMBASE, China National Knowledge Infrastructure (CNKI, in Chinese) and Wanfang database (in Chinese) through May 2015 to identify relevant studies. The combination of the following three groups of terms (or their Chinese counterparts) were used for the search: (1) “lung” and “pulmonary”; (2) “cancer”, “carcinoma”, “adenocarcinoma”, and “tumor”; and (3) “sequenc*”, “amplification refractory mutation system”, “ARMS”, “allele-specific polymerase chain reaction ”, “allele-specific PCR”, “PASA”, and “ASP”. The search terms were limited to title/abstract and the studies were limited to “human” where possible. The abstracts of relevant conferences of American Society of Clinical Oncology, European Society for Medical Oncology European Lung Cancer Conference were also searched. The reference lists of eligible studies and relevant reviews were manually checked for additional studies.

Study selection

One reviewer (JTZ) screened the titles and abstracts of the retrieved records to judge their relevance, with the potentially eligible studies subject to full text examination. This process was double-checked by a second reviewer (QF). The disagreements between the two, if any, were resolved by discussion or by consulting a third reviewer (ZYY). To be eligible for inclusion into this meta-analysis, original studies had to meet the following criteria: (1) Patients were diagnosed with advanced NSCLC; (2) Pre-treatment EGFR mutation status of some or all patients was tested by both ARMS and direct sequencing; (3) For each patient tested, the samples used for ARMS and directing sequencing were from the same source, e.g. tumor tissue; (4) Mutation rate detected by each method was reported or could be calculated from reported data; and (5) (optional) the associations of EGFR mutation status detected by different methods with clinical outcomes of EGFR TKIs treatment were reported. If more than one record were identified for an eligible study, only the most complete one was included.

Data extraction

The following information was extracted by one reviewer(JLZ) from eligible studies using a pre-designed data extraction form and double-checked by a second reviewer (QF): (1) bibliographic information, such as first author’s name, year of publication, study country, and sample size; (2) patients’ characteristics, such as the proportions of male, smokers and adenocarcinoma; (3) results of EGFR mutation testing, such as the number of patients with EGFR mutations detected by ARMS and direct sequencing, respectively, the types of specimen used; (4) clinical outcomes of EGFR TKIs treatment, including objective response rate, progression-free survival and overall survival, and their corresponding risk ratios (RRs) or hazard ratios (HRs); and (5) other information needed for study quality assessment (see below).

Quality assessment

The data on EGFR mutation rate were obtained from cross-sectional studies or the baseline data of cohort studies, which was similar to cross-sectional studies in nature. We employed an 11-item tool recommended by the Agency for Healthcare Research and Quality (AHRQ) to assess the quality of those data, which had been employed by previous systematic reviews [41, 42]. As the item 8 for confounding and item 11 for follow-up of the tool were not applicable in the quality assessment of prevalence study, only nine items remained in our assessment, with 1 score assigned to each item if it was satisfied and 7 or above out of 9 scores regarded as high quality.

The data on association of EGFR mutations with clinical outcomes were obtained from cohort studies, for which the Newcastle-Ottawa Scale was employed for quality assessment. The Newcastle-Ottawa Scale consists of three main constructs, i.e. selection (4 items), comparability (1 item) and outcome (3 items) [43]. One star was given to each item if it was met by the study being assessed, except item 5 for comparability, which could be given a maximum of 2 stars. A study with 7 or above out of 9 stars were regarded as high quality [44]. We did not assess the quality of conference abstracts, because the information needed for quality assessment were usually not available from them and their quality could turn out to be misleadingly low.

Statistical analysis

The mutation rates detected by ARMS and direct sequencing, respectively, were combined to obtain an overall mutation rate for each method. To evaluate the difference between the two method, a mutation rate difference (ARMS minus direct sequencing) was calculated within each study and then the rate differences from all relevant studies were synthesized to obtain an overall estimates.If the difference was statistically greater than zero, then the mutation rate by ARMS was considered as higher than that by direct sequencing. To compare the predictive ability of EGFR mutation status detected by ARMS with that by direct sequencing, firstly the associations of EGFR mutation status detected by each method with clinical outcomes were meta-analyzed across studies to obtain overall estimates, and then the two overall estimates for each outcome were compared to see which one was stronger. Three clinical outcomes, i.e. objective response rate, progression-free survival and overall survival, were of our interest. The association of EGFR mutation status with the outcomes was measured by RR (for objective response) or HR (for progression-free survival and overall survival).

Meta-analyses were conducted with the random-effects model. Heterogeneity was assessed by the Cochran’s Q test and the I2 statistic. A P value ≤ 0.10 for the Q test or an I2 ≥ 50% suggested the presence of substantial heterogeneity between studies. The difference in association of clinical outcomes and EGFR mutation status detected by two methods was examined by a method similar to the heterogeneity test. Subgroup analyses stratified by sample size, study quality, ethnicity, specimen types and meta-regression incorporating all these factors were conducted to investigate the potential sources of heterogeneity. We planned to assess publication bias by using funnel plots, but did not actually do so because of the substantial heterogeneity in the meta-analyses of EGFR mutation rates and the limited number (<10) of studies in the meta-analyses of RRs and HRs [35, 36]. We employed a two-tailed significance level of 0.05 for all the statistical tests except for the heterogeneity tests, for which the significance level is 0.10. Data analyses were conducted with RevMan 5.1, STATA 11.0 and Comprehensive Meta-Analysis 2.2.

Compliance with ethical standards

Ethical approval: This article does not contain any studies with human participants or animals performed by any of the authors.

Abbreviations

EGFR, epidermal growth factor receptor; TKI, tyrosine kinase inhibitor; NSCLC, non-small cell lung cancer; ARMS, amplification refractory mutation system; CI, confidence interval; RR, risk ratios; HR, hazard ratio.

Author contributions

Study design: JLT, ZYY; literature search and screen: ZYY, JTZ; data extraction: JTZ, QF; quality assessment: QF, ZYY; data analysis: QF, ZYY; manuscript draft: QF; critical review: JLT, ZYY.

Conflicts of interest

All authors declare no conflicts of interest.

Funding

This study has not received any funding from any sources.

References

1. Goto K, Satouchi M, Ishii G, Nishio K, Hagiwara K, Mitsudomi T, Whiteley J, Donald E, McCormack R, Todo T. An evaluation study of EGFR mutation tests utilized for non-small-cell lung cancer in the diagnostic setting. Ann Oncol. 2012; 23: 2914-9. doi: 10.1093/annonc/mds121.

2. Hartmann JT, Haap M, Kopp HG, Lipp HP. Tyrosine kinase inhibitors - a review on pharmacology, metabolism and side effects. Curr Drug Metab. 2009; 10: 470-81.

3. Lynch TJ, Bell DW, Sordella R, Gurubhagavatula S, Okimoto RA, Brannigan BW, Harris PL, Haserlat SM, Supko JG, Haluska FG, Louis DN, Christiani DC, Settleman J, et al. Activating mutations in the epidermal growth factor receptor underlying responsiveness of non-small-cell lung cancer to gefitinib. N Engl J Med. 2004; 350: 2129-39. doi: 10.1056/NEJMoa040938.

4. Ellison G, Zhu G, Moulis A, Dearden S, Speake G, McCormack R. EGFR mutation testing in lung cancer: a review of available methods and their use for analysis of tumour tissue and cytology samples. J Clin Pathol. 2013; 66: 79-89. doi: 10.1136/jclinpath-2012-201194.

5. Li J, Wang L, Mamon H, Kulke MH, Berbeco R, Makrigiorgos GM. Replacing PCR with COLD-PCR enriches variant DNA sequences and redefines the sensitivity of genetic testing. Nat Med. 2008; 14: 579-84. doi: 10.1038/nm1708.

6. Ogino S, Kawasaki T, Brahmandam M, Yan L, Cantor M, Namgyal C, Mino-Kenudson M, Lauwers GY, Loda M, Fuchs CS. Sensitive sequencing method for KRAS mutation detection by Pyrosequencing. J Mol Diagn. 2005; 7: 413-21. doi: 10.1016/S1525-1578(10)60571-5.

7. Franklin WA, Veve R, Hirsch FR, Helfrich BA, Bunn PA. Epidermal growth factor receptor family in lung cancer and premalignancy. Semin Oncol. 2002; 29: 3-14. doi: 10.1053/sonc.2002.31520.

8. Dufort S, Richard MJ, Lantuejoul S, de Fraipont F. Pyrosequencing, a method approved to detect the two major EGFR mutations for anti EGFR therapy in NSCLC. J Exp Clin Cancer Res. 2011; 30: 57. doi: 10.1186/1756-9966-30-57.

9. Yang JC, Wu YL, Chan V, Kurnianda J, Nakagawa K, Saijo N, Fukuoka M, McWalter G, McCormack R, Mok TS. Epidermal growth factor receptor mutation analysis in previously unanalyzed histology samples and cytology samples from the phase III Iressa Pan-ASia Study (IPASS). Lung Cancer. 2014; 83: 174-81. doi: 10.1016/j.lungcan.2013.11.021.

10. Ellison G, Donald E, McWalter G, Knight L, Fletcher L, Sherwood J, Cantarini M, Orr M, Speake G. A comparison of ARMS and DNA sequencing for mutation analysis in clinical biopsy samples. J Exp Clin Cancer Res. 2010; 29: 132. doi: 10.1186/1756-9966-29-132.

11. Kimura H, Fujiwara Y, Sone T, Kunitoh H, Tamura T, Kasahara K, Nishio K. High sensitivity detection of epidermal growth factor receptor mutations in the pleural effusion of non-small cell lung cancer patients. Cancer Sci. 2006; 97: 642-8. doi: 10.1111/j.1349-7006.2006.00216.x.

12. Lee DH. Use of Scorpion-amplified refractory mutation system is more sensitive to detect EGFR mutant and responders in patients with advanced/metastatic NSCKL before starting firstline erlotinib therapy. AACR-IASLC Joint Conference on Molecular Origins of Lung Cancer. Coronado: International Association for the Study of Lung Cancer; 2010. p. s236.

13. Chu H, Zhong C, Xue G, Liang X, Wang J, Liu Y, Zhao S, Zhou Q, Bi J. Direct sequencing and amplification refractory mutation system for epidermal growth factor receptor mutations in patients with non-small cell lung cancer. Oncol Rep. 2013; 30: 2311-5. doi: 10.3892/or.2013.2709.

14. Dou Y, Qi X, Xin L, Chen D, Zhang H, Cui Q. [Value of detecting epidermal growth factor receptor mutations in non-small cell lung cancer tissue by TaqMan-amplification refractory mutation system] [in Chinese]. Chin Med J (Engl). 2013; 93: 768-71. doi: 10.3760/cma.j.issn.0376-2491.2013.10.014.

15. Horiike A, Kimura H, Nishio K, Ohyanagi F, Satoh Y, Okumura S, Ishikawa Y, Nakagawa K, Horai T, Nishio M. Detection of epidermal growth factor receptor mutation in transbronchial needle aspirates of non-small cell lung cancer. Chest. 2007; 131: 1628-34. doi: 10.1378/chest.06-1673.

16. Li C, Wu J, Wang Z, Feng J. [A comparison of direct sequencing and ARMS assay performance in EGFR mutation analysis of non-small cell lung cancer patients] [in Chinese]. Chinese J lung cancer. 2014; 17: 606-11. doi: 10.3779/j.issn.1009-3419.2014.08.05.

17. Li H, Mao Y, Tang S, Jiang Y. [A feasibility analysis of detecting K-ras/EGFR mutation in colorectal cancer patients and lung cancer patients by ARMS] [in Chinese]. Annual Conference on laboratory medicine in 2011in Zhejiang China. Zhejiang, China; 2011.

18. Liu Y, Liu B, Li XY, Li JJ, Qin HF, Tang CH, Guo WF, Hu HX, Li S, Chen CJ, Gao HJ, Liu XQ. A comparison of ARMS and direct sequencing for EGFR mutation analysis and tyrosine kinase inhibitors treatment prediction in body fluid samples of non-small-cell lung cancer patients. J Exp Clin Cancer Res. 2011; 30: 111. doi: 10.1186/1756-9966-30-111.

19. Morinaga R, Okamoto I, Fujita Y, Arao T, Sekijima M, Nishio K, Ito H, Fukuoka M, Kadota JI, Nakagawa K. [Association of epidermal growth factor receptor (EGFR) gene mutations with EGFR amplification in advanced non-small cell lung cancer]. Cancer Sci. 2008; 99: 2455-60. doi: 10.1111/j.1349-7006.2008.00962.x.

20. Qian X, Wang S, Shen Y, Wu D, Zhang Z, Feng Y, Han X, Shi Y. [Methodology comparison and influence factors analysis of epidermal growth factor receptor mutation detection] [in Chinese]. Chin Med J (Engl). 2015; 95: 106-11. doi: 10.3760/cma.j.issn.0376-2491.2015.02.007.

21. Qin L, Zhong W, Zhang L, Li L, Wang M. Comparison of three methods for detecting epidermal growth factor receptor mutations in plasma DNA samples of Chinese patients with advanced non-small cell lung cancer. Chin Med J (Engl). 2011; 124: 887-91. Available from http://www.ncbi.nlm.nih.gov/pubmed/21518597.

22. Shujie A, Hou Y, Liu Y, Xu C, Xin Z, Huang J. [Comparison of PCR direct sequencing and ARMS in detecting EGFR mutation in NSCLC patients] [in Chinese]. J Clin Exp Pathol. 2014; 30: 328-31.

23. Wang J, Cao L, Wu Q, Xu X, Hu W, Chen K. [EGFR mutation in NSCLC patients in Anhui China] [in Chinese]. China J Gerontol. 2012; 5: 1822-4. doi: 10.3969/j.issn.1005-9202.2012.09.019.

24. Wang S, Han XH, Li JL, Hu XS, Wang XY, Gui L, Zhao L, Sun Y, Shi YK. [Detection of epidermal growth factor receptor gene mutations in advanced non-small cell lung cancer] [in Chinese]. Chinese J Pathol. 2012; 41: 530-3. doi: 10.3760/cma.j.issn.0529-5807.2012.08.005.

25. Wang Z, Wu X, Shi Y, Han X, Cheng G, Li L, Zhang L, Zhang Y, Mu X, Zhu G, Fan Z, Yang L, Di J, et al. [A standard protocol for detection of EGFR mutations in cytologic specimens] [in Chinese]. Chinese J Oncol. 2014; 36: 516-21. doi: 10.3760/cma.j.issn.0253-3766.2014.07.008.

26. Xu H, Hong W, Xie S, Pan X, Bai Y, Xiangyang L, Chen B. [Detecting EGFR mutation in NSCLC patients by ARMS and its clinical significance] [in Chinese]. China J Appl Physiol. 2014; 30: 158-9. doi: 10.13459/j.cnki.cjap.2014.02.017.

27. Zhang J, Liang ZY, Zeng X, Wu SF, Gao J, Liu TH. [Detection of epidermal growth factor receptor gene mutations in non-small cell lung cancers by real-time polymerase chain reaction using scorpion amplification refractory mutation system] [in Chinese]. Chinese J Pathol. 2008; 37: 294-9. doi: 10.3321/j.issn:0529-5807.2008.05.002.

28. Zhang X, Wu YL, Wang J, Chu XY, Liu ZD, Shen Y, Ma H, Fu XN, Hu J, Zhou NK, Liu Y, Zhou X, Wang JJ, et al. A prospective comparison study on EGFR mutations by direct sequencing and ARMS in completely resected Chinese non-small cell lung cancer with adenocarcinoma histology (ICAN). ASCO Annual Meeting. 2013. p. 1547.

29. Zhao J, Wang X, Xue L, Xu N, Ye X, Zeng H, Lu S, Huang J, Akesu S, Xu C, He D, Tan Y, Hong Q, et al. The use of mutation-specific antibodies in predicting the effect of EGFR-TKIs in patients with non-small-cell lung cancer. J Cancer Res Clin Oncol. 2014; 140: 849-57. doi: 10.1007/s00432-014-1618-2.

30. Zhao J, Xiaoying W, Haiying Z, Jie H, Qunying H, Guanshan Z, Yingyong H, Zhang X. [The comparison of EGFR mutationn detection in clinical biopsy samples of lung cancer by ARMS and direct sequecing] [in Chinese]. China Oncol. 2013; 23: 106-13. doi: 10.3969/j.issn.1007-3969.2013.02.005.

31. Zhao J, Zhao J, Huang J, Chen Y, Jiang J, Wu W, Wang P, Liu L, Li L, Wu L, Wang M, Chen W. A novel method for detection of mutation in epidermal growth factor receptor. Lung Cancer. 2011; 74: 226-32. doi: 10.1016/j.lungcan.2011.02.015.

32. Zhou Q, Zhang XC, Chen ZH, Yin XL, Yang JJ, Xu CR, Yan HH, Chen HJ, Su J, Zhong WZ, Yang XN, An SJ, Wang BC, et al. Relative abundance of EGFR mutations predicts benefit from gefitinib treatment for advanced non-small-cell lung cancer. J Clin Oncol. 2011; 29: 3316-21. doi: 10.1200/JCO.2010.33.3757.

33. Zhou S, Zhou M, Peng H, Zeng A, Yu Q, Song X. Comparison of ARMS and direct sequencing for detection of EGFR mutation and prediction of EGFR-TKI efficacy between surgery and biopsy tumor tissues in NSCLC patients. Med Oncol. 2014; 31: 926. doi: 10.1007/s12032-014-0926-3.

34. Zuo M, Zhou S, Yu Q, Zeng A, Song X. [Determination of mutations in exons 19 and 21 of epidermal growth factor receptor in tumor tissue ans serum samples from Chinese patients with non-small-cell lung cancer] [in Chinese]. Chinese J Pathophysiol. 2013; 29: 839-44.

35. Stuck AE, Rubenstein LZ, Wieland D. Bias in meta-analysis detected by a simple, graphical test. Asymmetry detected in funnel plot was probably due to true heterogeneity. BMJ. 1998; 316: 469; author reply 470-1.

36. Tang JL, Liu JL. Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol. 2000; 53: 477-84.

37. Higgins JP, Sally G. Cochrane handbook for systematic reviews of interventions (version 5.1.0). 2011. Available from www.cochrane-handbook.org.

38. Castellanos E, Feld E, Horn L. Driven by Mutations: The Predictive Value of Mutation Subtype in EGFR-Mutated Non-Small Cell Lung Cancer. J Thorac Oncol. 2016; 12: 612-623. doi: 10.1016/j.jtho.2016.12.014.

39. Liu Y, Ren Z, Wang J, Zhang S. Epidermal growth factor receptor-tyrosine kinase inhibitor therapy is especially beneficial to patients with exon 19 deletion compared with exon 21 L858R mutation in non-small-cell lung cancer: Systematic review and meta analysis. Thorac cancer. 2016; 7: 406-14. doi: 10.1111/1759-7714.12343.

40. Lindeman NI, Cagle PT, Beasley MB, Chitale DA, Dacic S, Giaccone G, Jenkins RB, Kwiatkowski DJ, Saldivar JS, Squire J, Thunnissen E, Ladanyi M. Molecular testing guideline for selection of lung cancer patients for EGFR and ALK tyrosine kinase inhibitors: guideline from the College of American Pathologists, International Association for the Study of Lung Cancer, and Association for Molecular Patho. J Thorac Oncol. 2013; 8: 823-59. doi: 10.1097/JTO.0b013e318290868f.

41. Chen X, Rong SS, Xu Q, Tang FY, Liu Y, Gu H, Tam POS, Chen LJ, Brelén ME, Pang CP, Zhao C. Diabetes mellitus and risk of age-related macular degeneration: a systematic review and meta-analysis. PLoS One; 2014; 9: e108196. doi: 10.1371/journal.pone.0108196.

42. Colson J, Koyyalagunta D, Falco FJE, Manchikanti L. A systematic review of observational studies on the effectiveness of opioid therapy for cancer pain. Pain Physician. 2011; 14: E85-102.

43. Wells G, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa (NOS) for assessing the quality of nonrandomised studies in meta-analysis. The Ottawa Hospital Research Institute. 2008 Available 2017 Mar 7, fromhttp://www.ohri.ca/programs/clinical_epidemiology/oxford.asp

44. Liu YJ, Zhan J, Liu XL, Wang Y, Ji J, He QQ. Dietary flavonoids intake and risk of type 2 diabetes: a meta-analysis of prospective cohort studies. Clin Nutr. 2014; 33: 59-63. doi: 10.1016/j.clnu.2013.03.011.


Creative Commons License All site content, except where otherwise noted, is licensed under a Creative Commons Attribution 3.0 License.
PII: 19110