Diagnostic accuracy of DNA methylation for head and neck cancer varies by sample type and number of markers tested

Abnormal methylation of certain cancer related genes strongly predicts a diagnosis of head and neck cancer (HNC), while the predictive power of methylation of other DNA markers for HNC remains unclear. To systemically assess the diagnostic value of DNA methylation patterns for HNC and the effect of methylation platform techniques and sample types, we performed a PubMed search for studies of the correlation between DNA methylation and HNC completed before July 2016, and extracted the sensitivity and specificity for methylated biomarkers. Across these studies, DNA methylation showed high sensitivity for diagnosing HNC in solid tissue (0.57), and high specificity in saliva (0.89). Area under the curve (AUC) from summary receiver operating characteristic (SROC) curves revealed that DNA methylation had more diagnostic power in solid tissue (AUC = 0.82) than saliva (AUC = 0.80) or blood (AUC = 0.77). Combinations of multiple methylated genes were more sensitive diagnostic markers than single methylated genes. Our results suggest that the diagnostic accuracy of methylated biomarkers for HNC varied by sample type and were most accurate when results from multiple sample types were considered.


INTRODUCTION
For this study, we selected from the literature reports of common squamous-cell carcinomas of the oral cavity, pharynx, and larynx (HSCC), which account for 90% of HNC [1].Approximately 650,000 cases of HNC are identified per year [2] and the disease has high recurrence rates and poor prognoses due to distant metastasis [3].Late diagnosis results in poorer prognosis [4].Improved diagnostic accuracy for HNC could lead to earlier diagnosis, increasing patient survival rates.
Variations in the epigenetic modifications, such as DNA methylation in gene promoters, often inhibit gene transcription and protein translation, important factors in human carcinogenesis.A number of genes are frequently methylated in HNC, including p16, DAPK1, and RASSF1A [5,6], or hypermethylated in CpG islands, such as hMLH1 [7], KIF1A, and EDNRB [8].Many groups have identified abnormally methylated genes as HNC diagnostic biomarkers but their predictive accuracies fluctuated among different sample types.Moreover, there are no systematic diagnostic accuracy studies or metaanalyses regarding the various sample types in HNC.We performed a systematic review and stratified metaanalysis of previous HNC studies based on sample types and diagnostic markers.We aim to provide more reliable evidence to clarify the diagnostic accuracy of DNA methylation markers, according to published reports that computed sensitivity and specificity.

Study characteristics
We identified 108 papers in a search of the PubMed database.Seventy-nine were excluded based on screening the title and abstract, including twenty-eight papers that did not involve HNC, thirty-six papers that did not investigate the cancer diagnoses, eleven papers that did not include a diagnosis based on DNA methylation, and four reviews.We obtained the full texts of twentynine papers; of these five further papers were excluded,

Exploration of heterogeneity analysis
To determine the effect model of diagnostic accuracy, we conducted heterogeneity tests for PLR, NLR, and DOR and found a significant heterogeneity of NLR in the solid tissue and saliva studies (Table 2).DOR showed no heterogeneity in the solid tissue or blood studies.PLR showed low heterogeneity in the solid tissue studies and no heterogeneity in the saliva or blood studies.The heterogeneity of NLR varied among the sample types.
We used meta-regression analysis to assess whether publication year, sample type, DNA methylation detection technique, or the methylation panel corresponding to single or multiple methylated biomarkers affected the diagnostic accuracy for HNC.The true and false positive rates were used as the responses in meta-regression analyses.As shown in Table 3, the p values of sensitivity and false positive rates were not significant, suggesting that publication year, biomarker technique, and sample types did not affect the false positive rate.
In addition, we evaluated the diagnostic power based on the types of methylation biomarkers.The single methylation markers had overall sensitivity and specificity of 0.51 (95% CI: 0. 45 S represented single methylated gene as diagnostic marker, and M represented combination of multiple methylated genes as diagnostic marker. 0.90), respectively.The diagnostic sensitivity of multiple methylation markers was 0.55 (95% CI: 0.47-0.63)and the specificity was 0.88 (95% CI: 0.85-0.90).In general, methylated biomarkers showed differential diagnostic accuracy in all three sample types, and the diagnostic power of integrating multiple methylated genes was better than with a single gene.
According to the sensitivity and specificity results from each trial, the regression coefficients of the SROC curves were near 0 for the three sample types (Table 4).The AUC curve indicated that samples from solid tissue had the highest diagnostic accuracy, with an AUC value of 0.82 (Figure 3) and a Q* metric of 0.81 (95% CI: 0.77-0.85),whereas the sensitivity was identical to the specificity (Table 4, Figure 3).In addition, the panel of multiple methylated genes showed higher AUC value than a single methylated gene (0.85 vs. 0.77).These results suggest that the combination of multiple methylation biomarkers in solid tissue has better diagnostic accuracy, with higher sensitivity in saliva, which could be useful for HNC screening.

Publish bias and sensitivity analysis
The risk of bias for each study was first assessed (Supplementary Figure 1).As shown in Figure 4, 92% of studies showed a low or unclear risk of bias for many bias items and only 5 ~ 18% of studies clearly reported a non-  C    random sequence generation, no blinding, or incomplete blinding.More than 50% of the studies had independent data collection, assessment of DNA methylation, and interpretation of outcomes.In total, 75% of the studies show the sensitivities and specificities for all of the evaluated methylation biomarkers, which suggested no selective reporting.Ten studies were reported to be free of other sources of bias.Based on these metrics we deemed the quality of the studies included in the following metaanalysis to be acceptable.By testing the relationship between the DOR and its standard error, we assessed the publication bias effects of the sample size for each diagnostic consequence.The potential publication bias was ascertained in these studies using symmetrical funnel plots for the saliva, solid tissue, and blood samples.We found that some studies corresponding to saliva (Figure 5A) or solid tissue (Figure 5B) were not inside the funnel.Begg's testing demonstrated that there was no significant publication bias in the three sample types from HNC patients.The studies with smaller sample sizes did not tend towards higher levels of accuracy.
A one-way sensitivity analysis was performed to evaluate the robustness of the results of this meta-analysis with respect to study and biomarker.The pooled specificity was not influenced when removing one study or diagnostic biomarker (Supplementary Table   2).The exclusion of individual methylated markers had no effect on diagnostic sensitivity.

DISCUSSION
DNA methylation has previously been demonstrated to be a potentially useful marker for multiple cancers [52].
In this study, we analyzed the accuracy of methylated genes for diagnosing HNC based on previously published studies.Overall, the sensitivity of the DNA methylation was 0.47 in saliva, 0.57 in solid tissue and 0.46 in blood, and the specificity was 0.89, 0.88 and 0.85, respectively.We found that DNA methylation had low sensitivity but high specificity in the diagnosis of HNC.Different samples showed similar specificity but differential sensitivity.Seven papers corresponding to eleven studies were used to estimate the diagnostic accuracy of DNA methylation in blood.The small number of studies may provide misleading conclusions on the diagnostic power in blood that should be further evaluated.Moreover, testing for multiple methylated genes showed higher sensitivity than single methylated genes.Ideally, we should assess the overall diagnostic accuracy of the same combinations of methylated genes or single genes in three different sample types, but were limited by the number of studies available in the literature.We provide detailed information on the combinations of multiple methylated genes in Figure 2. Behind each author's name is the applicable DNA methylation marker information for the specific study.The evidence from this study suggests that DNA methylation biomarkers might be effective tools for detecting HNC.It should also be noted that the diagnostic accuracy of DNA methylation depends on the sample type and diagnostic markers studied.
Many of the studies in our analysis detected DNA methylation based on methylation-specific PCR (MSP), one of the principle methods of investigating DNA methylation.MSP typically overestimates the extent of methylation, which would affect the diagnosis of HNC.We studied whether the assay method of DNA methylation affects the HNC diagnosis, but found no significant differences among these methods.

Data sources and search strategy
We searched for diagnostic studies in PubMed published before July 2016.The search strategy for PubMed was ("head and neck neoplasms" [MeSH Terms] OR "head and neck cancer" [All Fields]) AND ("sensitivity and specificity"[MeSH Terms] OR "sensitivity and specificity" [All Fields] AND ("DNA methylation" [MeSH Terms] OR "DNA methylation" [All Fields]) to find appropriate studies published in English prior to July 25, 2016.We searched for published trials that evaluated the diagnostic accuracy of one or more methylated biomarkers.In addition, we added all studies included in a previous meta-analysis into our analysis [56].

Study selection
Two reviewers independently filtered the search results by the title and abstract.Studies were excluded if they did not pertain to DNA methylation, were not related to HNC, were not diagnostic studies, or were reviews.Two authors obtained the full text of each paper and further filtered out the studies that did not supply sensitivity or specificity data for HNC diagnosis or that concerned the diagnosis of recurrence.We collected the authors' names; institutions; publication dates; sample types, including saliva, solid tissue, and blood; methylated biomarkers; and techniques of DNA methylation detection for all of the studies.All studies were evaluated independently and discussed by the authors until any inconsistencies were resolved.

Data extraction and quality assessment
A standardized data extraction form was used to extract the information from each paper including the first author, year of publication, country in which the study was performed, number of cases and controls, sample types, methylated gene names, DNA methylation detection techniques, number of methylated genes used as diagnostic biomarkers, and records of true positive, false positive, true negative and false negative results in head and neck cancer.Simultaneously, we evaluated the risk of bias according to pre-specified criteria from the Cochrane Collaboration's tool for assessing the risk of bias [57].The two authors independently checked the risk of bias assessment for each trial using standardized methods including the following (Supplementary Table 4): whether a study showed selection bias including sequencing generation and allocation concealment; whether the performance was biased regarding the blinding of patients and study personnel; whether the detection was biased according to the assessment of the blinding of the outcome; and whether the attrition and reporting were biased by being based on incomplete outcome data and selective reporting, respectively.

Data synthesis and statistical analysis
We extracted the number of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) results based on the remaining studies.The summary effects of the positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), and 95% confidence intervals (CI) were further computed to estimate the statistical heterogeneities through Cochran's Q test, that approximately follows a χ 2 distribution with k-1 degrees of freedom (where k is the number of included studies) [58].We assessed I 2 = ((Q-(k-1))/Q) ×100%, which ranged from 0-100%.I 2 represents different degrees of heterogeneity, including low (0-25%), moderate (25-50%), high (50-75%) and very high (75-100%) [58].The p value of the heterogeneity test determined whether a fixed-or random-effects model was used to estimate the diagnostic effects, and the significance level of heterogeneity was considered to be 0.05.The overall sensitivity and specificity were estimated to represent the diagnostic power of DNA methylation for the detection of head and neck cancer.For the overall diagnostic accuracy, an SROC curve was generated based on the sensitivity and specificity of each study using the equation D=a+b×S, where D = logit(Se) -logit(1-Sp) = log(OR) and S = logit(Se) +logit(1-Sp) [59].In the regression equation, D represents the diagnostic power of the methylated biomarkers, and S represents the threshold of the classification between positive and negative.Because the parameters D and S are from different studies, the values of the regression coefficient closer to 0 suggested less significant heterogeneity in various studies, which corresponds to diagnostic accuracy.The area under the SROC curve (AUC) value was estimated to measure the overall diagnostic power of DNA methylation in individual studies.In addition, Q* = Se = 1-Sp was computed according to the regression equation of SROC, where Se = exp(a/2)/[1+exp(a/2)] and 1-Sp = 1/[1+exp(a/2)], which suggested that the diagnostic threshold for a correct diagnosis was constant for all of the subjects.Publication bias was evaluated using funnel plot analyses and Begg's and Egger's tests, with a significance level defined as 0.01.We used the mada and metafor package in R to performed the statistical analysis [60].

Figure 1 :
Figure 1: Flow chart showing the study retrieval process.

Figure 2 (
Figure 2 (Continued ): B. Forest plot of estimate of sensitivity and specificity of methylated biomarkers in tissue.C. Forest plot of estimate of sensitivity and specificity of methylated biomarkers in blood.
77 0.71 (0.59 -0.80) Sensitivity and Specificity represent the independent pooled sensitivity (Se) and specificity (Sp) using fixed effect model. a and b represent the intercept and slope of SROC curve.AUC represent the area under SROC curve.Q* represents the diagnostic threshold at which the probability of a correct diagnosis is constant for all subjects and calculated as exp(a/2)/ [1+exp(a/2)].

Figure 3 :
Figure 3: SROC curves of studies relating to the detection of HNC in different biopsy types.

Figure 4 :
Figure 4: Risk of bias graph.Two authors independently evaluated the items of bias.If the study reported all of the sensitivities and specificities of genes that were measured DNA methylation status, selective reporting was defined as low risk.

2
, 3).The sensitivity was increased when the studies by Carvalho et al. and Adams et al. were excluded, and was decreased when the study by Arantes et al. was excluded (Supplementary Table

Figure 5 :
Figure 5: Funnel plot to assess bias in estimates of diagnostic odds ratio caused by small-study effects.A. Saliva.B. Solid tissue.C. Blood.

Table 1 : The included studies investigating the diagnosis of DNA methylation biomarkers in head and neck cancer Study Country Case# Control# Sample Biomarker Technique Methylated genes
-0.57) and 0.87 (95% CI: 0.84-

Table 2 : Heterogeneity analysis of diagnostic effects
PLR: positive likelihood ratio.NLR: negative likelihood ratio.DOR: diagnostics odd ratio.Estimate [95% CI]: the pooled effect measure with the corresponding 95% confidence interval.Log (Estimate) [95% CI]: logarithmic transformation of the pooled effect measure with the corresponding 95% confidence interval.df: degrees of freedom.Q and P-value were the Q value and p value of Cochran's Q test.

Table 3 : Meta-regression analysis
Figure 2: Forest plot of estimate of diagnostic accuracy using methylated biomarkers. A. Forest plot of estimate of sensitivity and specificity of methylated biomarkers in saliva.