Cross-platform comparison for the detection of RAS mutations in cfDNA (ddPCR Biorad detection assay, BEAMing assay, and NGS strategy)

CfDNA samples from colon (mCRC) and non-small cell lung cancers (NSCLC) (CIRCAN cohort) were compared using three platforms: droplet digital PCR (ddPCR, Biorad); BEAMing/OncoBEAM™-RAS-CRC (Sysmex Inostics); next-generation sequencing (NGS, Illumina), utilizing the 56G oncology panel (Swift Biosciences). Tissue biopsy and time matched cfDNA samples were collected at diagnosis in the mCRC cohort and during 1st progression in the NSCLC cohort. Excellent matches between cfDNA/FFPE mutation profiles were observed. Detection thresholds were between 0.5–1% for cfDNA samples examined using ddPCR and NGS, and 0.03% with BEAMing. This high level of sensitivity enabled the detection of KRAS mutations in 5/19 CRC patients with negative FFPE profiles. In the mCRC cohort, comparison of mutation results obtained by testing FFPE to those obtained by testing cfDNA by ddPCR resulted in 47% sensitivity, 77% specificity, 70% positive predictive value (PPV) and 55% negative predictive value (NPV). For BEAMing, we observed 93% sensitivity, 69% specificity, 78% PPV and 90% NPV. Finally, sensitivity of NGS was 73%, specificity was 77%, PPV 79% and NPV 71%. Our study highlights the complementarity of different diagnostic approaches and variability of results between OncoBEAM™-RAS-CRC and NGS assays. While the NGS assay provided a larger breadth of coverage of the major targetable alterations of 56 genes in one run, its performance for specific alterations was frequently confirmed by ddPCR results.


Comparison of ddpCr, Beaming and nGS panels
Biorad's KRAS digital droplet PCR panel is composed of a single PCR reaction that measures the seven most frequently mutated somatic mutations observed in NSCLC and mCRC patients in KRAS exon 2. Thus, NRAS mutations were not possible to assess with this assay. The OncoBEAM-TM-RAS-CRC™ Panel kit detects 16 KRAS mutations and 18 NRAS mutations, enabling the analysis of 34 somatic RAS alterations in a single run. The NGS sequencing panel fully explores exons 2, 3, and 4 of both KRAS and NRAS, providing the largest breadth of coverage to enable a complete survey of mutations within 263 amplicons of 56 genes, including coverage of 104 exon regions (Figure 1). Similarly, the custom panel employed for the FFPE NGS sequencing fully sequenced exon 2, 3 and 4 of KRAS and NRAS.

evaluation of assay sensitivity and specificity
Sensitivity and specificity of the assays were defined as their ability to accurately detect the true-positive and true-negative statuses of KRAS and NRAS mutations for each sample, compared to results obtained using Horizon reference standards (Horizon Diagnostics, Cambridge, UK) and to paired FFPE samples (used as clinical reference material). Horizon's cfDNA products are all derived from human cell lines, with DNA fragmented to an average size of 160 bp, which mimic the length of DNA fragments observed in plasma cfDNA from patients with cancer.
Sensitivity was also assessed as the ability of each assay to detect a specific fraction of MT/WT signal. For OncoBEAM-TM-RAS-CRC and Biorad's ddPCR method, we used a standard commercial sample of Horizon's Multiplex I cfDNA Reference Standard (Horizon Diagnostics, HD780, Cambridge, UK). This ladder set covers multiple engineered single nucleotide variants/ polymorphisms (SNVs/SNPs) with eight mutations at the specific allelic frequencies of 6.3%, 1.3%, and 0.13% (Table S1A). Copy number values measured via Biorad's ddPCR on KRAS exon 2 are provided with each control. At least three independent assays were performed. Using this reference standard, we were able to: (i) analyze the internal sensitivity and specificity of each assay, (ii) ascertain detection and quantification thresholds, and (iii) routinely assess the performance of each detection assay.
For NGS, the commercial Tru-Q5 DNA reference standard (Horizon Diagnostics, HD732, Cambridge, UK) was used since it is appropriate for all NGS library preparations including targeted amplicon panels. This control covers multiple endogenous SNPs, insertions and deletions and covers multiple engineered mutations at 2.5% (for verified variants). The Tru-Q5 Reference Standard was manufactured using twenty engineered cell lines and pooled to generate a multiple Allelic Frequency multiplex sample (Supplementary Table 1B).

Supplementary File 2: Supplementary data for results internal performance and sensitivity of Biorad's ddpCr
To deal with internal specificity, we first assessed the number of MT haploid GE detected in the commercial WT control DNA. The maximum false-positive count was observed with 20,000 WT haploid GE. Here, we found a mean of 10 MT haploid GE (±1.1) associated with a 0.05% allelic frequency false-positive rate inferior to 0.08%. By contrast, when using the 1,000 WT haploid GE control, we found a GE mean of 0.8 (±2.6) corresponding to a 0.08% allelic frequency of false-positive cases (Supplementary Figure 2Ai). In this case the absolute haploid GE count was inferior to the threshold of positivity (≥5 haploid GE), thus classifying it as negative case. Taken together, if any one of the indicators results in a value less than the pre-determined threshold which includes either (i) absolute haploid GE count <5 or (ii) mutated allelic frequency < 0.08%, the sample is deemed negative. We then validated these findings on patient cfDNA samples with known WT KRAS in FFPE routine samples (Supplementary Figure 2Aii). Although, we observed three well-defined mismatches, the maximum false-positive haploid GE determined by ddPCR was 2.8 haploid GE, corresponding to an allelic frequency of 0.3% for the 18 patients of the cohort (including mismatches).
Next, the sensitivity of the assay was investigated using Horizon's commercially available MT controls for a single KRAS p.G12D mutation set to 6.3%, 1.3% and 0.1% allelic frequencies. As displayed in Supplementary  Figure 2Bi, no meaningful differences were observed between the expected and measured allelic frequencies, although a slight background (around 0.05%) of falsepositive cases was obtained with the WT control. This false positive rate amongst WT controls has recently been determined to result from sonication induced oxidative DNA damage resulting in high rates of C > A and G > T transversions when genomic DNA is fragmented with focused ultrasonication (F. Holtrup, Sysmex Inostics R&D). These results were then confirmed in colon/lung cancer patients with a known KRAS mutational status in FFPE samples (Supplementary Figure 2Bii). Among the 23 FFPE biopsy-positive colon and lung cancer samples analyzed, 11 displayed KRAS mutations that exceeded the ≥5 MT haploid GE threshold in cfDNA.

internal performance and sensitivity of Beaming
The performance of BEAMing was assessed using an identical methodology to that applied to evaluate the performance of ddPCR. Thus, the false-positivity background ranged from 0 to 16 MT haploid GE (0.000-0.028% mutated allelic fraction), much less than the prespecified 50 haploid GE threshold determined by Sysmex Inostics (Supplementary Figure 3A). We then assessed the BEAMing assay in patients with known WT KRAS/ NRAS in FFPE samples. Among the 19 specimens that were KRAS/NRAS WT in biopsies, 14 cfDNA samples were below the threshold of 50 MT haploid GE and were deemed to have no mutation detected, while a mutation was detected in 5 specimens (+25% additional positive cases) (Supplementary Figure 3B).
Horizon's Multiplex I cfDNA standard set was then used to assess the sensitivity of the OncoBEAM-TM-RAS-CRC assay using two MT allelic frequencies of 1% and 0.1% for one mutation in KRAS on codons 12 (p.G12D); and two mutations in NRAS in codons 59 (p.A59K) and 61 (p.Q61K) (Figure 1, Supplementary  Table 1A). In Horizon's 1% controls, we found allelic frequencies very close to expected values (0.71-0.85%), and similar determinations were made with 0.1% controls (0.08-0.11%) (Supplementary Figure 3A). Furthermore, the absolute number of MT haploid GE detected exceeded the positivity threshold (including for 0.1% controls). Finally, we assessed KRAS/NRAS mutation detection with BEAMing in cfDNA samples from patients with KRAS/NRAS mutations determined in FFPE. We found 7/29 mismatches (Supplementary Figure 3C), including 4 alterations not covered with the OncoBEAM-TM-RAS-CRC assay.

internal performance and sensitivity of the nGS assay
We determined the specificity of the NGS assay using a commercial NGS Horizon Tru-Q5 DNA reference standard. We measured the background for three sequenced KRAS and NRAS exons on wild-type regions of the Horizon control DNA, and obtained a maximum of 0.06% false-positive rate (Supplementary Figure 4Aii). The secured threshold of positivity was defined at 0.5% allelic frequency, avoiding false-positives due to potential sequencing background error rates inherent to NGS technology, even if it is commonly used as a threshold at 2.5% to 5% for FFPE NGS samples. The quality indicator, herein named Q30, was at least superior to 85% of sequenced bases, indicating that the error of sequencing is around 1/1,000 in all of sequencing runs (22). Among the 19 lung and colon cancer patients with negative biopsies, 4 additional KRAS mutations were detected (Supplementary Figure 4Aii). Secondly, using the Tru-Q5 DNA gold reference standard, containing KRAS and NRAS somatic alterations at known allelic fractions, we found very close results between expected and measured values at 2.5% and 25% levels (Supplementary Figure 4Bi).