Research Papers:

A predictive nomogram improved diagnostic accuracy and interobserver agreement of perirectal lymph nodes metastases in rectal cancer

PDF |  HTML  |  How to cite

Oncotarget. 2016; 7:14755-14764. https://doi.org/10.18632/oncotarget.7548

Metrics: PDF 2223 views  |   HTML 2262 views  |   ?  

Yongfeng Liu, Renjie Wang, Ying Ding, Shanshan Tu, Yi Liu, Youcun Qian, Linghui Xu, Tong Tong, Sanjun Cai and Junjie Peng _


Yongfeng Liu1,*, Renjie Wang2,3,*, Ying Ding4, Shanshan Tu4, Yi Liu5, Youcun Qian1, Linghui Xu3,6, Tong Tong3,6, Sanjun Cai2,3, Junjie Peng2,3

1Institute of Health Sciences, Shanghai Jiao Tong University School of Medicine (SJTUSM) and Shanghai Institutes for Biological Sciences (SIBS), Chinese Academy of Sciences (CAS), Shanghai, China

2Department of Colorectal Surgery, Fudan University Shanghai Cancer Center, Shanghai, China

3Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China

4Department of Biostatistics, University of Pittsburgh, Pittsburgh, PA, USA

5Department of Statistics, Ohio State University, Columbus, OH, USA

6Department of Radiology, Fudan University Shanghai Cancer Center, Shanghai, China

*These authors contributed equally to this work

Correspondence to:

Junjie Peng, e-mail: [email protected]

Keywords: nomogram, rectal cancer, lymph node, metastases, MRI

Received: October 07, 2015     Accepted: January 29, 2016     Published: February 21, 2016


Objective: To develop a predictive nomogram to improve the diagnostic accuracy and interobserver agreement of pre-therapeutic lymph nodes metastases in patients with rectal cancer.

Materials and Methods: An institutional database of 411 patients with rectal cancer was used to develop a nomogram to predict perirectal lymph nodes metastases. Patients’ clinicopathological and MRI-assessed imaging variables were included in the multivariate logistic regression analysis. The model was externally validated and the performance was assessed by area under curve (AUC) of the receiver operator characteristics (ROC) curves. The interobserver agreement was measured between two independent radiologists.

Results: The diagnostic accuracy of the conventional MRI-assessed cN stage was 68%; 14.2% of the patients were over-staged and 17.8% of the patients were under-staged. A total of 35.1% of the patients had disagreed diagnosis for the cN stage between the two radiologists, with a kappa value of 0.295. A nomogram for predicting pathological lymph nodes metastases was successfully developed, with an AUC of 0.78 on the training data and 0.71 on the validation data. The predictors included in the nomogram were MRI cT stage, CRM involvement, preoperative CEA, tumor grade and lymph node size category. This nomogram yielded improved prediction in cN stage than the conventional MRI-based assessment.

Conclusions: By incorporating clinicopathological and MRI imaging features, we established a nomogram that improved the diagnostic accuracy and remarkably minimized the interobserver disagreement in predicting lymph nodes metastases in rectal cancers.


The goal for the treatment of rectal carcinoma is cure or local control of the disease with maintenance of an acceptable quality of life. As progresses achieved in the last two decades, total mesorectal excision and combined modality therapy, especially the preoperative chemoradiotherapy (CRT), has been the standard therapy for locally advanced rectal cancer. Compared to the postoperative CRT, preoperative CRT has demonstrated improvement in patient’s local control and quality of life. However, the radiation or CRT related toxicities are still not insignificant; a substantial incidence rate of impaired short-term or long-term quality of life, including fecal incontinence and erectile dysfunction, has been reported in patients with preoperative radiotherapy or CRT [13]. Improving the accuracy of preoperative staging system can potentially help identify patients who may be optimal candidates to undergo preoperative CRT.

Currently, magnetic resonance imaging (MRI) is one of the best preoperative staging techniques, and is especially preferred in locally advanced rectal cancer by its assessment of T category, N category and circumferential resection margin (CRM) [4, 5]. However, although the preoperative MRI was proven to predict both T category and circumferential resection margin with good accuracy, the accuracy for predicting lymph node metastasis was poor [6].

Previous studies also found clinicopathological features, such as T category, lymphovascular invasion or tumor grade, which may help identify patients at high risk of lymph node metastases [7, 8]. However, very few study established a reliable prediction model, using both imaging variables and clinicopathological variables, to predict pre-therapy lymph nodes metastases. Furthermore, although very small lymph nodes can be observed by high resolution MRI, the diagnosis of metastasis from detected lymph nodes is challenging, and the interobserver agreement in diagnosis of lymph nodes metastases varies among radiologists, especially for sub-centimeter lymph nodes [9, 10].

Therefore, the objective of the current study was to develop a predictive nomogram to improve the diagnostic accuracy and interobserver agreement of pre-therapy perirectal lymph nodes metastases in patients with rectal cancer, by incorporating pre-therapeutic clinicopathological and MRI imaging variables.


Clinicopathological and conventional MRI characteristics

The clinicopathological characteristics of all the 411 patients were listed in Table 1, which were comparable between the training group and the validation group. The pathological stage I, II and III (AJCC staging, sixth edition [11]) patients were 28.5%, 28.7% and 42.8%, respectively. The median number of sampled lymph nodes was 13 (range, 4–58) in the training group and 14 (range, 6–49) in the validation group. Of the 411 patients, lymph nodes were not found by MRI assessment in 130 patients (31.6%), of which 26 (20%) out of 130 patients proved node positive by histopathology.

Table 1: The clinicopathological characteristics of 411 patients with rectal cancer

Abbreviation: AV, anal verge; CEA, carcinoembryonic antigen; * The tumor location was measured endoscopically.

Of 288 patients in the training group, the detailed MRI imaging characteristics were studied and presented in Table 2. There were a total of 808 lymph nodes detected by MRI, in which 478 nodes (59.2%) were size < 5 mm, 310 nodes were size 5 mm–10 mm (38.4%), and 20 nodes (2.4%) were size > 10 mm. The mean number of detected lymph nodes was 2.8, ranging from 0–27.

Table 2: Univariate analysis of relationship between clinicopathological/imaging variables and pathological N stage

Abbreviation: AV, anal verge; CEA, carcinoembryonic antigen; CRM, circumferential resection margin; LN, lymph node.

The associations between MRI-assessed cN stage and pathological N stage (pN stage) were assessed in the training group. The diagnostic accuracy of MRI-assessed cN stage was 68%; the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were 58.2%, 75.3%, 63.4% and 71%, respectively. 14.2% of patients were overstaged and 17.8% of patients were understaged. While for the T stage, the diagnostic accuracy of MRI-assessed cT stage was 81.3% with12.2% of patients being overstaged and 6.5% of patients being understaged.

Interobserver agreement of conventional MRI

Of 288 patients in the training group, the overall diagnostic accuracy of cN stage was comparable between the two radiologists (65.6% for Doc 1 and 68.1% for Doc 2). However, for a case-to-case comparison, 35.1% of patients (101/288 cases) were disagreed between the two radiologists, with a kappa value of 0.295. Further investigation found that patients with undetected lymph nodes or with lymph nodes ≥ 10 mm yielded complete agreement of diagnoses between Doc1 and Doc2, while patients with disagreed diagnoses were those with small sub-centimeter lymph nodes. Of these 101 patients with disagreed diagnoses, 57.4% of cases had lymph nodes sized < 5 mm and 42.6% of cases had lymph nodes sized 5–10 mm (Figure 1).

The comparison of diagnostic agreement of cN stage in MRI-assessed lymph nodes, categorized by their sizes.

Figure 1: The comparison of diagnostic agreement of cN stage in MRI-assessed lymph nodes, categorized by their sizes. (Black bars and grey bars represent agreed cases and disagreed cases by two radiologists, respectively.)

Nomogram development and performance

The univariate analysis result of the clinicopathological and MRI imaging variables was presented in Table 2. For the ease of clinical application, the lymph node size category in our nomogram was classified into four groups using the largest size of detected lymph nodes within each patient: no LN detected, largest LN size < 5 mm, largest LN size 5–10 mm, largest LN size > 10 mm. The variables with a p-value < 0.1 were included in the multivariate analysis. The final nomogram was provided in Figure 2, with an AUC of 0.78 for the training data and 0.71 for the validation data. The predictors included in the nomogram were MRI cT stage, MRI CRM involvement, preoperative CEA, tumor grade and lymph node size category. From Figure 2, we can get a predicted probability of N stage positive for a given subject in the following way. First, use the top “points” line to get a point for each predictor (according to its value). Next, sum up the points from all predictors to get the total point. Last, map the total point to the probability line (i.e., “Probability of N+”) to get the predicted probability. For example, if the total point is 100, then the predicted probability of N positive is 0.75. Table 3 presents the odds ratio (OR) with 95% CI and the p value for each predictor.

Rectal cancer nomogram for predicting lymph node metastases.

Figure 2: Rectal cancer nomogram for predicting lymph node metastases. Each variable value is assigned a score, and the sum of scores is converted to a probability of pathological lymph node metastasis in the lowest scale.

Table 3: Multivariate analyses of predicting pathological lymph nodes metastases: the final predictor for developing the nomogram

Abbreviations: MRI, magnetic resonance image; CRM, circumferential resection margin; OR, odds ratio; CI, confidence interval; AUC, area under ROC curve; CEA, carcinoembryonic antigen; LN, lymph node.

To compare the prediction performance, the ROC curves based on all 411 patients were plotted for both the nomogram and conventional MRI-assessed cN stage (Figure 3). The AUCs of the nomogram and conventional MRI-assessed cN stage were 0.761 (95% CI, 0.715–0.807) and 0.621 (95% CI, 0.574–0.668), respectively. Therefore, our nomogram was superior to the conventional MRI-assessed cN stage in predicting pathological lymph node metastases.

The ROC curves of the developed nomogarm and conventional MRI-assessed cN stage based on all patients.

Figure 3: The ROC curves of the developed nomogarm and conventional MRI-assessed cN stage based on all patients.


In the current study with a large number of patients, we successfully developed a nomogram for predicting lymph node metastases in rectal cancers, by including both clinicopathological risk factors and magnetic resonance imaging features. Our nomogram showed improved diagnostic accuracy and could eliminate interobserver variability in the assessment of lymph nodes. Although lymph node metastasis is not the only criterion for identifying patients to undergo preoperative treatment, improved preoperative staging will reduce overtreatment or undertreatment, and ultimately improve patient’s outcome and quality of life. Meanwhile, by informing patients the likelihood of lymph node metastases, patients can potentially become more actively involved in the decision-making process with regard to preoperative treatment with their doctors. It will also provide additional support to improve patient’s treatment compliance.

Lymph node metastasis is a key indication for preoperative CRT and is an important prognostic factor for local or distant recurrence, which has been greatly emphasized in various treatment guidelines for rectal cancer [12, 13]. Consequently, the precision of treatment in patients with rectal cancer will be compromised by the large amount of patients with overstaged or understaged disease. In a specific series, 22% of patients still had lymph node metastases even after preoperative CRT, which were all diagnosed cT3 and node negative before treatment [14]. Although the authors stated that potential overtreatment was warranted in cT3N0 patients due to unsatisfactory preoperative node staging, a considerable number of patients may benefit from improved preoperative staging methods and obviate from the radiotherapy. Several meta-analyses have investigated the diagnostic accuracy of MRI for local stage of rectal cancer, including T category, lymph node metastases and CRM [46, 15]. The performance of MRI in diagnosing lymph node metastases was consistently poor, with varied sensitivity (60-84%) and specificity (59–81%) [6, 16]. In our series, using the conventional clinical staging, the overall diagnostic accuracy, sensitivity and specificity were similar to these previous studies, and both the percentages of overstaged cases and understaged cases were as high as around 15%.

One limitation in detecting positive lymph nodes by MRI is that a great number of detected nodes turned out to be normal or benign reactive nodes. In a node-to-node study of lymph nodes in rectal cancer, only 76 out of 521 nodes (14.6%) were confirmed malignant nodes by histology [17]. Currently, identifying malignant nodes mainly relies on the size of nodes. Studies confirmed that the proportion of positive nodes increased with larger nodal size [1820]. However, there was no widely accepted consensus on the size criterion of enlarged lymph nodes, and substantial overlap exists between benign and malignant nodes. Brown et al. studied 476 lymph nodes in 42 patients with rectal cancers. In their study, although a cutoff of 5 mm gave optimal sensitivity and specificity for nodal status, 58% of positive nodes were less than 5 mm in diameter, and the overall predictive value by size was poor [20]. Similarly, 41.4% of patients in our series, who only had lymph nodes < 5 mm, were found node positive by histology. Other imaging parameters in MRI, such as border, shape or intrinsic signal of lymph nodes, were concurrently analyzed in various studies. However, the results were inconsistent. Kim et al. found that besides nodal size, lymph nodes with spiculated or indistinct border and mottled heterogeneous appearance were useful to predict positive nodes [19]. Brown et al. also found higher accuracy was achieved when lymph nodes with irregular border and mixed intrinsic signal were considered malignant [20]. However, in another study, even the criteria of positive lymph nodes were strict (heterogeneous texture, irregular margin, and nodal size > 10 mm), the total accuracy was only 63%, with a sensitivity of 85% and specificity of 41% [21]. Most of these studies only had a small number of patients and multivariate analyses were rarely used to assess the relationship between these parameters and the nodal size. Besides, in our point of view, determining the border, shape or intrinsic signal is difficult and inaccurate for very small nodes (less than 3–5 mm) in MRI. Some low-signal-intensity rim in small lymph nodes is difficult to distinguish between normal lymph node capsule and tumor with necrosis, and tumor without necrosis in lymph node will present similar intensity with normal lymph nodes. In our study, the border and intrinsic signal of nodes were also found significantly associated with node metastases, but both of them were excluded in the multivariate analysis due to the reason that 59.2% of detected nodes were less than 5 mm in our series. Although not optimal, the size of lymph node is currently the most reliable parameter for diagnosing lymph node metastasis in rectal cancer.

Clinical or histopathological features have been proved to be related to lymph node metastases in patients with colorectal cancer [2225]. In early rectal cancer, some features, such as lymphovascular invasion, tumor size and T category, were used as risk factors to select candidates undergoing local excision without lymphadenectomy. Nevertheless, only a small number of studies focused on rectal cancer, and the examined clinicopathological features manifested varied significance in different studies [26]. Currently, no single clinicopathological feature of rectal cancer could optimally predict lymph node metastases. Using clinicopathological risk factors, a multivariate logistic model was established by Ogawa et al. which improved diagnostic accuracy of lymph node metastases by MRI [27]. However, the complicated criteria of MRI nodal interpretation and lack of external validation in that study restricted its practical use in clinical situations. In our study, with a large number of cases, a reliable and simple nomogram was successfully developed with external validation, based on variables from MRI and preoperative clinicopathological features. Compared with the standard MRI diagnosis, a significant improvement in predicting lymph node metastases was achieved by our nomogram.

The interobserver agreement was regularly examined in many studies of radiological lymph node assessment, and the results varied with different MRI techniques and study designs. The interobserver agreement was relatively higher in studies with node-to-node matched analysis than that with patient-to-patient analysis [2830]. However, bias existed in both types of studies. Node-to-node matched studies could not assess cases with invisible nodes in MRI, in which a considerable number of patients still had lymph node metastases; meanwhile, patient-to-patient studies could not clearly ascertain that the suspicious nodes in MRI can be sampled and assessed by pathologists. These reasons may attribute to the variability of interobserver agreement. In our study, even with similar diagnostic accuracy, the interobserver agreement between the two radiologists was only fair for lymph node assessment, as compared to the excellent interobserver agreement for T category assessment. In addition, all the disagreed cases were located in patients with sub-centimeter lymph nodes, and patients only with lymph nodes less than 5 mm had greatest disagreement in the diagnosis of node metastasis. Our nomogram consisted of preoperative collectable clinicopathological variables and simple MRI node categories, where different scores, representing different weights in predicting lymph node metastases, were assigned to these predictors. The included radiological variables, T category, CRM involvement and size-based lymph node categories, were all of good to excellent interobserver agreement. Such a nomogram provided clinicians a consistent and reliable tool to further improve nodal staging in patients with rectal cancer.

There are limitations to the current study. The study design was retrospective and the fact of single institutional study may limit its applicability in other patients. Thus, additional validations in patients from multi-institutions are needed in the future. Moreover, the treatment in patients with cT3N0 rectal cancers was the area of greatest controversy. Because of the small number of cT3N0 cases in this series, we are not able to perform a reliable subgroup analysis in these patients. These will be conducted in our future investigation.


By incorporating preoperative clinicpathological variables and magnetic resonance imaging features, we established a nomogram that improved the diagnostic accuracy and remarkably minimized the interobserver disagreement in diagnosing lymph nodes metastases in rectal cancers.



All patients with rectal cancer were collected from a single institution, Fudan University Shanghai Cancer Center, with a high volume of rectal cancer patients each year. To determine the accurate status of lymph nodes metastases, all patients selected in our series underwent preoperative high resolution MRI scanning and then followed by immediate surgeries for resection the primary tumor under the principle of total mesorectal excision in our institution. Patients were excluded for following reasons: lack of preoperative high-resolution MRI staging, undergoing preoperative treatment (chemotherapy, CRT or radiotherapy), distant metastases and unresectable primary tumors.

Between January 2005 and December 2014, a total of 411 patients with rectal cancer were included in the current study. All patients were rectal adenocarcinoma located within 12 cm from anal verge. The main reasons for the “surgery first” strategy included MRI-staged early disease, patient’s refusal of preoperative CRT, old age, doctor’s preference, financial considerations and local insurance issues. All patients in our series underwent resection of the primary tumors with the principle of total mesorectal excision. All visible resected specimens were embedded in paraffin for 24 hours and examined histologically with hematoxylin-eosin (HE) staining. The extent of local tumor spread in each slice was assessed and the total number of histological detected lymph node was recorded. According to the histological report in our hospital database, the tumor staging was restaged using the 6th edition of TNM (tumor, node and metastasis) system [11]. The lymph node status was graded as: N0, no lymph node metastasis; N1, one to three lymph node metastasis; N2, four or more lymph node metastasis.

This study was approved by the Fudan University Shanghai Cancer Center Institutional Ethics Committee.

MRI imaging

High-resolution scan parameters in preoperative MRI were used for each patient. The T2-weighted fast spin echo sequence with a thin 2-mm section was mainly used for preoperative assessment. Images were made in the sagittal, coronal and axial plane. The cranial border of the field of view was L5, the caudal border was below the anal canal. No intrarectal coli or contrast was used. Intravenous Gadolinium-enhanced scanning was used for most of the patients, but the enhancement of lymph nodes was not routinely assessed for each patient in this study.

The variables assessed by MRI included primary tumor and local or regional lymph nodes. In the training group, the assessment of primary tumor included cT category, CRM involvement, tumor spectrum within the bowel circle, tumor location related to peritoneal reflex; the assessment of lymph nodes included the number of detected nodes, sizes, irregularity of nodes’ border and uniformity of signal intensity within the nodes. We recorded the total number of lymph nodes and the number of lymph nodes by different node sizes (< 5 mm, 5 mm–10 mm, > 10 mm). In the validation group, the variables used in the predictive nomogram (developed from the training data) were assessed.

All MRI images were assessed by two independent observers: Doc1 (TT) and Doc2 (LHX) with 5 and 15 years of experience in MRI imaging, respectively. Each observer was asked to make a diagnosis of N category (N+/N−) for each patient, and a consensus N category for each disagreed case was determined after discussion. We refer to the studies by Brown et al. and Kim et al. [19, 20] for the criteria for lymph node metastasis. The interobserver agreement was measured between the two observers according to their original diagnosis of N category.


The clinicopathological characteristics between the training group and the validation group were analyzed by chi-square test (age was dichotomized). The sensitivity, specificity, PPV and NPV of conventional MRI-assessed cN stage were calculated. Weighted kappa values were calculated to evaluate the interobserver agreement in cT stage and cN stage categorization, respectively [31].

Statistical model development

To develop a nomogram with independent data to validate, of all the 411 patients, 288 patients, treated between January 2005 and December 2012, were assigned to the training group; and the other 123 patients, treated between January 2013 and December 2014, were assigned to the validation group.

In the training data, univariate logistic regression was performed for each potential predictive variable to assess its association with the pathological lymph node metastasis (N+/N−). The variables with univariate p-value < 0.1 were considered as candidate predictors in the multivariate logistic model. The final nomogram derived from the training data was selected by the stepwise multivariate logistic regression with post-hoc AUC examination.

Model validation

The predictive nomogram for N category was evaluated in both the training group and the validation group. The predicted probability for pathological lymph node metastasis (pN+) from the nomogram was compared with the actual lymph node metastasis status (determined post-surgery by pathologists) for each patient. The ROC curves were plotted with AUCs calculated.

All the statistical analyses were performed using R 3.1. A p < 0.05 was considered statistically significant.


We greatly appreciate our research nurses, Zhuzhu Qian and Jin Gao, at Department of Colorectal Surgery of Fudan University Shanghai Cancer Center for their excellent assistance.


This work was supported by the grants from Science and Technology Commission of Shanghai Municipality No. 134119a8602.


No conflicts of interest was declared.


1. Canda AE, Terzi C, Gorken IB, Oztop I, Sokmen S, Fuzun M. Effects of preoperative chemoradiotherapy on anal sphincter functions and quality of life in rectal cancer patients. Int J Colorectal Dis. 2010; 25:197–204.

2. Braendengen M, Tveit KM, Bruheim K, Cvancarova M, Berglund A, Glimelius B. Late patient-reported toxicity after preoperative radiotherapy or chemoradiotherapy in nonresectable rectal cancer: results from a randomized Phase III study. Int J Radiat Oncol Biol Phys. 2011; 81:1017–1024.

3. Horisberger K, Rothenhoefer S, Kripp M, Hofheinz RD, Post S, Kienle P. Impaired continence function five years after intensified chemoradiation in patients with locally advanced rectal cancer. Eur J Surg Oncol. 2014; 40:227–233.

4. Kwok H, Bissett IP, Hill GL. Preoperative staging of rectal cancer. Int J Colorectal Dis. 2000; 15:9–20.

5. Bipat S, Glas AS, Slors FJ, Zwinderman AH, Bossuyt PM, Stoker J. Rectal cancer: local staging and assessment of lymph node involvement with endoluminal US, CT, and MR imaging—a meta-analysis. Radiology. 2004; 232:773–783.

6. Al-Sukhni E, Milot L, Fruitman M, Beyene J, Victor JC, Schmocker S, Brown G, McLeod R, Kennedy E. Diagnostic accuracy of MRI for assessment of T category, lymph node metastases, and circumferential resection margin involvement in patients with rectal cancer: a systematic review and meta-analysis. Ann Surg Oncol. 2012; 19:2212–2223.

7. Nissan A, Stojadinovic A, Shia J, Hoos A, Guillem JG, Klimstra D, Cohen AM, Minsky BD, Paty PB, Wong WD. Predictors of recurrence in patients with T2 and early T3, N0 adenocarcinoma of the rectum treated by surgery alone. J Clin Oncol. 2006; 24:4078–4084.

8. Green JB, Timmcke AE, Mitchell WT, Hicks TC, Gathright JB, Jr., Ray JE. Mucinous carcinoma—just another colon cancer? Dis Colon Rectum. 1993; 36:49–54.

9. Urban M, Rosen HR, Holbling N, Feil W, Hochwarther G, Hruby W, Schiessel R. MR imaging for the preoperative planning of sphincter-saving surgery for tumors of the lower third of the rectum: use of intravenous and endorectal contrast materials. Radiology. 2000; 214:503–508.

10. Mizukami Y, Ueda S, Mizumoto A, Sasada T, Okumura R, Kohno S, Takabayashi A. Diffusion-weighted magnetic resonance imaging for detecting lymph node metastasis of rectal cancer. World J Surg. 2011; 35:895–899.

11. Greene FL PD, Fleming ID, et al. (eds). AJCC cancer staging manual, 6th edn. Springer, New York. 2002.

12. Augestad KM, Lindsetmo RO, Stulberg J, Reynolds H, Senagore A, Champagne B, Heriot AG, Leblanc F, Delaney CP. International preoperative rectal cancer management: staging, neoadjuvant treatment, and impact of multidisciplinary teams. World J Surg. 2010; 34:2689–2700.

13. Benson AB, 3rd, Venook AP, Bekaii-Saab T, Chan E, Chen YJ, Cooper HS, Engstrom PF, Enzinger PC, Fenton MJ, Fuchs CS, Grem JL, Grothey A, Hochster HS, et al. Rectal Cancer, Version 2.2015. J Natl Compr Canc Netw. 2015; 13:719–728.

14. Guillem JG, Diaz-Gonzalez JA, Minsky BD, Valentini V, Jeong SY, Rodriguez-Bigas MA, Coco C, Leon R, Hernandez-Lizoain JL, Aristu JJ, Riedel ER, Nitti D, Wong WD, et al. cT3N0 rectal cancer: potential overtreatment with preoperative chemoradiotherapy is warranted. J Clin Oncol. 2008; 26:368–373.

15. Lahaye MJ, Engelen SM, Nelemans PJ, Beets GL, van de Velde CJ, van Engelshoven JM, Beets-Tan RG. Imaging for predicting the risk factors—the circumferential resection margin and nodal disease—of local recurrence in rectal cancer: a meta-analysis. Semin Ultrasound CT MR. 2005; 26:259–268.

16. Branagan G, Chave H, Fuller C, McGee S, Finnis D. Can magnetic resonance imaging predict circumferential margins and TNM stage in rectal cancer? Dis Colon Rectum. 2004; 47:1317–1322.

17. Lambregts DM, Beets GL, Maas M, Kessels AG, Bakers FC, Cappendijk VC, Engelen SM, Lahaye MJ, de Bruine AP, Lammering G, Leiner T, Verwoerd JL, Wildberger JE, et al. Accuracy of gadofosveset-enhanced MRI for nodal staging and restaging in rectal cancer. Ann Surg. 2011; 253:539–545.

18. Akasu T, Iinuma G, Takawa M, Yamamoto S, Muramatsu Y, Moriyama N. Accuracy of high-resolution magnetic resonance imaging in preoperative staging of rectal cancer. Ann Surg Oncol. 2009; 16:2787–2794.

19. Kim JH, Beets GL, Kim MJ, Kessels AG, Beets-Tan RG. High-resolution MR imaging for nodal staging in rectal cancer: are there any criteria in addition to the size? Eur J Radiol. 2004; 52:78–83.

20. Brown G, Richards CJ, Bourne MW, Newcombe RG, Radcliffe AG, Dallimore NS, Williams GT. Morphologic predictors of lymph node status in rectal cancer with use of high-spatial-resolution MR imaging with histopathologic comparison. Radiology. 2003; 227:371–377.

21. Kim NK, Kim MJ, Park JK, Park SI, Min JS. Preoperative staging of rectal cancer with MRI: accuracy and clinical usefulness. Ann Surg Oncol. 2000; 7:732–737.

22. Rasheed S, Bowley DM, Aziz O, Tekkis PP, Sadat AE, Guenther T, Boello ML, McDonald PJ, Talbot IC, Northover JM. Can depth of tumour invasion predict lymph node positivity in patients undergoing resection for early rectal cancer? A comparative study between T1 and T2 cancers. Colorectal Dis. 2008; 10:231–238.

23. Cai G, Xu Y, Zhu X, Peng J, Li Z, Xiao C, Du X, Zhu J, Lian P, Sheng W, Guan Z, Cai S. A simple scoring system based on clinical features to predict locally advanced rectal cancers. J Gastrointest Surg. 2009; 13:1299–1305.

24. Bayar S, Saxena R, Emir B, Salem RR. Venous invasion may predict lymph node metastasis in early rectal cancer. Eur J Surg Oncol. 2002; 28:413–417.

25. Tateishi Y, Nakanishi Y, Taniguchi H, Shimoda T, Umemura S. Pathological prognostic factors predicting lymph node metastasis in submucosal invasive (T1) colorectal carcinoma. Mod Pathol. 2010; 23:1068–1072.

26. Glasgow SC, Bleier JI, Burgart LJ, Finne CO, Lowry AC. Meta-analysis of histopathological features of primary colorectal cancers that predict lymph node metastases. J Gastrointest Surg. 2012; 16:1019–1028.

27. Ogawa S, Itabashi M, Hirosawa T, Hashimoto T, Bamba Y, Kameoka S. A logistic model including risk factors for lymph node metastasis can improve the accuracy of magnetic resonance imaging diagnosis of rectal cancer. Asian Pac J Cancer Prev. 2015; 16:707–712.

28. Park JS, Jang YJ, Choi GS, Park SY, Kim HJ, Kang H, Cho SH. Accuracy of preoperative MRI in predicting pathology stage in rectal cancers: node-for-node matched histopathology validation of MRI features. Dis Colon Rectum. 2014; 57:32–38.

29. Kadioglu A, Terzi R, Saruhan N, Saglam A. Current advances in the investigation of leaf rolling caused by biotic and abiotic stress factors. Plant Sci. 2012; 182:42–48.

30. Heijnen LA, Lambregts DM, Mondal D, Martens MH, Riedl RG, Beets GL, Beets-Tan RG. Diffusion-weighted MR imaging in primary rectal cancer staging demonstrates but does not characterise lymph nodes. Eur Radiol. 2013; 23:3354–3360.

31. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968; 70:213–220.

Creative Commons License All site content, except where otherwise noted, is licensed under a Creative Commons Attribution 4.0 License.
PII: 7548