A quality analysis of clinical anaesthesia study protocols from the Chinese clinical trials registry according to the SPIRIT statement

Objective To learn about the overall quality of clinical anaesthesia study protocols from the Chinese Clinical Trials Registry and to discuss the way to improve study protocol quality. Methods We defined completeness of each sub-item in SPIRIT as N/A (not applicable) or with a score of 0, 1, or 2. For each protocol, we calculated the proportion of adequately reported items (score = 2 and N/A) and unreported items (score = 0). Protocol quality was determined according to the proportion of reported items, with values >50% indicating high quality. Protocol quality was determined according to the proportion of reported items. For each sub-item in SPIRIT, we calculated the adequately reported rate (percentage of all protocols with score 2 and NA on one sub-item) as well as the unreported rate (percentage of all protocols with score 0 on one sub-item). Results Total 126 study protocols were available for assessment. Among these, 88.1% were assessed as being of low quality. By comparison, the percentage of low-quality protocols was 88.9% after the publication of the SPIRIT statement. Among the 51 SPIRIT sub-items, 18 sub-items had an unreported rate above 90% while 16 had a higher adequately reported rate than an unreported rate. Conclusions The overall quality of clinical anaesthesia study protocols registered in the ChiCTR was poor. A mandatory protocol upload and self-check based on the SPIRIT statement during the trial registration process may improve protocol quality in the future.


INTRODUCTION
The overall quality of clinical trial reporting is considered to be inadequate [1], with limitations identified in the study design, conduct, and reporting stages [2,3]. These deficiencies may be avoided by increasing the completeness and transparency of study protocols [4][5][6]. Therefore, the Standard Protocol Items: Recommendations for Intervention Trials (SPIRIT) statement was developed in 2013 [7] as an evidence-based guidance for the publication of clinical study protocols. The statement checklist contains 33 items and 51 sub-items under 8 headings. In addition to providing information on how to complete a study protocol, the SPIRIT statement also provides recommendations on important aspects of study design, conduct, and reporting [8].
It is a requirement for clinical trials to be registered on the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) (http://www.who.int/ictrp) [9,10]. Of which the Chinese Clinical Trials Registry (ChiCTR) (www.chictr. org.cn) is a primary registry for clinical trials conducted in China as well as in other countries. Registered anaesthesia studies in ChiCTR have increased significantly in recent years. Despite researchers were encouraged to upload their study protocols during registered process in ChiCTR, there were no quality control of these protocols. Anaesthesia studies focus on anaesthesia technique as well as perioperative monitoring methods, which ensure patients' safe and comfort following a surgical or medical procedure. Quality assessment of randomized controlled trial was performed in endodontics [11], but there is no similar quality assessment in anaesthesiology.
Therefore, to learn about the overall quality of clinical anaesthesia study protocols and to discuss the way to improve study protocol quality, we identified clinical trials in clinical anaesthesia from the ChiCTR and analyzed the protocols based on the 51 sub-items of the SPIRIT checklist.

RESULTS
A total of 204 registered studies was retrieved from the ChiCTR, of which 126 study protocols were available for assessment. Among the 126 protocols there were 99 randomized controlled trials, 6 non-randomized controlled trials, and 21 observational studies ( Figure 1).

Overall quality assessment of protocols
Only 4 protocols (3.2%) were assessed as high quality and 111 protocols (88.1%) were assessed as low quality ( Figure 2; Table 1). We identified only 18 protocols registered from the date of the SPIRIT statement to April 2015. Among these, 1 protocol (5.6%) was assessed as

DISCUSSION
The completeness of protocol was one of the factors which determined clinical study quality. Protocol upload was required during registered process in ChiCTR, but not in other registry. So the anaesthesia study protocols in ChiCTR had a certain representation. Unfortunately, the overall quality of most anaesthesia study protocols submitted to the ChiCTR was poor.
To improve study quality, it has been a requirement of most clinical journals that clinical trials are registered in international databases, although current trial registry requirements appear to be insufficient [12]. Submission of a detailed protocol could help to prevent inadequate trial conduct and unnecessary protocol amendments [13]. However, this is not a mandatory requirement during the trial registration process and only 61.7% of anaesthesia study protocols in our review were available in the ChiCTR. This indicates a lack of awareness among some researchers regarding the importance of the study protocol. We suggest that a mandatory requirement to provide the protocol and perform a self-check of completeness according to the SPIRIT statement during the registration process may improve the quality of clinical studies.
The SPIRIT statement was issued at the end of 2013 [7,14]. However, we did not observe a significant improvement in the quality of protocols registered after this date (Table 1). Information related to the sub-items in the Administrative information, Introduction, Ethics and dissemination, and Appendices sections should be reviewed and confirmed during the study planning stage.  1 Adequately reported items: sub-items assessed as having a score of 2 or N/A. 2 Unreported items: sub-items assessed as having a score of 0. Only 4 protocols (3.2%) were located in the upper left quadrant and were therefore of high quality (>50% of sub-items assessed as adequately reported), and 111 protocols (88.1%) were located in the lower right quadrant and were therefore of low quality (>50% of subitems assessed as unreported), the remaining 11 protocols (8.7%) were located in the lower left quadrant and were of moderate quality. www.oncotarget.com Recommendations for Intervention Trials. * A score of 2 indicated that all important information was adequately reported in the protocol for that sub-item; a score of 1 indicated that some important information was inadequately reported; a score of 0 indicated that no information related to the sub-item was reported. N/A indicated that the item was not applicable to the specific protocol. Figure 3: Adequately reported rate 1 and unreported rate 2 for each SPIRIT # sub-item (n=126). 1 Adequately reported rate was the percentage of protocols with a score of 2 or NA for one sub-item among all protocols assessed. 2 Unreported rate was the percentage of protocols with a score of 0 for one sub-item among all protocols assessed. # SPIRIT: Standard Protocol Items: Recommendations for Intervention Trials. The adequately reported rate was lower than the unreported rate for 35 of the 51 sub-items. www.oncotarget.com Missing information within these sections may thus be the result of a lack of awareness among researchers about the SPIRIT statement. Therefore, to increase protocol completeness and quality, information on the SPIRIT statement and other relevant programs such as the Research to Publication initiative [15] should be widely disseminated.
Regarding the detailed content of the SPIRIT statement in our study, 16 sub-items were found to have an adequately reported rate greater than the unreported rate, with the remaining 35 sub-items requiring further information (Figure 3). Among these 35 sub-items, both the Participant timeline and Recruitment fields were frequently overlooked. Insufficient recruitment is a common feature of clinical trials, [16][17][18][19] and can prolong the study period, increase research costs, or reduce the statistical power. Increased focus on participant timelines and recruitment strategies during the study planning stage has been shown to improve recruitment and minimize loss to follow-up [8]. However, in the current study, only 27.0% of protocols adequately reported the participant timeline and only 11.9% adequately reported recruitment. In addition, data collection, management, and monitoring have been identified as important determinants of study quality and participant safety [20,21]. However, we found that these items were adequately reported in only 10% of protocols. Increased use of the SPIRIT guidelines will allow more researchers to recognize the benefits of establishing detailed protocols prior to study initiation. Meanwhile, adequate clinical research training on methodology, epidemiology, and statistics may improve the completeness of protocols and subsequently enhance the quality of clinical trials.
The limitation of our study is that only 61.8% of study protocol were available. Due to the low percentage, the result of our study may be biased. But even if we assumed that the rest unavailable protocols were high quality, the overall quality of anaesthesia protocols was still poor.

MATERIALS AND METHODS
We searched the ChiCTR for studies registered from 2005 to April 2015 using the term 'anaesthesia' in the title or 'anaesthesiology department' in the affiliated institution name. Study protocols were obtained from ChiCTR, and two of the authors (DY and SC) independently assessed the completeness of protocols based on the 51 sub-items of the SPIRIT checklist [7]. Any disagreement was resolved by discussion with the third author (LY).
We defined three levels of completeness for each sub-item, scored as 2, 1, and 0. A score of 2 was given where all important information was adequately reported in the protocol for the sub-item. A score of 1 meant that some of the important information was inadequately reported. A score of 0 indicated that none of the important information was reported. N/A (not applicable) was used where the sub-item was not applicable to the protocol.
Quality analysis was performed in two steps: 1. Overall quality assessment of protocols For each protocol, we calculated the proportion of adequately reported items (sub-items with a score of 2 or N/A) and unreported items (sub-items with a score of 0). Protocols were considered high quality if the proportion of adequately reported items was more than 50% or low quality if the proportion of unreported items was more than 50%; the remaining protocols were considered moderate quality.

Analysis of SPIRIT sub-items
We analyzed the reporting rate for each sub-item in the SPIRIT checklist by calculating the adequately reported rate (percentage of scores = 2 or NA for a sub-item across all protocols) as well as the unreported rate (percentage of scores = 0 for a sub-item across all protocols).

CONCLUSIONS
The quality of anaesthesia study protocols in the ChiCTR was assessed as poor in the current study. This issue might be addressed by introducing a mandatory requirement to provide the study protocol during the registration process along with performing a self-check of the protocol according to the SPIRIT guidelines. Awareness of the SPIRIT statement and adequate clinical research training should also be ensured within the clinical anaesthesia research community.

Author contributions
LY performed the search strategy and a quality analysis, and provided major editing of the manuscript; SC performed major quality analysis and provided assistant in editing of the manuscript; DY performed a quality analysis. TW and YZ conceived the study, retrieved the protocols and provided assistant in editing of the manuscript. JL performed the descriptive statistical analysis. LY and SC contributed equally to the article.