Circulating androgen and the risk of mortality in male patients with chronic heart failure: a metaanalysis of prospective cohort studies

INTRODUCTION

Despite significant improvements of the diagnosis and treatment during recent decades, chronic heart failure (CHF) remains one of the most important causes of mortality and morbidity for people in both the developed and developing countries [1–3]. Although many characteristics have been established as the potential predictors for the poor prognosis in CHF patients, such as the severity of cardiac dysfunction, anemia, renal dysfunction, and suboptimal use of evidence-based medications et al. [4], identification of novel prognostic factor remains important not only for the improvement of risk stratification in CHF, but also for the discovery of novel therapeutic targets [5].

Results of early studies suggested that androgen deficiency may be an important feature of male patients with CHF [6, 7]. Accumulating evidence indicated that male patients with CHF often had lower levels of circulating total testosterone (TT), free testosterone (FT), but higher level of sex hormone-binding globulin (SHBG) as compared with age-matched healthy volunteers [8, 9], suggesting gonadal dysfunction in these population. Moreover, circulating dehydroepiandrosterone sulfate (DHEAS), a component of androgen produced by adrenal glands, was also found to be lower in patients with CHF [8, 10]. Moreover, results of subsequent analyses suggest that reduced circulating TT, FT and DHEAS were correlated with the severity cardiac dysfunction in CHF patients [8, 11]. However, pilot results regarding the potential predictive values of androgen deficiency for the prognosis in CHF patients were inconsistent [8–13], and a quantitative summary of the findings by meta-analysis has not been performed. Moreover, although some randomized controlled trials (RCTs) have evaluated the role of testosterone replacement in male CHF patients [14, 15], to the best of our knowledge, none of these RCTs were statistically adequate to evaluate the efficacy of testosterone supplementation on clinical outcomes in these patients. Therefore, in this study, we performed a meta-analysis of prospective cohort studies to systematically evaluate the potential predictive efficacies of circulating TT, DHEAS, FT, and SHBG for the mortality in male CHF patients.

RESULTS

Database searching result

The process of literature searching was summarized in Figure 1. Briefly, 828 studies were identified by initial database searching after exclusion of the duplications. By screening via title and abstract, 808 studies were subsequently excluded mainly because they were irrelevant to the objective of the current study. The remaining 20 studies underwent full-text review, and 14 studies were further excluded because seven of them were RCTs, three were cross-sectional studies, two were not in CHF patients, one was repeated report of an already included cohort, and the other one did not report mortality outcome. Finally, six prospective cohort studies were included [8–13].

Figure 1: Flowchart of database searching and study selection.

Study characteristics and quality evaluation

The characteristics of the included studies were listed in Tables 1 and 2. Overall, our meta-analysis included six prospective cohort studies [8–13] with a total of 1342 male CHF patients. Five of them were performed in Europe [8–12], while the other one was performed in China [13]. The numbers of the included patients in each study varied from 104 to 501. The mean ages of the included patients ranged from 53.1 to 68.5 years. Both ischemic and non-ischemic CHF were included. Five of the studies only included patients with heart failure with reduced ejection fraction (HFrEF) [8–10, 12, 13], while the other one also included patients with heart failure with preserved ejection fraction (HFpEF) [11]. The mean left ventricular ejection fraction (LVEF) of the included patients at baseline varied from 28 to 42%, and the baseline N-terminal pro B-type natriuretic peptide (NT-proBNP) ranged between 993 and 1854 pg/ml. Patients included in each study had already received the evidence-based optimal medications for CHF, including angiotensin converting enzyme inhibitors or angiotensin II receptor inhibitors, β-blockers, and aldosterone receptor antagonists. The baseline circulating levels of TT, DHEAS, and SHBG were measured with immunoassays, while FT was calculated based on the equation proposed by Vermeulen [16]. With a mean follow-up between 2.2 to 3.5 years, 456 mortality cases occurred. Multiple confounding factors, including age, cardiac function, renal function, comorbidities, and concurrent medications, were adjusted when reporting the predictive efficacy of androgen deficiency for mortality in CHF. Four contacts were made by Email to the corresponding authors to obtain unavailable data, but only one of the authors provided the outcome data of interest [9]. The qualities of the included studies were generally good, with the Newcastle-Ottawa Scale (NOS) varying from 8 to 9 points.

Circulating TT and the mortality risk of in CHF patients

By pooling the results of six prospective cohort studies [8–13], we found that every increment of 1 ng/ml circulating TT was associated with 13% lowered mortality risk in male CHF patients (adjusted hazard ratio [HR] = 0.87, 95% confidence interval [CI] 0.81 to 0.94, p < 0.001; Figure 2A) with moderate heterogeneity (p for Cochrane’s Q test = 0.29, I² = 19%). Results of subgroup analyses showed that the association between TT and mortality risk were consistent in studies that reported cardiovascular (CV) mortality (adjusted HR = 0.83, 95% CI 0.74 to 0.93, p = 0.001) and those reported all-cause mortality (adjusted HR = 0.89, 95% CI 0.80 to 0.99, p = 0.03).

Figure 2: Forest plots for the meta-analyses of the predictive efficacies of circulating TT and DHEAS for the mortality risk in in male patients with CHF. (A) predictive efficacy of every increment of 1 ng/ml circulating TT for the mortality risk in CHF; (B) predictive efficacy of every increment of 100 ng/ml circulating DHEAS for the mortality risk in CHF.

Circulating DHEAS and the mortality risk of in CHF patients

By pooling the results of five studies [8–12], we found that every increment of 100 ng/ml DHEAS was associated with 7% lowered mortality risk in male CHF patients (adjusted HR = 0.93, 95% CI 0.88 to 0.98, p = 0.006; Figure 2B) with considerable heterogeneity (p for Cochrane’s Q test = 0.04, I² = 61%). Results of subgroup analyses showed that these results were consistent for studies that reported CV mortality (adjusted HR = 0.95, 95% CI 0.90 to 1.00, p = 0.03) and those reported all-cause mortality (adjusted HR = 0.86, 95% CI 0.74 to 1.00, p = 0.05).

Circulating FT and the mortality risk of in CHF patients

Meta-analysis of four cohorts [8, 9, 11, 13] showed that every increment of 10 pg/ml was associated with 7% lowered mortality risk in male CHF patients (adjusted HR = 0.93, 95% CI 0.89 to 0.97, p < 0.001; Figure 3A) with mild heterogeneity (p for Cochrane’s Q test = 0.88, I² = 0%). These results were consistent for studies that reported CV mortality (adjusted HR = 0.94, 95% CI 0.89 to 0.99, p = 0.02) and those reported all-cause mortality (adjusted HR = 0.92, 95% CI 0.86 to 0.97, p = 0.002).

Figure 3: Forest plots for the meta-analyses of the predictive efficacies of circulating FT and SHBG for the mortality risk in in male patients with CHF. (A) Predictive efficacy of every increment of 10 pg/ml circulating FT for the mortality risk in CHF; (B) predictive efficacy of every increment of 1 nmol/L circulating SHBG for the mortality risk in CHF.

Circulating SHBG and the mortality risk of in CHF patients

Meta-analysis of three studies [9, 11, 13], all reporting all-cause mortality, indicated that every increment of 1 nmol/L of SHBG confers 5% higher risk of mortality in male CHF patients (adjusted HR = 1.05, 95% CI 1.02 to 1.07, p < 0.001; Figure 3B) with mild heterogeneity (p for Cochrane’s Q test = 0.82, I² = 0%).

Publication bias

The funnel plots for the meta-analyses of the predictive efficacies of TT, DHEAS, FT and SHBG for mortality risk in CHF were shown in Figure 4A-D, which appears to be symmetry for the meta-analyses of TT, FT and SHBG, suggesting no significant publication biases. The funnel plot for the meta-analysis of the predictive efficacy of DHEAS seems to be asymmetry (Figure 4B). Therefore, a “trim-and-fill” analysis was performed to conservatively impute a hypothetical negative unpublished study in order to produce a symmetrical funnel plot. The pooled analysis by incorporating the hypothetical study also showed a statistically significant association between lower DHEAS and increased mortality risk in CHF (adjusted HR = 0.94, 95% CI 0.88 to 1.00, p = 0.002; Figure 5).

Figure 4: Funnel plots for the meta-analyses of the predictive efficacy of androgen deficiency for mortality risk in CHF. (A) funnel plot for the predictive efficacy of every increment of 1 ng/ml circulating TT for the mortality risk in CHF; (B) funnel plot for the predictive efficacy of every increment of 100 ng/ml circulating DHEAS for the mortality risk in CHF; (C) funnel plot for the predictive efficacy of every increment of 10 pg/ml circulating FT for the mortality risk in CHF; (D) funnel plot for the predictive efficacy of every increment of 1 nmol/L circulating SHBG for the mortality risk in CHF. For Figure 4B, the black square indicates the imputed study by “trim and fill” method to generate a symmetry funnel plot.

Figure 5: Forest plot for the meta-analysis of the predictive efficacy of every increment of 100 ng/ml circulating DHEAS for the mortality risk in CHF after including the hypothesized negative study imputed by “trim and fill” method. The imputed study was labeled in blue square.

DISCUSSION

In this study, by pooling the results of six prospective cohort studies, we found that circulating androgen deficiency is an independent predictor for the morality risk in male patients with stable CHF. Specifically, lower levels of circulating TT, DHEAS, and FT, but higher level of SHBG were associated with higher mortality risk in male CHF patients. Results of our study confirmed that androgen deficiency and higher SHBG as independent predictors for the mortality in male CHF patients. Moreover, these results highlight the need for an appropriately designed large-scale RCT to evaluate the potential influence of androgen supplementation on clinical outcomes in CHF patients.

Testosterone is the most important androgen in men. Moreover, it is also one of the anabolic hormones. Physiologically, most of testosterone is synthesized from gonad, and secreted into blood with majority of testosterone as a protein-bound form (about 98%, mainly with SHBG) [17]. Therefore, the activity of testosterone is dependent on the circulating level of FT, while the level of SHBG may serve as a negative regulator of circulating FT [17]. Moreover, androgen could also be secreted from adrenal gland, which forms DHEAS in peripheral circulation [17]. The above physiological interactions among the components of androgen may explain the results of our met-analysis which showed that lower circulating TT, DHEAS, and FT, but higher level of SHBG, were predictors of mortality in CHF. It could be estimated that the predictive effect of SHBG for mortality in CHF may depend on their influence on FT. Currently the exact mechanisms underlying the potential association between lower androgen and higher mortality in CHF remain to be determined. Previous studies suggest that androgen deficiency may lead to metabolic disorder [18] and enhanced inflammatory response [19], which may accelerate the deterioration of cardiac function. Moreover, results of the pilot RCTs indicated that supplementation of testosterone in CHF may improve the maximal oxygen consumption (VO_2max) [20] during cardiopulmonary exercise test, and restore the sensitivity of baroreflex [21], a marker of autonomic function [22]. In view of the fact that reduced VO_2max [23] and impaired autonomic function [24] have both been related to the poor prognosis in CHF, we hypothesized that improvement of exercise tolerance and correction of autonomic dysfunction may be the potential mechanisms underlying the beneficial effects of androgen to the prognosis in CHF patients. Moreover, androgen deficiency has been suggested to be correlated with the deterioration of the cardiac systolic and diastolic function [25, 26]. Therefore, androgen deficiency characterized of the lower circulating testosterone may be a maker of the severity of CHF rather than a direct cause of increased mortality. Obviously, further studies are needed.

Ours study has limitations which should be considered when interpreting the results. Firstly, since there is no consensus of androgen deficiency currently, androgen level was analyzed as continuous variable in all of the included studies rather than a categorized variable. Accordingly, results of our meta-analysis were also based on the effects of continuous changes of TT, FT, DHEAS, and SHBG on the mortality risk in CHF. Result of study highlights a need for a consensus definition of androgen deficiency and further investigation on its association with mortality risk in CHF. Secondly, only six studies were included. The limited number of the available studies and the lack of access to individual-patient data of the studies prevented us from further analyzing of the source of heterogeneity. Therefore, the potential influence of the study characteristics on the association between androgen deficiency and mortality risk deserves further investigation. Thirdly, only 6% of the included patients were with HFpEF. Therefore, whether the association between circulating levels of androgen and mortality risk remained in these patients should be evaluated in future studies. Fourthly, as our study is a meta-analysis of observational studies, results of our study did not indicated a causative relationship between androgen deficiency and mortality risk in CHF. Fifthly, the mean follow-up duration was about three years in the current meta-analysis. Whether androgen deficiency is a predictive factor for morality of CHF patients in long-term study should be confirmed. Sixthly, in our study, FT was not directly measured but calculated based on TT and SHBG, which may also introduce bias. Finally, although most-adequately adjusted HRs were used, we could not exclude the existence of residual confounding factors, which may affect the associations between circulating levels of androgen and mortality risk.

In conclusion, results of our meta-analysis showed that androgen deficiency and higher SHBG were independent predictors of the mortality for male patients with CHF. Whether androgen replacement could improve the prognosis of CHF patients with androgen deficiency should be investigated in large-scale RCTs.

MATERIALS AND METHODS

We followed the instructions of the Meta-analysis of Observational Studies in Epidemiology (MOOSE) [27] and the Cochrane’s Handbook [28] guidelines during the study design, implementation, data analysis, and results reporting processes of the meta-analysis.

Database searching

PubMed and Embase databases were searched for relevant studies using the combine of the following three groups of terms: (1) “testosterone”, “sex hormone-binding globulin”, “androgen”, or “dehydroepiandrosterone”; (2) “heart”, “cardiac”, or “ventricular”, couple with “failure”, “insufficiency”, “dysfunction” or “inadequacy”, or “cardiomyopathy”; and (3) “cohort”, “follow-up”, “followed”, “prospective”, “prospectively”, “risk” or “longitudinal”. We limited the searching to studies in humans and published in English. The reference lists of the related original and review articles were also manually screened for potential relevant studies. The final literature searching was performed on April 14, 2017.

Study selection

Studies were included if they fulfilled all of the following criteria: 1) published as full-length article in English; 2) designed as prospective cohort studies, without limitations of the sample size and follow-up duration; 3) including adult patients of stable CHF, regardless the etiologies of CHF; 4) circulating androgen levels, including TT, DHEAS, and FT, as well as the level of SHBG were measured at baseline; 5) documented outcomes of all-cause mortality or CV mortality during the follow-up; and 6) reported multivariable adjusted risks for mortality according to the baseline levels of androgen. Letters, editorials, none prospective cohort studies, baseline circulating androgens or SHBG not reported or measured, or those did not report outcomes of all-cause mortality or CV mortality were excluded. When repeated reports of the same cohort were found, the results with the longest follow-up durations were included in the meta-analysis.

Data extraction and quality evaluation

Two authors independently performed literature searching, data extraction, and quality assessment according to the predefined inclusion criteria. Discrepancies were solved by consensus and discussion with the third author. The extracted data included the details regarding study characteristics (name of the first author, year of publication, the location of the study, and total number of CHD patients included), patient characteristics (mean age, mean body mass index [BMI], proportions of patients with diabetes mellitus [DM], proportions of patients ischemic etiology, proportions of patients with each subtypes of HF according to systolic function, and baseline levels for the parameters of cardiac function, including LVEF and NT-proBNP), administration of concurrent medications for CHF. Follow-up information and the data related to study outcomes were also extracted, including measurements of TT, DHEAS, FT, and SHBG, mean follow-up duration, numbers of patients with interested outcomes, and potential confounding factors adjusted when presenting the results. The effect sizes reported including HR with their corresponding 95% CIs were extracted. If the data of interest was not reported in original articles, the corresponding authors were contacted for help. When data of various degrees of adjustment were reported, we only extracted the most adequately adjusted data. We evaluated the quality of the include studies using the NOS [29]. This scale judges the quality of each cohort study regarding three aspects: selection of the study groups; the comparability of the groups; and the ascertainment of the outcome of interest [29].

Statistical analyses

We used HRs as the measures for the associations between levels of TT, DHEAS, FT, and SHBG at baseline, and the subsequent risks of mortality in CHF patients. Data of HRs and their corresponding stand errors (SEs) were calculated from 95% CIs or p values, and were logarithmically transformed to stabilize variance and normalized the distribution [28]. The heterogeneity among the included studies was detected by the Cochrane’s Q test and the I² test. If I² > 50%, a significant heterogeneity was considered [30]. A random-effect model was applied to synthesize the results because it is a more generalized method which incorporates the heterogeneity of the included studies [28]. Predefined subgroup analyses were performed according to the different mortality outcome reported (all-cause or CV). Potential publication bias was assessed by visual inspection of the funnel plot [31]. The nonparametric “trim and fill” procedure was also performed to further assess the possible effect of publication bias in our meta-analysis. This method considers the possibility of hypothetical “missing” studies that might exist, imputes their HRs, and recalculates a pooled HR that incorporates the hypothetical missing studies as though they actually existed [28]. Since the HRs for mortality were reported according to a standard changes of the androgens and SHBG in each of the included studies, we summarized the HRs for morality according to standard increments of TT (per 1 ng/ml), DHEAS (per 100 ng/ml), FT (per 10 pg/ml), and SHBG (per 1 nmol/L). RevMan (Version 5.1; Cochrane Collaboration, Oxford, UK) and STATA software (Version 12.0; Stata Corporation, College Station, TX) were used for the meta-analysis and statistics.

Author contributions

Yunwei Li, Xiaoming Zhong and Guanchang Cheng designed the study. Yunwei Li and Xiaoming Zhong performed the literature retrieval and information extraction. All authors performed the statistical analysis, drafted the manuscript, and approved the final version of the manuscript.

ACKNOWLEDGMENTS

We thank Dr. Pascual-Figal for providing the unpublished data of their original study for this meta-analysis.

CONFLICTS OF INTEREST

None.

FUNDING

None.

REFERENCES

1. Benjamin EJ, Blaha MJ, Chiuve SE, Cushman M, Das SR, Deo R, de Ferranti SD, Floyd J, Fornage M, Gillespie C, Isasi CR, Jimenez MC, Jordan LC, et al. Heart Disease and Stroke Statistics-2017 Update: A Report From the American Heart Association. Circulation. 2017; 135:e146–e603.

2. Chinese Society of Cardiology of Chinese Medical Association; Editorial Board of Chinese Journal of Cardiology. [Chinese guidelines for the diagnosis and treatment of heart failure 2014]. [Article in Chinese]. Zhonghua Xin Xue Guan Bing Za Zhi. 2014; 42:98–122.

3. Ponikowski P, Voors AA, Anker SD, Bueno H, Cleland JG, Coats AJ, Falk V, Gonzalez-Juanatey JR, Harjola VP, Jankowska EA, Jessup M, Linde C, Nihoyannopoulos P, et al. 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC). Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur J Heart Fail. 2016; 18:891–975.

4. Ferrero P, Iacovoni A, D’Elia E, Vaduganathan M, Gavazzi A, Senni M. Prognostic scores in heart failure - Critical appraisal and practical use. Int J Cardiol. 2015; 188:1–9.

5. Nessler J, Rostoff P. Prognostic scores in advanced heart failure: where are we now and where are we going? Pol Arch Intern Med. 2017; 127:235–237.

6. Malkin CJ, Channer KS, Jones TH. Testosterone and heart failure. Curr Opin Endocrinol Diabetes Obes. 2010; 17:262–268.

7. Jankowska EA, Tkaczyszyn M, Kalicinska E, Banasiak W, Ponikowski P. Testosterone deficiency in men with heart failure: pathophysiology and its clinical, prognostic and therapeutic implications. Kardiol Pol. 2014; 72:403–409.

8. Jankowska EA, Biel B, Majda J, Szklarska A, Lopuszanska M, Medras M, Anker SD, Banasiak W, Poole-Wilson PA, Ponikowski P. Anabolic deficiency in men with chronic heart failure: prevalence and detrimental impact on survival. Circulation. 2006; 114:1829–1837.

9. Pascual-Figal DA, Tornel PL, Nicolas F, Sanchez-Mas J, Martinez MD, Gracia MR, Garrido IP, Ruiperez JA, Valdes M. Sex hormone-binding globulin: a new marker of disease severity and prognosis in men with chronic heart failure. Rev Esp Cardiol. 2009; 62:1381–1387.

10. Jankowska EA, Rozentryt P, Ponikowska B, Hartmann O, Kustrzycka-Kratochwil D, Reczuch K, Nowak J, Borodulin-Nadzieja L, Polonski L, Banasiak W, Poole-Wilson PA, Anker SD, Ponikowski P. Circulating estradiol and mortality in men with systolic chronic heart failure. JAMA. 2009; 301:1892–1901.

11. Guder G, Frantz S, Bauersachs J, Allolio B, Ertl G, Angermann CE, Stork S. Low circulating androgens and mortality risk in heart failure. Heart. 2010; 96:504–509.

12. Jankowska EA, Drohomirecka A, Ponikowska B, Witkowska A, Lopuszanska M, Szklarska A, Borodulin-Nadzieja L, Banasiak W, Poole-Wilson PA, Ponikowski P. Deficiencies in circulating testosterone and dehydroepiandrosterone sulphate, and depression in men with systolic chronic heart failure. Eur J Heart Fail. 2010; 12:966–973.

13. Wu HY, Wang XF, Wang JH, Li JY. Testosterone level and mortality in elderly men with systolic chronic heart failure. Asian J Androl. 2011; 13:759–763.

14. Toma M, McAlister FA, Coglianese EE, Vidi V, Vasaiwala S, Bakal JA, Armstrong PW, Ezekowitz JA. Testosterone supplementation in heart failure: a meta-analysis. Circ Heart Fail. 2012; 5:315–321.

15. Wang W, Jiang T, Li C, Chen J, Cao K, Qi LW, Li P, Zhu W, Zhu B, Chen Y. Will testosterone replacement therapy become a new treatment of chronic heart failure? A review based on 8 clinical trials. J Thorac Dis. 2016; 8:E269–277.

16. Vermeulen A, Verdonck L, Kaufman JM. A critical evaluation of simple methods for the estimation of free testosterone in serum. J Clin Endocrinol Metab. 1999; 84:3666–3672.

17. Matsumoto AM. Andropause: clinical implications of the decline in serum testosterone levels with aging in men. J Gerontol A Biol Sci Med Sci. 2002; 57:M76–99.

18. Rodriguez A, Muller DC, Metter EJ, Maggio M, Harman SM, Blackman MR, Andres R. Aging, androgens, and the metabolic syndrome in a longitudinal study of aging. J Clin Endocrinol Metab. 2007; 92:3568–3572.

19. Laaksonen DE, Niskanen L, Punnonen K, Nyyssonen K, Tuomainen TP, Salonen R, Rauramaa R, Salonen JT. Sex hormones, inflammation and the metabolic syndrome: a population-based study. Eur J Endocrinol. 2003; 149:601–608.

20. Stout M, Tew GA, Doll H, Zwierska I, Woodroofe N, Channer KS, Saxton JM. Testosterone therapy during exercise rehabilitation in male patients with chronic heart failure who have low testosterone status: a double-blind randomized controlled feasibility study. Am Heart J. 2012; 164:893–901.

21. Caminiti G, Volterrani M, Iellamo F, Marazzi G, Massaro R, Miceli M, Mammi C, Piepoli M, Fini M, Rosano GM. Effect of long-acting testosterone treatment on functional exercise capacity, skeletal muscle performance, insulin resistance, and baroreflex sensitivity in elderly patients with chronic heart failure a double-blind, placebo-controlled, randomized study. J Am Coll Cardiol. 2009; 54:919–927.

22. Rydlewska A, Maj J, Katkowski B, Biel B, Ponikowska B, Banasiak W, Ponikowski P, Jankowska EA. Circulating testosterone and estradiol, autonomic balance and baroreflex sensitivity in middle-aged and elderly men with heart failure. Aging Male. 2013; 16:58–66.

23. Corra U, Mezzani A, Giordano A, Pistono M, Gnemmi M, Caruso R, Giannuzzi P. Peak oxygen consumption and prognosis in heart failure: 14 mL/kg/min is not a “gender-neutral” reference. Int J Cardiol. 2013; 167:157–161.

24. Schwartz PJ, La Rovere MT, De Ferrari GM, Mann DL. Autonomic modulation for the management of patients with chronic heart failure. Circ Heart Fail. 2015; 8:619–628.

25. Kloner RA, Carson C 3rd, Dobs A, Kopecky S, Mohler ER 3rd. Testosterone and Cardiovascular Disease. J Am Coll Cardiol. 2016; 67:545–557.

26. Culic V. Testosterone and Cardiac Diastolic Function. J Am Coll Cardiol. 2016; 68:573–574.

27. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000; 283:2008–2012.

28. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from http://handbook.cochrane.org.

29. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2010; http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.

30. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002; 21:1539–1558.

31. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997; 315:629–634.