|Year : 2016 | Volume
| Issue : 2 | Page : 116-120
|Quality of reporting of randomized controlled trials in ten academic Indian dental journals
Sudhakar Vundavalli1, Guntipalli M Naidu1, A.S.K. Bhargav2, BH Praveen2, B Pavani2, Suresh Babburi3
1 Department of Public Health Dentistry, Dr. SNR Siddhartha Institute of Dental Sciences, Gannavaram, Andhra Pradesh, India
2 Department of Public Health Dentistry, Sri Sai College of Dental Surgery, Hyderabad, Telangana, India
3 Department of Oral Pathology, Dr. SNR Siddhartha Institute of Dental Sciences, Gannavaram, Andhra Pradesh, India
Click here for correspondence address and email
|Date of Submission||10-Oct-2013|
|Date of Decision||30-Oct-2013|
|Date of Acceptance||25-Mar-2016|
|Date of Web Publication||30-May-2016|
| Abstract|| |
Background: Biased results from poorly reported trials can mislead decision-making in health care at all levels, from treatment decisions for the individual patient to formulation of national public health policies.
Objective: To evaluate the quality of reporting of randomized controlled trials (RCTs) in ten Indian dental journals over the period 2011–2012.
Materials and Methods: This study included all RCTs published as full-text articles reported in ten Indian dental journals over the period from 2011 to 2012. The relevant trials were identified by searching Medline. Hand searching of the journals was also carried out by three of the authors to check if any potential trial was missing. Each article was assessed against the Consolidated Standards of Reporting Trials criteria tool, as described by Schulz et al. (2010).
Results: The mean number of criteria present per article was 12.2 (standard deviation [SD] =2.2) and only 5 of 106 articles got total possible score. Most of the articles (69%) did not mention about justification for sample size calculation, 89% of the articles did not mention about allocation concealment, 86% of the articles did not mention about funding and 63% of the articles did not mention about limitations of the study.
Conclusion: The quality of reporting of Randomized clinical trials in ten Indian academic journals was poor.
Keywords: Clinical trials, Consolidated Standards of Reporting Trials statement, Indian dental journals, quality of reporting, randomized controlled trials
|How to cite this article:|
Vundavalli S, Naidu GM, Bhargav A, Praveen B H, Pavani B, Babburi S. Quality of reporting of randomized controlled trials in ten academic Indian dental journals. Indian J Dent Res 2016;27:116-20
|How to cite this URL:|
Vundavalli S, Naidu GM, Bhargav A, Praveen B H, Pavani B, Babburi S. Quality of reporting of randomized controlled trials in ten academic Indian dental journals. Indian J Dent Res [serial online] 2016 [cited 2020 Jul 15];27:116-20. Available from: http://www.ijdr.in/text.asp?2016/27/2/116/183132
Research as a part of curriculum is not new to dental and medical education in India but efforts to incorporate clinical research into academics has been accelerated from mid-1990's in India. Randomized controlled trials (RCTs) were considered gold standard in evaluating health care intervention.
However, randomized trials can yield biased results if they lack methodological rigor. Only high-quality research, in which proper attention has been given to design, will consistently eliminate bias. The design and implementation of an RCT require methodological as well as clinical expertise; meticulous effort; and a high index of suspicion for unanticipated difficulties, potentially unnoticed problems, and methodological deficiencies. Reports of RCTs should be written with similarly close attention to minimizing bias. To assess a trial accurately, readers of a published report need complete, clear, and transparent information on its methodology and findings.
Recent methodological analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects. Such systematic error is seriously damaging to RCTs, which boast the elimination of systematic error as their primary hallmark. Systematic error in RCTs reflects poor science, and poor science threatens proper ethical standards. For example, information on the method used in a trial to assign participants to comparison groups was reported in only 21% of 519 trial reports indexed in PubMed in 2000, and only 34% of 616 reports indexed in 2006. Similarly, only 45% of trial reports indexed in PubMed in 2000 and 53% in 2006 defined a primary end point and only 27% in 2000 and 45% in 2006 reported a sample size calculation.
Reporting is not only often incomplete but also sometimes inaccurate. To rectify this problem of improper reporting of RCT, researchers have prepared a Consolidated Standards of Reporting Trials (CONSORTs) statement for the first time in 1996 and the objective of CONSORT is to facilitate critical appraisal and interpretation of RCTs by providing guidance to authors about how to improve the reporting of their trials. Later this CONSORT statement was updated in 2010.
Biased results from poorly designed and reported trials can mislead decision-making in healthcare at all levels, from treatment decisions for the individual patient to formulation of national public health policies. Little research has been conducted on reporting of RCTs in important dental journals in India; hence, the aim of this study was to evaluate the quality of reporting of RCTs in ten Indian dental journals over the period 2010–2011.
| Materials and Methods|| |
This study included all RCTs published as full-text articles reported in ten Indian dental journals over the period from 2010 to 2011. The journals included were:
- Indian Journal of Dental Research
- Journal of Indian Association for Public Health Dentistry
- Journal of Indian Society for Pedodontics and Preventive Dentistry
- Journal of Indian Society of Periodontology
- Journal of Maxillofacial and Oral surgery
- Journal of Conservative Dentistry
- Journal of Indian Academy of Oral Medicine and Radiology
- Journal of Indian Prosthodontic Society
- Journal of Indian Orthodontic Society
- Journal of Oral and Maxillofacial Pathology.
We decided to include trials published in these journals because they are leading dental journals, published by their respective national associations and their methodological reporting has not been systematically studied until date. We studied the trials published in 2010–2011 since the revised CONSORT statement was published in 2010 to improve the quality of reporting of RCTs.
Medline and PubMed databases were searched and trials were considered to be RCTs if the words “random,” “randomly,” “randomization,” or “randomized” were used to describe the allocation method in the text. The relevant trials were identified by searching Medline using a highly sensitive search strategy developed and validated by Robinson and Dickersin  and all phases of the search strategy are shown in [Table 1]. Hand searching of these journals was also carried out by three of the authors to check if any potential trial was missing.
|Table 1: Mean Consolidated Standards of Reporting Trials scores for articles from various journals|
Click here to view
Inclusion and exclusion criteria
Abstracts of all the articles were read first and if the articles appeared to be describing an RCT, then the full-length paper was obtained. RCTs were defined as controlled experiments designed to evaluate an intervention using a random method to assign study units to a test or comparison group. All RCT's which met this definition were included. Articles reporting studies that did not meet this definition were excluded. The following exclusion criteria were then applied:
- Articles that were found during the quality assessments to be nonrandomized trials orin vitro experimental studies were excluded from further analysis
- Articles reporting pilot studies
- Articles describing secondary analysis of primary data
- Articles which describes only part of the methodology and in detail methodology described elsewhere.
After applying the inclusion and exclusion criteria, the journal and date of publication of each included study was recorded. Each article was assessed against the CONSORT criteria tool, as described by Schulz et al. This tool evaluates the presence of 25 item checklist and provides guidance for reporting all randomized controlled trials, but focuses on the most common design type-individually randomized, two groups, parallel trials. Other trial designs, such as cluster randomized trials and noninferiority trials, require varying amounts of additional information. For each criterion, the articles were given a score of “1” when the criterion was assessed as being present and “0” for those that did not meet the criterion. A total score was, therefore, obtained for each article. A data collection sheet, containing the criteria, was developed and piloted. All the reviewers then applied the criteria to eight papers, for training purposes. The interobserver agreement of the three authors (VS, ASKB, GP) was rated by calculation of Kappa values (0.88, 0.76, 0.80). Any disagreement was resolved by discussion between the reviewers; if the disagreement could not be resolved by discussion, the opinion of the senior reviewer was sought.
| Results|| |
One hundred and twenty articles were found from the PubMed and Medline databases and sixty four articles were found through hand search which made the pool of 184 articles. Of these 184 articles, 29 were found to be non-RCTs, eight articles were pilot studies and 11 articles involved secondary analysis of data [Flow chart 1].
The remaining 106 articles were derived as follows: 25 from Indian Journal of Dental Research, 10 from Journal of Indian Association for Public Health Dentistry, 9 from Journal of Indian Society for Pedodontics and Preventive Dentistry, 14 from Journal of Indian society of Periodontology, 10 from Journal of Maxillofacial and Oral Surgery, 18 from Journal of Conservative Dentistry, 4 from Journal of Indian Academy of Oral Medicine and Radiology, 6 from Journal of Indian Prosthodontic Society, 8 from Journal of Indian Orthodontic Society, and 2 from Journal of Oral and Maxillofacial Pathology.
The mean number of criteria present per article was 12.2 (standard deviation [SD] = 2.2) and only 5 of 106 articles got total possible score. The mean number of criteria per articles published in Indian Journal of Dental Research was 14.3 (SD = 2.9), Journal of Indian Association for Public Health Dentistry was 11 (SD = 2), Journal of Indian society for Pedodontics and Preventive Dentistry was 11.6 (SD = 2.8), Journal of Indian Society of Periodontology was 13.2 (SD = 3.4), Journal of Maxillofacial and Oral Surgery was 9.5 (SD = 2.3), Journal of Conservative Dentistry was 12.6 (SD = 3.8), Journal of Indian Academy of Oral Medicine and Radiology was 11.8 (SD = 2.9), Journal of Indian Prosthodontic Society was 10 (SD = 1.2), Journal of Indian Orthodontic Society was 13 (SD = 2.2) and Journal of Oral and Maxillofacial Pathology was 09 (SD = 1.1) [Table 1].
Most of the articles (69%) did not mention about justification for sample size calculation, 47% of the articles had not mentioned about statistical methods, 89% of the articles had not mentioned about allocation concealment, 86% of the articles had not mentioned about funding and 63% of the articles had not mentioned about limitations of the study [Table 2].
|Table 2: Adherence of randomized controlled trialreports to Consolidated Standards of Reporting Trials statement|
Click here to view
| Discussion|| |
This study was intended to assess the quality of reporting of randomized controlled trails in ten Indian dental journals. There are no published studies to answer this research question; this is the first study in India. 106 articles published in ten journals were assessed using the CONSORT statement proposed by Shultz, Altman, and Moher. The consort tool used in this study was proposed in the year 2010; before it many similar studies were reported using CONSORT statement described by Scherer and Crawly (1998). This revised CONSORT statement provides guidance for reporting of all randomized controlled trials.
The quality of reporting of randomized clinical trials in ten Indian academic journals were poor with mean number of criteria 12.2 which is similar to the study conducted by Pandis et al. where the authors has assessed the quality of Randomized clinical trials in six major dental journals. One of the reasons in deficiencies in the reports may be due to the word limits imposed by the journals and none of these ten journals mentioned about CONSORT statement in their guidelines for authors.
When the results of this study and previous Chinese studies were compared, the Indian reports had a higher reporting rate of each key methodological item. The study evaluating 7422 Chinese RCT studies published during 1999–2004 (13 Chinese medical journals extracted by hand search) revealed that 1.1% of the studies reported sample size determination, 7.9% reported random sequence generation, and 0.3% reported allocation concealment. According to another study evaluating 142 RCT papers from 2004 to 2007 published in five leading Chinese medical journals, none of the studies reported sample size determination, 26.8% reported random sequence generation, 4.2% reported allocation concealment, 10.6% reported blinding, and 0.7% contained flow diagrams. Our study showed that the reported percentages for all key methodological items were greater than those reported in either Chinese studies.
Another important finding of this study is that only 14% articles adequately mentioned about source of funding and conflict of interest which is less compared to another study reported from India wherein the sources of funding and conflicts of interest were not declared in over three-fourths.
One of the limitations of this study is that we evaluated the quality of reporting, which is not the same as the methodological quality of the study. It is possible that a poorly reported study is well designed and executed, and a well-reported one may have several shortcomings. However, empirical evidence exists that indicates poorly reported studies are associated with larger estimates of intervention effect, i.e. poor reporting reflects poor methodology, which in turn is associated with biased results.
Poor adherence of RCT reports from India to the CONSORT reporting guidance may involve a variety of factors. The most likely reason is unfamiliarity with the CONSORT statement. To improve familiarity, more journals should include the CONSORT statement in their “Information for Authors” and require adherence to the statement for RCT reports. Similarly, increased education and training in the designing and reporting of RCTs may also help improve reporting quality. We believe that deficiencies in RCT reporting might reflect poor research. Although it is evident that education in clinical trial methodology and biostatistics is essential to better adhere to the CONSORT statement, such resources have been limited in India. Previous studies reported that the involvement of a trial methodologist (epidemiologist or biostatistician) improves reporting quality. In addition, the skills of good medical content writers may help to improve the quality of reports. Future studies to assess the quality of RCT's should include barriers preventing the diffusion of CONSORT adoption and ways to improve reporting quality should further be explored.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Jüni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 2001;323:42-6.
Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet 2005;365:1159-62.
Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: Comparative study of articles indexed in PubMed. BMJ 2010;340:c723.
Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al.
Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-9.
Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:834-40.
Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol 2002;31:150-3.
Pandis N, Polychronopoulou A, Eliades T. An assessment of quality characteristics of randomised control trials published in dental journals. J Dent 2010;38:713-21.
Wang G, Mao B, Xiong ZY, Fan T, Chen XD, Wang L, et al.
The quality of reporting of randomized controlled trials of traditional Chinese medicine: A survey of 13 randomly selected journals from mainland China. Clin Ther 2007;29:1456-67.
Xu L, Li J, Zhang M, Ai C, Wang L. Chinese authors do need CONSORT: Reporting quality assessment for five leading Chinese medical journals. Contemp Clin Trials 2008;29:727-31.
Tharyan P, Premkumar TS, Mathew V, Barnabas JP, Manuel R. Editorial policy and the reporting of randomized controlled trials: A survey of instructions for authors and assessment of trial reports in Indian medical journals (2004-05). Natl Med J India 2008;21:62-8.
Al-Namankany AA, Ashley P, Moles DR, Parekh S. Assessment of the quality of reporting of randomized clinical trials in paediatric dentistry journals. Int J Paediatr Dent 2009;19:318-24.
Cobo E, Cortés J, Ribera JM, Cardellach F, Selva-O'Callaghan A, Kostov B, et al.
Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: Masked randomised trial. BMJ 2011;343:d6783.
Guntipalli M Naidu
Department of Public Health Dentistry, Dr. SNR Siddhartha Institute of Dental Sciences, Gannavaram, Andhra Pradesh
Source of Support: None, Conflict of Interest: None
[Table 1], [Table 2]
| Article Access Statistics|
| Viewed||2020 |
| Printed||37 |
| Emailed||0 |
| PDF Downloaded||175 |
| Comments ||[Add] |