| Abstract|| |
Context: There is a need to evaluate the quality of postgraduate dissertations of dentistry submitted to university in the light of the international standards of reporting.
Aims: We conducted the review with an objective to document the use of sampling methods, measurement standardization, blinding, methods to eliminate bias, appropriate use of statistical tests, appropriate use of data presentation in postgraduate dental research and suggest and recommend modifications.
Settings and Design: The public access database of the dissertations from Rajiv Gandhi University of Health Sciences was reviewed.
Subjects and Methods: Three hundred and thirty-three eligible dissertations underwent preliminary evaluation followed by detailed evaluation of 10% of randomly selected dissertations. The dissertations were assessed based on international reporting guidelines such as strengthening the reporting of observational studies in epidemiology (STROBE), consolidated standards of reporting trials (CONSORT), and other scholarly resources.
Statistical Analysis Used: The data were compiled using MS Excel and SPSS 10.0. Numbers and percentages were used for describing the data.
Results: The “in vitro” studies were the most common type of research (39%), followed by observational (32%) and experimental studies (29%). The disciplines conservative dentistry (92%) and prosthodontics (75%) reported high numbers of in vitro research. Disciplines oral surgery (80%) and periodontics (67%) had conducted experimental studies as a major share of their research. Lacunae in the studies included observational studies not following random sampling (70%), experimental studies not following random allocation (75%), not mentioning about blinding, confounding variables and calibrations in measurements, misrepresenting the data by inappropriate data presentation, errors in reporting probability values and not reporting confidence intervals. Few studies showed grossly inappropriate choice of statistical tests and many studies needed additional tests.
Conclusions: Overall observations indicated the need to comply with standard guidelines of reporting research.
Keywords: Academic dissertations, peer review of research, quality of research, statistical methods, study design
|How to cite this article:|
Shirahatti RV, Hegde-Shetiya S. Review of research designs and statistical methods employed in dental postgraduate dissertations. Indian J Dent Res 2015;26:427-34
|How to cite this URL:|
Shirahatti RV, Hegde-Shetiya S. Review of research designs and statistical methods employed in dental postgraduate dissertations. Indian J Dent Res [serial online] 2015 [cited 2019 May 23];26:427-34. Available from: http://www.ijdr.in/text.asp?2015/26/4/427/167634
Systematic use of the research methods is an integral part of any scientific discipline including the fields of medicine and dentistry. In our country, research is an integral part of the dentistry's postgraduate curriculum. In fact, research conducted by postgraduate students virtually accounts for the majority of the research output by dental professionals. The teachers and faculty employed in dental institutions are mostly involved in academic and clinical activities, with very little time devoted for independent research. The research activity of the faculty in dental institutions is mostly restricted to providing guidance to the postgraduate students. A very negligible number of undergraduate students or general dental practitioners are involved in the research. Thus, the postgraduate dissertations are essentially the solitary research activity in Indian dentistry.
Over 2000 postgraduate students submit the research reports in over 100 dental colleges all over the country each year. The postgraduate student dissertations are subsequently presented at conferences, and some of them are published in national and international journals. However, a critical look into the international peer reviewed journals reveals that very few of these studies actually get published. There have been various reports concerning the quality of research and strategies for the incorporation of research in dental curriculum.,,
A recent questionnaire survey of research challenges and opportunities faced by dental students revealed that the postgraduate students face a considerable amount of challenges while conducting research and reporting. One among the major issues revealed in the aforementioned survey is the lack of understanding of the research methods and statistical analysis by postgraduate students.
The International Biomedical Literature has considerably evolved. There are several excellent guidelines available for reporting research and statistics.,,,,, These guidelines are described in depth various aspects of reporting research based on the study designs. There are some general guidelines of reporting such as the need to mention the study design in the title itself, aspects that need to be reported in methodology such as sampling, calibration and standardization of instruments and measurements, methods employed to ensure unbiased selection and allocation and appropriate use of statistics. Such guidelines are applicable to all reported research. Some guidelines are more design specific such as the need to mention the method of randomization in a randomized controlled trial (RCT) or the need to mention the method of selecting a control in case–control study. Both general guidelines of reporting and guidelines specific to individual research design were reviewed in this study. However, the utilization of those guidelines by the Indian dental postgraduate students is not known as the challenges faced by Indian research workers are different from that of the developed world. The quality of the dissertations submitted is mostly evaluated by the faculty of the respective disciplines. Although, postgraduate teachers in each discipline have some experience of conducting research; most of them may not be systematically trained in research methods and statistics thus resulting in errors while reporting research., Thus, there is a need to evaluate the quality of these submitted dissertations in the light of the international standards. As per the knowledge of the investigators, there are no reports available that document the study designs and statistical methods employed by postgraduate dental researchers in India.
The Rajiv Gandhi University of Health Sciences has the highest number of postgraduate dental students graduating each year than any other university in India. It is also the university that has some of the oldest and most reputed dental institutions of the country. The university has a public access database of the dissertations submitted. Hence, the investigators conducted the present review documenting the use of various study designs and statistical methods employed in dental postgraduate dissertations submitted to the university. The authors also intended to develop recommendations to improve the postgraduate dissertation reporting. Thus, the aim of this study was to document the use of various study designs and statistical methods employed in dental postgraduate dissertations submitted to Rajiv Gandhi University of Health Sciences, Bengaluru, between years 2006 and 2009. Our objectives were:
- To enumerate and classify the studies according to the main type of study design and postgraduate discipline
- To document the use of sampling methods, measurement standardization, blinding and other methods to eliminate bias in postgraduate dental research
- To enumerate the types of variables and the appropriate use of statistical tests employed in research
- To document the data presentation methods and appropriate use of recommended methods of reporting
- To suggest and recommend modifications in current postgraduate dissertation reporting methods, if necessary.
| Subjects and Methods|| |
Study design, sampling frame and selection criteria
A systematic review was conducted by obtaining a systematic selection of secondary data. Dental post-graduate dissertations submitted to Rajiv Gandhi University of Health Sciences, Bengaluru, between years 2006 and 2009 and available for public view at the official website constituted the sampling frame. All listed dissertations were included and evaluated for any duplicating listing. Of the 348 items in the list 11 were duplications of the same title and hence were excluded from the review.
Preliminary evaluation of eligible dissertations
The titles of the studies were read, and those studies mentioning a study design in the title were further classified. The study designs as mentioned by authors were summarized to calculate the frequency of the type of study designs as per authors' assessment. This was followed by reviewing of each study to ascertain the actual study design as per the assessment of the investigators according to the standard definitions of the study design.
Stratification, sampling method and sample size
There were 337 eligible dissertations which were stratified into three main study groups according to the following definitions:
- Group 1: Experimental studies (n = 108) – Were defined as “any study involving any intervention or manipulation (Therapeutic, or prophylactic drugs, devices, or techniques) with an active attempt to change a disease determinant or progress of a disease in humans or animals"
- Group 2: Observational studies (n = 97) – Defined as any study “allowing the nature to take its own course, the investigator measures but does not intervene"
- Group 3: Laboratory based and in vitro studies (n = 132) – In vitro study was defined as “study using excised tissues." Laboratory-based studies and studies done on human parts such as skull, cadaver, or studies using a duplicate model of human body (study casts) or virtual simulation of human body (finite element analysis) were also included here.
Detailed evaluation of selected dissertations
Ten percentage of dissertations were randomly chosen from each group using computer generated random numbers using MS Excel book (Microsoft Corporation, One Microsoft Way, Redmond, WA 98052-6399.) employing stratified random sampling. Thirty-two dissertations thus selected constituted the final sample size. Since this was a descriptive study, selecting about 10% of studies in each group was arbitrarily deemed as adequate by the investigator on consultation with guide and hence no further attempt was made to calculate the required sample size.
Evaluation of methods and results reported in observational studies
Evaluation of observational group of studies was done according to strengthening the reporting of observational studies in epidemiology (STROBE) statement. The randomly selected studies were categorized as cohort, case–control and cross-sectional studies. Cohort studies were defined according to STROBE explanation and elaboration document  as studies where “Investigators follow people over time, obtain information about people and their exposures at baseline, let time pass, and then assess the occurrence of outcomes." The definition of case–control study was also obtained from STROBE document  defined as “In case–control studies, investigators compare exposures between people with a particular disease outcome (cases) and people without that outcome (controls)." Accordingly the cross-sectional studies were defined according to STROBE  as study where “investigators assess all individuals in a sample at the same point in time, often to examine the prevalence of exposures, risk factors or disease.” Each of these selected studies was evaluated according to the corresponding STROBE checklists for cohort studies, case–control studies, and cross-sectional studies. Any observational study that could not be categorized as any of the above; a general 22-item checklist was used.
Evaluation of methods and results reported in experimental studies
Consolidated standards of reporting trials (CONSORT) statement checklist was consulted, and the included studies were reviewed accordingly. The RCTs were defined as experimental studies involving a process of random allocation of participants to the intervention groups. A special group of experimental studies; the split-mouth design was defined as the allocation of teeth, quadrants or sextants in a patient are allocated to two or more interventions.
Evaluation of methods and results reported inin vitro and laboratory based studies
As there was no standard guideline available for reporting of these studies when the study was conducted investigators prepared a general checklist for evaluation of these studies and the aspects considered here for review have been reported in [Table 1],[Table 2],[Table 3] of the results section.
|Table 1: Use of scales of measurement, summary measures, and statistical analysis reported in dissertations|
Click here to view
|Table 2: Aspects of probability values used in reported statistical analysis of dissertations|
Click here to view
|Table 3: Appropriateness of the use of data presentation in the dissertations|
Click here to view
Statistical evaluation of all studies combined was also done by enumerating methods of data presentation, types of variables, types of statistical tests used, the appropriateness of the test and errors in the presentation of statistical results.
Data compilation and analysis
The data were compiled using MS Excel and Statistical Package for Social Sciences (SPSS for Windows, Version 10.0. Chicago, SPSS Inc.) software programs. Numbers and percentages were used for describing the data. Studies and the pertaining variables were categorized into broad nominal categories indicating the observations. Data were presented in the form of tables and figures, including pie charts and bar diagrams.
| Results|| |
Preliminary evaluation of dissertations
The titles of 331 dissertations were reviewed to ascertain the common study designs. A total of 287 (86%) of the dissertations mentioned the study design in the title itself. The study designs as mentioned by the authors in the title were: Comparative study (n = 111, 38.7%), in vitro (n = 78, 27.2%), clinical (n = 61, 21.3%), histopatholoical (n = 10, 3.5%), prevalence study (n = 9, 3.1%), diagnostic study (n = 7, 2.4%), epidemiological (n = 5, 1.7%), retrospective (n = 3, 1%), survey (n = 2, 0.7%), and randomized clinical trial (n = 1, 0.3%).
We reviewed the dissertation to evaluate and re-categorize the study designs and have presented our results in [Table 4]. The articles termed as “comparative” study when reclassified were found to be a mix of all three major types of studies; the observational, experimental and in vitro when individually analyzed. The studies termed “clinical” when closely examined did not fit the classical definition of a “clinical trial" with many of them being just observational studies wrongly classified.
|Table 4: Preliminary evaluation of eligible dissertations and categorization according to study type and branch|
Click here to view
Prosthodontics (61), orthodontics (55) and periodontics (48) were the disciplines that contributed the highest number of research reports out of overall 337 eligible reports [Table 4]. Overall, the “in vitro” studies were the most common type of research (39%) followed by observational (32%) and experimental studies (29%) [Table 4]. In the discipline “conservative dentistry” 92% of the studies were in vitro, followed by “prosthodontics” in which 75% of reported studies were in vitro or laboratory based. Disciplines “oral surgery” (80%) and “periodontics” (67%) contributed to experimental studies as a major share of their research. “Orthodontics” and “pedodontics” had a uniform mix of all three study types in their contributions [Table 4].
Detailed evaluation of randomly selected dissertations
The thirty-two dissertations randomly selected for detailed evaluation were categorized as observational (n = 11), experimental (n = 9) and in vitro (n = 12). Among the observational studies (n = 11) majority were classified to be cross-sectional (n = 7, 64%) followed by case–control (n = 3, 27%) and retrospective (n = 1, 9%) study designs. Among the experimental studies (n = 9) majority were split-mouth designs (n = 3, 33%) followed by case series (n = 2, 22%), before and after study without control (n = 1, 11%), RCT with concurrent parallel design (n = 1, 11%), RCT with block randomization (n = 1, 11%) and controlled clinical trial (n = 1, 11%). Of the 32 dissertations selected majority were conducted on human participants (n = 20, 62.5%) followed by extracted teeth (n = 8, 25%). Study settings in majority of studies were in hospitals (n = 16, 50%) followed by laboratory settings (n = 12, 37.5%). Cross-sectional design was the most popular among the observational studies (7 out of 11.) Controlled clinical trials with “split-mouth design” were the most common among the experimental studies (3 out of 9). Also, the majority of studies were done in humans (62.5%), in hospital setting (50%), and interval scale was the most common scale of measurement.
The majority of observational studies that required random sampling did not follow random sampling (70%) or just mentioned the word “random” without actually explaining, which random sampling method was used. Similarly, about 75% of the studies allocated interventions by convenience instead of random allocation. A number of studies (64.3%) showed inadequate efforts to prevent bias such as the need for blinding and adjusting for confounding variables. Many studies the conclusions were drawn despite the presence of confounding variables that could have changed the conclusions of the study if accounted for. Variables were defined clearly, and methods of measurements were clearly mentioned in most of the studies, but calibration of instruments and examiners was needed in most (65%) of the studies [Table 5]. Calibration of measurements would have improved the credibility of the results in about 64% of studies [Table 5].
|Table 5: Detailed evaluation of studies with inappropriate aspects of study design|
Click here to view
Some studies (64%) required baseline comparability between the groups for ensuring unbiased conclusions. It was advisable to have followed methods such as stratified allocation, improved selection criteria and by the choice of statistical tests to account and adjust for baseline differences. In one study, there was a need for control group for meaningful comparison. As many as 60% of human/animal studies did not mention prior approval from Institutional Ethics Committee and about 36% experimental studies and 44% observational studies conducted in human volunteers did not mention whether consent was obtained or not. About 93% studies did not mention how the sample size was calculated for the study [Table 5].
Interval measurements were the most common types (50%) and mean and standard deviations (>50%) were the most common summary measures [Table 1]. Unnecessary high precisions were reported in representing mean and standard deviations. Chi-square test and ANOVA were the most common type of statistical tests (17%) followed by student's independent sample test (15%) [Table 1]. Few studies showed the grossly inappropriate choice of statistical tests such as the application of the inappropriate statistical test for the corresponding scale of measurement. Many studies needed additional tests to adjust the confounding variables [Table 2]. Some bar diagrams and pie charts misrepresented the data with an unnecessary three-dimensional representation of data. Probability values were misinterpreted in one of the studies by assuming nonsignificant difference as no difference. In one more study, there was too small a sample size to make any meaningful interpretation of the P values. Confidence intervals were rarely used for reporting the observed differences [Table 2]. Tables bar diagrams and pie charts were most popular methods of data presentations [Table 3]. Overall observations indicated the need to comply with standard guidelines of reporting.
| Discussion|| |
The review was conducted to document the use of various study designs and statistical methods employed in dental postgraduate research in India. We also reviewed the appropriateness of methodology, presentation of results, application of statistical tests, and interpretation of results with an aim to identify the common errors and misrepresentations. The dissertations submitted to Rajiv Gandhi University of Health Sciences, Bengaluru, between 2006 and 2009 were openly available for evaluation and hence were taken to represent the postgraduate research in India. The results of the present investigation demonstrate a need to improve the quality of reporting postgraduate dissertations. However, the results should be viewed in the light of this limited selection of postgraduate student dissertations to specific years of submission.
Both CONSORT and STROBE reports ,,, insist that the author reports the type of the study in the title itself. In our review, although most of the titles indicated a study design in their title, a close look showed that the choice of the words was ambiguous. As “comparison” is one of the basic tenets of epidemiology, it's natural that the majority of titles reported their study as “comparative.” However, this does not help the reader to get an idea whether it was observational or experimental or any other type of study. A simple guideline provided to students to classify their study could greatly help in improving the reporting of study designs in the title.
In the era of evidence-based dentistry, the RCTs form the “Gold standard” among the research designs. Only one “study title” out of 337 indicated the use of this classical design. Out of the surveyed 33 studies, only two were RCTs. Also “Experimental” studies were the least popular of all the designs despite the fact that the “clinical branches” form the majority in postgraduate disciplines. In fact, an overwhelming majority (>80%) of studies were in vitro studies in the two major clinical branches of conservative dentistry and prosthodontics. This is in contrast to the studies published in the international journals of these disciplines. A PubMed search of prosthodontics revealed that there are 3300 studies with clinical trials whereas only 600 studies were in vitro (5:1 ratio). A PubMed search of operative dentistry revealed that there were 2300 clinical trials compared to 520 “in vitro” studies (5:1). This is in complete contrast to our observations, where minority of studies were experimental studies (1:5) compared to in vitro. This could be due to the lack of understanding of the importance of better designs, lack of infrastructure and resources, inadequacies in curriculum design, including time constraints, as revealed in a recent survey.
The majority of observational studies did not follow important aspects of research methodology such as random sampling, Blinding, adjusting for confounding variables, calibration of instruments, and examiners. Such deficiencies have been reported in international literature as well ,, Zlowodzki et al. reported the pitfalls in conducting clinical studies and observed the following deficiencies such as lack of randomization, lack of concealment, lack of blinding, and errors in hypothesis testing (Type I and II errors). These errors can seriously affect the credibility of the conclusions drawn out of the study.
Nominal and interval measurements were the most common types and mean, and standard deviations were the most common summary measures. Other relevant measures such as median and quartiles were used very infrequently. Lang, reported that use of other measures of central tendency are more relevant in medical literature as “most medical data does not follow normal distributions and also is subject to variability.” This could be due to lack of understanding among the postgraduate students and teachers about other measures and limited time and effort devoted in postgraduate curriculum for understanding of statistics. This is also reflected by unnecessary high precisions reported in representing mean and standard deviations, misinterpreted probability values, seldom use of confidence intervals, grossly inappropriate choice of statistical tests and need for additional tests to adjust the confounding variables.
The importance of consulting a methodologist and statistician is of paramount importance during the study design. Altman et al. reported that consulting a statistician should start early during the design of the study. A study proved that not consulting a statistician substantially decreases the chances of a successful publication. A recent survey also reported deficiencies in postgraduate students in understanding the publication process and inability to relate their research in international context.
Overall observations indicated the need to comply with standard guidelines of reporting. The need for consulting the international guidelines for reporting studies is greatly emphasized, with many journals asking their authors to submit articles with a checklist of these guidelines. This initiative has resulted in better reporting of observational and experimental studies. Ellen Weber et al. reported that a greater number of authors have started complying with the international guidelines. There needs to be a greater dissemination of information among dental postgraduate students about the availability of such designs and need to adhere to them.
Teachers and the students also need to be updated with the latest knowledge on research methodology, study design, sampling, etc. Attending refresher courses on research may be advisable to strengthen the quality of the research itself as well as improved reporting. It is also advisable that the Health Universities and Dental Council should encourage the attendance of such programs by the postgraduate guides. The development of research attitude may well be developed during undergraduation itself by offering elective modules on research to those undergraduate students who wish to undertake postgraduation later in a career. The short term studentship projects commissioned by Indian Council of Medical Research are a welcome step in this direction.
The present study has reviewed only the openly available dissertations to analyze the dissertation quality amongst dental postgraduate students. The direct implication of the study is thus limited to the dissertations submitted to Rajiv Gandhi University of Health Sciences only. Nonetheless, the present investigation may give a fair thought about the quality of dissertations reported to other Indian universities as well as the postgraduate curriculum is expected to be more or less uniform throughout the country and a single regulatory body governs the curriculum. Another limitation of this study was that there was no guideline available for analyzing in vitro studies when the study was planned and conducted. Recent guidelines for reporting in vitro studies are available. However, the checklist prepared by us incorporated the important contents in such guidelines.
The quantity of research generated by dental postgraduate students is enormous and is the single largest contributor of research in Indian dentistry. There is the large potential that remains untapped in the form of these postgraduate dissertations because of the quality of the study designs and reporting. More research is needed to identify the facilitating factors to improve postgraduate dissertations to make them on par with the international literature. The findings of this review indicate a need to sensitize the postgraduate students to the standard reporting guidelines and provide them knowledge and skills needed for to translate these findings into the practice.
| Conclusions|| |
Many studies did not follow random sampling, random allocation of study subjects, blinding, accounting for the confounding variables and calibration of instruments and examiners was needed in most of the studies. Few studies showed the grossly inappropriate choice of statistical tests and many studies needed additional tests to adjust the confounder variables. Probability values were misinterpreted in few of the studies and confidence intervals were rarely used for reporting the observed differences. Overall observations indicated the need to comply with standard guidelines of reporting. A greater dissemination of information among dental postgraduate students is recommended to appraise the students about the availability of better research designs, more rigorous methodologies and international quality of reporting their dissertations.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Edmunds RK. Strategies for making research more accessible to dental students. J Dent Educ 2005;69:861-3.
Bertolami CN. The role and importance of research and scholarship in dental education and practice. J Dent Educ 2002;66:918-24.
Iacopino AM, Lynch DP, Taft T. Preserving the pipeline: A model dental curriculum for research non-intensive institutions. J Dent Educ 2004;68:44-9.
Shirahatti RV, Sura S, Sumanthprasad GR, Khurana L. Dental students research inventory: A questionnaire to assess research challenges and opportunities. J Dent Educ 2010;74:1308-18.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. J Clin Epidemiol 2008;61:344-9.
Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al.
Strengthening the reporting of observational studies in epidemiology (STROBE): Explanation and elaboration. Epidemiology 2007;18:805-35.
Lang T. Twenty statistical errors even you can find in biomedical research articles. Croat Med J 2004;45:361-70.
Rahmatulla M. Vision and challenges for dental research worker. Indian J Dent Res 2009;20:135.
Altman DG, Goodman SN, Schroter S. How statistical expertise is used in medical research. JAMA 2002;287:2817-20.
Zlowodzki M, Jonsson A, Bhandari M. Common pitfalls in the conduct of clinical research. Med Princ Pract 2006;15:1-8.
Rajeev Gandhi University of Health Sciences. Available from: http://www.rguhs.ac.in/
. [Last accessed on 2010 Jul 25].
Park K. Textbook of Preventive and Social Medicine. 18th
ed. Jabalpur: Ms. Bhanot Publishers; 2005.
Moher D. CONSORT 2010 explanation and elaboration. Br Med J. Available from: http://www.bmj.com
. [Last accessed on 2010 Jul 25].
Weber EJ, Callaham ML, Wears RL, Barton C, Young G. Unpublished research from a medical specialty meeting: Why investigators fail to publish. JAMA 1998;280:257-9.
Faggion CM Jr. Guidelines for reporting pre-clinical in vitro
studies on dental materials. J Evid Based Dent Pract 2012;12:182-9.
Ravi V Shirahatti
Department of Public Health Dentistry, Maratha Mandal's Nathajirao G Halgekar Institute of Dental Sciences and Research center, Belagavi, Karnataka
Source of Support: None, Conflict of Interest: None
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]