Year : 2009 | Volume
: 20 | Issue : 1 | Page : 81--85
Impact factor and other standardized measures of journal citation: A perspective
Vijay Prakash Mathur1, Ashutosh Sharma2,
1 Assistant Professor, Centre for Dental Education and Research, All India Institute of Medical Sciences, New Delhi 110029, India
2 Consultant, Dr. Moopen's polyclinic, Dubai, United Arab Emirates
Vijay Prakash Mathur
Assistant Professor, Centre for Dental Education and Research, All India Institute of Medical Sciences, New Delhi 110029
The impact factor of journals has been widely used as glory quotients. Despite its limitations, this citation metric is widely used to reflect scientific merit and standing in one«SQ»s field. Apart from the impact factor, other bibliometric indicators are also available but are not as popular among decision makers. These indicators are the immediacy index and cited half-life. The impact factor itself is affected by a wide range of sociological and statistical factors. This paper discusses the limitations of the impact factor with suggestions of how it can be used and how it should not be used. It also discusses how other bibliometric indicators can be used to assess the quality of publications.
|How to cite this article:|
Mathur VP, Sharma A. Impact factor and other standardized measures of journal citation: A perspective.Indian J Dent Res 2009;20:81-85
|How to cite this URL:|
Mathur VP, Sharma A. Impact factor and other standardized measures of journal citation: A perspective. Indian J Dent Res [serial online] 2009 [cited 2022 Sep 25 ];20:81-85
Available from: https://www.ijdr.in/text.asp?2009/20/1/81/49072
In the past, a scientific publication used to be a more leisurely venture. The present day scenario however, is completely different. With publication patterns of any investigator becoming the pervasive arbiters of his/her scientific career,  modern scientists are perpetually in a state of dilemma because increasingly their publications are now being used for the evaluation of their research quality. That is why the presumed quality of a defined set of journals has become the principle evaluation criteria.  Indubitably, the hallmark of academic achievement is to have a publication/publications in a high impact journal. Not far from this scenario are the confused librarians and journal editors. A librarian has to identify "the core" of his/her journal collection  and the editor needs to evaluate if the journal is meeting desired standards. With the escalating cost of journals on one hand and budgetary limitations on the other, the librarian and the faculty are often called upon to identify a core of quality journals for an institutes' library using suitable assessment criteria.  As a consequence, the scientific investigator, the editor, and the librarian have one thing in common i.e., they are required to base their decisions on certain objective measures of assessing journal quality. Over the past 50 years, these measures of assessing journal quality, also known as bibliometric indicators, have emerged as the chief quantitative measures of the quality of the research papers published, the authors, and that of the institution with which these researchers are associated.
The objectives of this paper are to provide a historical backdrop of such quantitative measures, to discuss their limitations, and to provide suggestions for the appropriate use of these measures.
Gross and Gross first reported the use of counting references to rank scientific journals.  It was Garfield and Sher of the Institute of Scientific Information (ISI) who first suggested how reference counting could measure impact but the term impact factor was not used until the publication of the 1961 Science Citation Index (SCI) in 1963.  The ISI, which was founded by Eugene Garfield, is a Philadelphia-based company and is presently owned by the Thomson Corporation of Toronto. The aim of creating the Journal Impact Factor (JIF) was to help select journals for the SCI. The inventors recognized a core group of highly cited large journals that needed to be covered in the SCI, however, they felt that this way a small but important group of review journals would go unrecognized. As a result, the JIF was created to compare journals regardless of their size.  A bi-product of the SCI was the Journal Citation Reports (JCR), which was first published in 1975. From 1975 to 1989, the JCR appeared as supplementary volumes in the annual SCI. From 1990 to 1994, they have appeared in microfiche and in 1995 a CDROM edition was launched.  The current JCRs have two editions covering journals in the areas of science, technology, and social sciences. These JCRs cover a total of 8,400 journals with a total of 5,876 journals from the science and technology industries alone.
Using optical character recognition software, the journals are first scanned. To store a research paper in its database, ISI employees highlight the following indicators/fields: author, address, journal title, volume, year, and page number. Next, a computer takes a few bytes of information from each highlighted field to build up an identifying code or 'tag' that is unique to that paper. A similar data capture and tagging process occurs for the references at the end of the paper. Algorithms then compare the citation tags with any article tags already in the database and each successful match counts as a citation. 
The ISI has three standardized measures for calculating the citations and articles received over time. These measures are impact factor, immediacy index, and cited half life.
Some Key Definitions
The impact factor is defined as: the number of times articles from a journal are cited within 2 years divided by the total number of articles published in the same journal during the 2-year period. The impact factor of a journal is intended to measure how often, on average, authors cite moderately recent articles from that particular journal.
The impact factor is a way of measuring which journal receives citations to its articles over time. The build up of citations tends to follow a curve like that shown in [Figure 1]. Citations for articles published in a given year rise sharply to a peak between 2 and 6 years after publication. From this peak, citations decline exponentially. The citation curve of any journal can be described by the relative size of the curve in terms of the area under the line, the extent to which the peak of the curve is close to the origin, and the rate of decline of the curve. The window period counted is 2 years.
The immediacy index of journal is intended to measure how often, on average, authors cite very recent articles form that particular journal, and hence how rapidly the average paper from that journal is adopted into the literature. The immediacy index gives a measure of the skewness of the curve, i.e., the extent to which the peak of the curve lies near to the origin of the graph [Figure 1].
The cited half-life is the calculated point (age in year) where 50% of the citations are under the age and 50% of the citations are over that age. The cited half-life is a measure of the rate of decline of the citation curve. It is the number of years that the number of current citations takes to decline to 50% of its initial value (the cited half-life is 6 years in the example given in [Figure 1]). It is a measure of how long articles in a journal continue to be cited after publication.
Optimum Use of Bibliometric Indicators
Measuring the scientific quality of a publication is a notoriously difficult task. The traditional method of evaluation has been peer review. However, peer review is time consuming and experts with available time are a scarce commodity.  Often, some quantitative measurement of bibliometric analysis like the JIF is used by decision-makers to ascertain the quality of research. This practice has its proponents and opponents. Much controversy has been generated on the use of this citation metric for ranking the quality of the research of individuals and research groups. Despite this, many in the biomedical sciences community continue to pay attention to the impact factor based rankings and base their decisions on these. The dilemma is can we really do without it? Or we need to understand it completely and then use it. The following section addresses these issues with the help of figures by Amin and Mabe. As pointed out by Amin and Mabe,  the JIF is affected by sociological and statistical factors like the type of journal, subject area of the journal, size of the journal, etc.
Type of discipline and its effect on citation density
As seen in [Figure 2], the impact factor varies from subject to subject, it being higher in fundamental and pure subject areas and lower in specialized or applied fields. This means that as far as is possible, the comparison of impact factors should only be made for journals in the same subject area. 
Number of authors and impact factor
Also closely connected to the subject area is the effect of the number of authors in a paper, which varies from subject to subject [Figure 3]. The number of multiple authors is generally fewer in the social sciences and more in fundamental life sciences.  Since many authors have a tendency to cite their own work, this practice usually distorts the true picture.  Most unfortunate is the tendency to cite one's own work, however tangential it might be to the topic at hand.  Therefore, one should use a sense of scientific propriety while citing one's own or parallel work.
Publication type and impact factor
Within the same subject area, there may also be a marked variation in the impact factor [Figure 4]. This is influenced by the type of the journal and the articles. In a sense, the impact factor may be unintentionally tweaked in rapid publication journals and articles of the current review type  because of a virtual deluge of papers published. It is natural that the discerning reader is always on the lookout for a review on the topic of interest.  Without doubt there is always a lot more information in a review than in original papers. Consequently, journals with a high number of reviews have an advantage in the impact factor league over those that published primary research papers. Furthermore, journals that are very selective or even restrictive can reduce the number of papers per issue and limit them to currently trendy topics. This may be affecting their impact factor rating in a positive way. 
Journal size and impact factor
The size of the journal here means the number of articles published per annum and the size of the measurement window, which in case of the JCR is 2 years. For example, if a large number of journals (4,000, arranged in quartiles based on the size of the journal) are examined and the means variation in impact factor from one year to the next is plotted against the size of the journal, there is a clear correlation between the extent of the impact factor fluctuation [Figure 5]. This means that journal size should always be taken into consideration when impact factors are compared.
Time lag in publication and impact factor
Impact factor is also affected by the time taken by the reviewers of a journal to assess a submitted research publication. If reviewing and publications are delayed and references are no longer current, they will not be included in the impact calculation. As shown by Opthof,  this point has great relevance for both authors and reviewers alike. Since the impact factor calculation by the JCR works on a 2-year time frame, it really favors research that takes less time to complete. Because research in most branches of medicine takes longer than that, it is a type of unsolicited penalty on the investigators. So what can the editors and the editorial board do about it? They can make efforts to reduce the review time and enhance the publication time. The review process should be rigorous yet fast.
Title/Abstract and the impact factor
The impact factor rating may also be affected by the way an author writes the abstract and the title of his/her article. As suggested by Sieck,  words and phrases used throughout the entire abstract and title of an article become the key words for finding an article in an online database. Thus, selection of words and phrases is critical to ensuring that an article is found and more importantly referenced in future publications.
Journal title and the impact factor
The impact factor calculation is also effected by the title of a journal as explained by Lindhe.  When a journal changes its name, the old and new titles are not unified. A suitable example is the Scandinavian Journal of Dental Research (SJDR) now called the European Journal of Oral Sciences. The SJDR has a history of 105 yeas of continuous publication, however, when the titled was changed in 1996, the impact factor for that year was split as both the journals were now listed with an impact factor of 1.14 for the former and 0.597 for the latter, respectively.
Language and the impact factor
As the English language dominates international research and clinical literature, something that displeases the editors of most foreign language journals, is that publications in these foreign language journals are not included in the calculation of the impact factor.  Higher impact work is mostly reported in English and the availability of a work in English alone increases its citation.
Variability in impact factor due to numerator/denominator
The SCI database includes only normal articles, notes, and reviews as citable items in the denominator, but citation of all types of articles, such as editorials, letters, and meeting abstracts are included in the numerator. It has been conclusively shown that the inclusion of these additional items causes a substantial increase in the impact factor. Editors could raise the impact factor of a journal by frequent reference to their previous editorials, since the database makes no correction for self-citation, or by running a large correspondence section.  This so called numerator/denominator problem exemplifies why considerable care needs to be taken when using impact factors. 
Citation half-life or impact factor
Another controversial issue is whether the impact factor rating or the cited half-life of a publication is more relevant? While the impact factor only tells us about how many times an article has been cited, it is the cited half-life that, according to some, is more relevant. Citation half-life is the number of publication years from the current year that accounts for 50% of the current citations received. This citation metric provides an estimate of how long a publication will continue to impact the literature.
Now one may ask, if the assessment of intellectual salience is being trivialized by the use of a system that has so many imperfections,  then why is it being done? Why are the journals that have the highest impact factors considered to be the best? Why is it always that a journal in which it is the most difficult to have an article accepted has a high impact factor?  If science should be judged by its content and not its wrapping then why undermine supplant true peer review? With so many questions the most suitable answers we believe were given by Hoeffel  and Garfield  who expressed the situation succinctly as shown below:
"Impact factors is not a perfect tool to measure the quality of articles but there is nothing better and it has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted, and these are the journals that have a high impact factor. These journals existed long before the impact factor was devised. The use of impact factor as a measure of quality is widespread because it fits well with the opinion we have in each field of the best journals in our specialty".  Finally Garfield "cautioned the use of impact factor to weigh the influence of a paper amounts to a prediction, albeit coloured by probabilities." 
|1||Gannon F. The impact of the impact factor. EMBO Rep 2000;1:293.|
|2||Martin-Sempere MJ, Rey-rocha J and Garzon-Garcia B. Assessing quality of domestic scientific journals in genographically oriented disciplines: Scientists' judgments versus citations. Res Eval 2002;11:149-54.|
|3||Buchtel HA. Libraries and the Academy. Cortex 2001;37:455-6.|
|4||Davis PM. Where to spend our e-journal money? Defining a university library's core collection through citation analysis. Vol. 2. Baltimore, USA: John Hopkins University Press; 2002. p. 155-66. |
|5||Garfield E. How cam impact factors be improved? Br Med J 1996;313:411-3.|
|6||Adam D. The counting house. Nature 2002;415:726-9.|
|7||Whitehouse GH. Citation rates and impact factors: should they matter? Br J Radiol 2001;72:1-3.|
|8||Amin M, Mabe M. Impact factors: Use and abuse. Perspect Publishing 2000;1:1-6. |
|9||Adams KM. Impact factors: Aiming at the wrong target. Cortex 2001;37:600-3.|
|10||Opthof T. Submission, acceptance rate, rapid review system and impact factor. Cardiovasc Res 1999;41:1-4.|
|11||Sieck GC. The "impact factor": What it means to the impact of applied physiology. J Appl Physiol 2000;89:865-6.|
|12||Linde A. On the pitfalls of journal ranking by impact factor. Eur J Oral Sci 1998;106:525-6.|
|13||Seglen PO. Citations and journal impact factors: questionable indicators of research quality. Allergy 1997;52:1050-6.|
|14||Garfield E. Journal impact factor: A brief review. CMAJ 1999;161:979-80.|
|15||Hoeffel C. Journal impact factors. Allergy 1998;53:1225.|