Thursday, 3 March 2016

The issue of self-citation


Ever wondered where the greatest Indian scientists of present times are hiding? Well, they are not hiding, they are just not there. Scientific excellence is measured by the quality and quantity of research articles published in international journals (let's count national journals also to minimize the risk of sedition charges).  There needs to be some comparisons drawn with other countries. It would possibly be unfair to compare India with top 20 countries ranked as per their H-index (http://www.scimagojr.com/). Accordingly, as of today, India holds 22nd position  and just above is Russia and just below is Brazil. Taking Russia for comparison, I suppose, would still be brutal. Let's take Brazil here for comparison. I will also take couple of asian countries: Singapore and Pakistan (a developed and a developing country). Well for some excitement. Let us assess through following criteria:

1. Total citable documents
2. Citations/document
4. Number of international collaborations

1. Total citable documents




The wow plot of indian science. Isn't it? One could simply argue that india and brazil being disproportionately more populous as compared to Singapore would certainly show greater number citable documents. However, the accelerated publication  observed in last 10 years for india and Brazil looks remarkable. Even greater acceleration observed in last 5-6 years for india as compared to Brazil is the "wow" I mentioned. Here is the news: this wow trend is projected by the bureaucrats  of indian science in every other general public lecture on indian science. In general, Fractional Count  or FC (number of authors belonging to that particular country) for each article is calculated and sometimes further down weighted for some research journals like that of astrophysics for balancing reasons. Nevertheless, FC or weighted FC would essentially be number of articles with some adjustments like the ones mentioned above that, I suppose, would only change the trend quantitatively and not qualitatively. Some studies also attempt to normalize #articles by per capita expenditure to claim the better performance of indian science. Well, on those lines, one can also argue to  normalize it further by the purchasing power of currency. For example, the PhD salaries in Singapore and Brazil are five and two times higher than that in India. Let's, for the time being, buy that Indian science performs better when scaled to per capita expenditure. The values of per capita expenditure in R & D in India is USD 39 and for Pakistan it's USD 13. Later, in this post these values will be revoked.

2.  Citations/document

This is an important criteria that directly reflects how relevant a particular article is. Following plot clearly suggests that not Singapore, but Brazil also outperforms India. Papers published in recent times would still not be sufficiently cited and, therefore, the trend of last 10 years can be ignored for this analysis. Moreover, closeness of India and Pakistan might worry nationalists! But hold on riot is yet to come.



















One important factor, which is ignored in the above plot is the tendency of scientist to cite their own papers, known as self-citations. This reminds me an incident, where one of my senior colleague in IGIB (in the year 2005) was asked by the reviewer of an ordinary journal to delete ref # 1-12 (or so) from the article because these were  not directly relevant in the context. Coincidentally these were  also self-citations. Clearly, it would be interesting to replot the the ciations/document after subtracting the self-citations. Here it is:

















Even a layman would appreciate that a more  appropriate assessment of scientific performance should be measured through its quality like the plot above rather than quantitative measures that count the bulk of documents while ignoring the underlying relevance. In this regard, India is hardly better than Pakistan, a country that we unanimously considered inferior to ours wrt science and education. Remember the per capita expenditure for pakistan is 13 USD, one third that of India. Contrary to this observation, at my present institution, I have seen many many numbers and plots that claim bloom of indian science. It is also being claimed through articles like this one: http://scroll.in/article/777202/science-research-in-india-is-flourishing-despite-funding-cuts-and-indifference-says-new-study , which was recently floated around scientific community in India. It says:

"The new study, while taking stock of prevailing problems, suggests that the negative perception is partially because of a lack of media interest and studies the impact of Indian publications in comparison to the countries with similar spending on science research."

Does Paki media broadcasts, advertises and encourages their science to achieve the same level of citations/article as of India? Chutzpah!

he same article also claims:

This, coupled with, “good collaborative ties” of Indian researchers with the rest of the world, ensures a steady stream of output in the face of budget cuts and abysmal spending, even as the country constantly tries to punch above its weight.

This can be again be directly challenged as following:

4. Number of international collaborations

International collaborations can arguably be considered as one of the criteria to judge the scientific accomplishment for following reasons:

  • It helps collating sufficient evidence of the scientific claims
  • Helps training the manpower in cutting edge techniques
  • Helps in converging different ideas. Two heads are better than single.  
  • Many great scientific discoveries were achieved through collaborations. Discovery of Higgs bosons is an example.
Here is the plot:



















You see the riots? Pakistan is a way ahead of India and Brazil and in fact the difference has expanded in last 5 years. When it comes to science, India seems more conservative and shy than the one who is labeled. Recall the similar or probably worse purchase value of currency and lower (by 1/3) per capita expenditure on R&D of Pakistan as compared to India.

It is also notable that I am comparing with pakistan and did not even talk about Singapore's performance. The problems with most of the scientometric studies is that the authors tend to project numbers, controls and plots to their convenience. Ideally, I could have sat to work on the issue in a more comprehensive manner, the question is whether it is worth. If we count the articles published in top journals in last 5 years from India, it will be on our finger counts and probably will not go to double digits. Isn't that sufficient to guess the quality?

PS: All the data and plots were obtained from http://www.scimagojr.com/



No comments:

Post a Comment