“This is the biggest research evaluation ever carried out at international level. It is an x-ray scan we make available for the Country. Transparency has been the watchword during these two years of work and, from today, the collected data can be accessed by everyone”. Stefano Fantoni – the President of ANVUR (Italian acronym of National Agency for the Evaluation of University and Research) – chooses these words opening the conference on the Research Quality Evaluation 2004-2010.
Carried out on 14 different discipline areas identified by the Italian National University Council, the Evaluation also separated the rankings by type of Structure (such as Universities, Research Institutes, etc…). First of all, however, it is important to underline that the Research Quality Evaluation does not compare the different scientific areas or evaluates single researchers.
In the time period taken into consideration (2004-2010), the number of publications in Italy has grown faster than the European and OECD average, placing itself just below the worldwide average. Scientific specialization in Italy has increased especially for Industrial and Information Engineering, Mathematical and Computer Sciences, Earth Science, Agricultural and Veterinary sciences; on the other side, a decrease has been registered for Physics, Chemistry, Biology and Medicine.
In respect to the worldwide average, however, Italy results more specialized in Mathematical, Physical, Earth and Medical Sciences. The impact factor is higher both than the worldwide and OECD one and in line with the European average. Within Europe, the Italian impact-factor is higher than the Spanish one, in line with France and inferior to Germany, Netherland and United Kingdom.
The quality of research
The ANVUR experts have conducted an analysis of all the products of research (articles, monographs, conference proceedings, software, etc…) based on relevance, originality and internationalization by means of two different methodologies: bibliometrics evaluation and peer review.
The ANVUR has received 184,878 products, instead of the 194.763 expected. On average and per area, 5.1% of products are missing. The National Research Council has submitted half of the totality of products and the experts of the Industrial Engineering Area have received the 97% of expected products (submitted by more than 5,500 researchers from 91 Research Structures), a data confirming the extremely high vivacity of Italian research in the Engineering field.
For Mathematics and Computer Science, Biology, Economics and Statistics the overall majority of submitted products are papers. For Historical and Forensic sciences on the contrary, monographs are the majority, as for Social and Political Sciences.
English represents the 62.1% of the products’ publishing languages, with remarkable differences among the areas (from 88.6% for Industrial and Information Engineering to 5.7% for Law). Humanistic areas like Classics, Philology and Literature, History and Arts have submitted a 13.2% of products written in a foreign language different from English.
The Internationalization factor for Medicine is 81.3%. This data has been well-established for many years and no changes have been highlighted for the years 2004-2010. Another confirmation regards the trend of publishing results on journals indexed by database like Web of Science (WoS) or Scopus: almost the 90% of products, in fact, has a bibliometrics evaluation.
The ANVUR experts could then choose whether evaluating the products by bibliometrics analysis (based on number of citations and journal’s impact factor) or by means of peer review (with referees appointed by the same experts, usually two for product). The peer review evaluation has involved 14,770 referees, 10,150 Italian and 4.620 with foreign affiliation.
“Inactive” subjects
From the report emerges a severe penalization for those Structures with strong presence of inactive or partially inactive subjects. For example, in the Mathematical Sciences Area, the average mark without the missing or penalized products is 0.70, whilst it is 0,595 when including them.
Even though many prestigious universities abroad commonly employ subjects who are very active in areas completely different from research (like teaching, coordination, etc…), greatly advantaging the Structure itself, these benefits have not been taken into consideration by the ANVUR evaluation process and the presence of inactive subjects has considerably affected the final results.
“Third mission” and patents
This parameter measures the direct impact of research institutes on society, beyond the traditional missions of education and research. “Third mission” means, for example, enhancing the economic, cultural or social value of knowledge, patents included.
Italian Universities register a patent every 19 subjects, whilst for Research Bodies is one patent every 26 subjects. It is interesting to notice that in the Biological Sciences Area, patents represent only the 0.26% of the totality of submitted products, which is in contrast with the high quality of the scientific products and their potential applications.
This data suggests that in Italian Universities and Research Institutes, scientific knowledge and technologies development are not very much regarded in terms of patents and technological transfer. The situation is even worse for Medical Sciences, where patents represent only the 0.13% of all submitted products.
Ranking by Areas
Rankings of products by areas go from 0, which means “of limited value”, to 1, which represents excellence. Chemistry and Physics “shine” with a 0.78, followed by Industrial and Information Engineering and Classics, Philology-Literature and History -Arts (both 0.66).
Quite far away, we can find Economics and Statistics (0.32) and Social and Political Sciences (0.45). In regards to the percentage of excellent products, the Civil Engineering Area reached only a poor 8.9%. The top is again for Physics and Chemistry (67.08%, 56.88%), followed by Industrial and Information Engineering (53.82%), Architecture (42.03%), Mathematical and Computer Sciences (41.94%), Biological Sciences (40.06%), Earth Sciences (34.7%), Medicine (33.96%) and Psychological Sciences (33.91%).
The best research is in the North
Universities have been classified as large, medium and small, based upon the number of subjects evaluated for each Structure. Padova ranks as the best large University, at the top for 7 scientific areas on 14. Just behind, Milano-Bicocca and Verona Universities, while Trento, Bolzano and Ferrara are the best medium Universities.
Among the small ones, Normale and Saint Anna of Pisa are at the first places, followed by Luiss University, in Rome. Among the Research Institutes, the National Institute of Geophysics and Volcanology and the National Institute of Nuclear Physics perform extremely well, whilst the National Research Council (CNR) does not. According to its President, Luigi Nicolais, these negative results should be read considering that the CNR’s mission involves also the transfer of knowledge at the service of the Country development enhancement, not “just” research. This mission should be somehow recognized by the evaluation process.
In conclusion, in spite of some criticisms and suggestions for the future (the next evaluation should be carried out in 5 years), an epochal change has been unanimously acknowledged: finally the evaluation has been focused on merit and transparency.
“Italy enters the Europe of evaluation” cheers the Minister of Education, Maria Grazia Carrozza “and I really hope that this will enhance the Secondary Education System. It’s a revolution at the service of citizens.”
