SCIENTISTS STUDY THE WORLD AS IT IS,
ENGINEERS CREATE THE WORLD
THAT NEVER HAS BEEN
THEODORE VON KÁRMÁN
The European Rankings of Engineering Programmes EngiRank fills a gap in current and trustworthy information on engineering education, as well as research and innovation, in European universities and other higher education institutions (HEIs)[1] with a strong engineering profile.
Our main concern in developing EngiRank was to ensure the highest reliability of the rankings. The geographical scope of EngiRank covers the European Union, Norway and Switzerland. One of the reasons for this coverage of the rankings is related to the right of EU citizens to study in other Member States under the same conditions as nationals, and the promotion of student mobility and graduate employability are the key objectives of the European Higher Education Area (EHEA), which extends beyond the EU. Increased student mobility creates a demand for information on the quality of European higher education institutions. In addition, consortia of institutions from different Member States and Horizon Europe associated countries can apply for research and innovation funding under the Horizon Europe programmes. The European Universities Initiative, launched in 2017, develops long-term cooperation between European higher education institutions. These measures are levelling the playing field for higher education institutions across the European Union and making comparisons between them more meaningful.
Another factor considered essential for the credibility of the EngiRank is the quality and reliability of the data – the rankings are based only on trustworthy external databases containing information on European HEIs collected in a uniform way, such as the Scopus bibliographic database, the EPO Worldwide Patent Statistical Database (PATSTAT), information on participation in European Commission initiatives (Community Research and Development Information Service - CORDIS, European Education Area website), databases of programmes accredited by quality assurance agencies. No information needed to compile the rankings was collected directly from the HEIs.
It was then decided to reduce the diversification of HEIs included in the rankings in order to avoid comparing unmatchable institutions and to increase the relevance of the rankings.
The ENTRY CRITERIA include a qualitative condition:
and quantitative conditions:
The subject ranking in each of the above disciplines includes HEIs that meet the qualitative condition and both quantitative conditions: a share of publications in engineering and technology of at least 30% and a number of publications in a given discipline of at least a threshold value[3]. The institutional ranking within EngiRank includes HEIs that are classified in at least three subject rankings.
In designing the EngiRank, we paid particular attention to the degree of commitment of institutions to their economic and social missions. In order to reflect the transfer and application of academic knowledge by universities, we have included in the rankings indicators that measure the collaboration between academia and industry researchers, the use of research outputs in successful patent applications, the universities' own patent activity and - where appropriate - the contribution to sustainable development goals.
EngiRank is a composite of different categories of indicators. We believe that the scale of an institution’s activities matters: research and innovation exhibits increasing returns to scale, and the larger the scale of an HEI's activities, the more opportunities there are for students and academic staff. Therefore, size indicators measuring the volume of research output, the amount of research and innovation funding, the number of patents or the number of publications related to the selected sustainable development goals play an important role in the rankings. The inclusion of the subject rankings scores into the institutional ranking reflects returns to scope and the benefits of interdisciplinarity. Then there are the conventional efficiency indicators expressed in relative terms, such as citations per publication, share of publications in the top 10% journals, number of patent citations received on average by a publication, percentage of publications that are co-authored by industry researchers or foreign researchers. The introduction of a dynamic indicator, the change in citation impact, is something of a novelty in the universe of university rankings. Finally, we have included qualitative indicators representing engineering degree programmes accreditations and membership in a European university alliance.
[1] The term higher education institutions other than universities refers to institutions such as grandes écoles in France, Sant'Anna School of Advanced Studies in Pisa, Italy or Higher Institute of Engineering of Lisbon, Portugal.
[2] The classification of disciplines adopted in EngiRank corresponds to the OECD Fields of Research and Development (FORD), both at the level of 1-digit major fields (i.e. 2. Engineering and technology, 3. Medical and health sciences, 5. Social sciences) and at the level of 2-digit categories (2.1 Civil engineering; 2.2 Electrical engineering, electronic engineering, information engineering etc.).
[3] Only in exceptional cases, in order to include leading HEIs with an engineering profile from all EU27 countries in the EngiRank institutional ranking, has the threshold for the share of E&T publications been lowered (and when the scores of the leading institutions were very close, we decided to include all of them in the ranking). This is the case for Croatia, Malta and Flanders.
The institutional ranking is based on five criteria. The most important criteria in terms of weight are Research (28%) and Innovation (25%) - together they account for 53% of the ranking. The third criterion, SDG 9: Industry, Innovation and Infrastructure, recognises the institution's efforts to make progress towards Sustainable Development Goal 9 - the weight of this single indicator criterion is 10%. The next criterion is Internationalisation, with a weight of 16%, and the last one is Engineering and Technology Capability (weight of 21%), which links the institutional ranking with the subject rankings.
This criterion is made up of four indicators taken from the Scopus bibliographic database. Publications is the indicator representing the research output of the institution in absolute terms. Then Citations and Publications in Top 10% Journals are indicators expressed in relative terms. The last one, Change of Impact, is a dynamic indicator reflecting a change in citation impact. More detailed information on each indicator is given below.
The criterion consists of four indicators. Firstly, Research and Innovation Funding and Patents are expressed in absolute terms and refer to the European frameworks for research and innovation funding and patenting respectively. The Scopus bibliographic database was the source for the calculation of the other two indicators: Patent-Citation Count per Scholarly Output and Academic-Corporate Collaboration. Both are expressed in relative terms. More detailed information on the indicators is given below.
The single indicator criterion. The indicator measures the institution's contribution to Sustainable Development Goal 9, with the number of publications assigned to SDG9 and the field Engineering and technology.
The criterion is made up of three indicators. The International Collaboration and International Collaboration Impact indicators are derived from the Scopus bibliographic database. The last indicator relates to the institution’s participation in a European university alliance. More detailed information on each indicator is provided below.
The criterion linking the institutional ranking with the subject rankings. This criterion rewards institutions that are classified in a larger number of subject rankings and that achieve higher scores in these rankings.
[1] Under Swedish law, intellectual property rights belong to the inventors, i.e. the researchers, and not to the universities. Therefore, the zero value of the Patents indicator for Swedish universities is misleading and doesn't imply a lack of inventive activity. In order not to penalise Swedish universities for this reason, the weighted sum of the remaining indicators has been multiplied by the factor 100/95.
The EngiRankrankingsin the following disciplines:
are based on three criteria: Research (64%), Innovation (26%) and Teaching Quality (10%).
We then looked at the potential and expected contribution of university activities in the remaining disciplines to the following Sustainable Development Goals (SDGs):
Thus, for each of the above-mentioned disciplines, the additional criterion: Contribution to SDGs (5%) was added for each of the above disciplines, while at the same time the weights of the other criteria were reduced: Research - to 60% and Innovation - to 25%.
Each indicator in the subject rankings refers strictly to a given discipline. Some indicators in the institutional ranking and in the subject rankings are based on the same metrics, differing only in the scope of publications or grants taken into account. Although such indicators from the subject rankings are indirectly included in the institutional ranking (via the Engineering and Technology Capability criterion), this should not be considered as a duplication of indicators. It should be noted that some HEIs are not classified in all subject rankings (in particular, about half of the HEIs listed in the institutional ranking are classified in the medical engineering ranking). Moreover, even for HEIs classified in all subject rankings, there are publications and grants not covered by the subject ranking indicators, such as those in general engineering; nuclear energy and engineering; control and systems engineering; safety, risk, reliability and quality; media technology; bioengineering; other miscellaneous engineering. They are only included in the institutional ranking.
The criterion is made up of four indicators calculated on the basis of the Scopus bibliographic database.
The criterion consisted of two indicators. The Scopus bibliographic database was the source for the calculation of the Academic-Corporate Collaboration indicator. The values for the Research and Innovation Funding are based on information from the CORDIS database.
The single indicator criterion. The indicator is measured by the number of the degree programmes accredited by ENAEE authorised agencies or by ABET (more information on ENAEE and ABET in the frame below)
The European Network for Accreditation of Engineering Education (ENAEE) is a non-profit organisation that brings together accreditation and quality assurance agencies from different countries (mostly EU Member States) and provides a pan-European framework for the accreditation of engineering education programmes. ENAEE has established the EUR-ACE (European Accredited Engineer) label, although it does not directly accredit engineering programmes. Following a positive evaluation of the policies and procedures followed by the member accreditation and quality assurance agencies, ENAEE authorises them to award the EUR-ACE label to engineering programmes accredited by these agencies. At present, 15 agencies are authorised by ENAEE; they have signed a mutual recognition agreement, known as the EUR-ACE Accord, whereby they accept each other's accreditation decisions in respect of bachelor's and master's degree programmes. Since the foundation of ENAEE in 2006, the EUR-ACE label has been awarded to more than 1800 engineering programmes offered by more than 300 universities in 30 countries in Europe and worldwide.
The Accreditation Board for Engineering and Technology (ABET) is a non-governmental organisation founded in the United States in 1932 (originally as the Engineers' Council for Professional Development). There are currently 4,674 post-secondary education programmes in applied and natural sciences, computing, engineering and engineering technology accredited by ABET in 920 institutions in 42 countries (including Austria, the Netherlands, Poland, Portugal and Spain).
Other accreditation schemes considered were those of the Royal Institute of British Architects (RIBA) and the Nautical Institute Accreditation. Due to the very small number of universities that hold these accreditations and are also listed in the EngiRank, they were ultimately not included in the indicator.
The single indicator criterion. The indicator measures the institution’s contribution to selected SDGs in defined disciplines through the number of publications.
The table below summarises the weighting of the indicators in specific subject rankings.
RANKINGS BY SUBJECT | Publications | Publications in | Citations | Change of Impact | Academic-Corporate | Research and Innovation | Accreditations | SDG 3: Good health and | SDG 6: Clean water and | SDG 11: Sustainable cities and communities |
Chemical engineering | 16% | 16% | 16% | 16% | 16% | 10% | 10% | - | - | - |
Civil engineering | 15% | 15% | 15% | 15% | 15% | 10% | 10% | - | - | 5% |
Electrical, electronic, information engineering | 16% | 16% | 16% | 16% | 16% | 10% | 10% | - | - | - |
Environmental engineering | 15% | 15% | 15% | 15% | 15% | 10% | 10% | - | 5% | - |
Materials engineering | 16% | 16% | 16% | 16% | 16% | 10% | 10% | - | - | - |
Mechanical engineering | 16% | 16% | 16% | 16% | 16% | 10% | 10% | - | - | - |
Medical engineering | 15% | 15% | 15% | 15% | 15% | 10% | 10% | 5% | - | - |
The EngiRank includes 225 HEIs from all countries of the European Union countries as well as Norway and Switzerland:
Austria - 4 | Latvia - 3 |
Belgium - 4 | Lithuania - 2 |
Bulgaria - 3 | Luxembourg - 1 |
Croatia - 3 | Malta - 1 |
Cyprus - 2 | Netherlands - 4 |
Czechia - 10 | Norway - 2 |
Denmark - 2 | Poland - 21 |
Estonia - 1 | Portugal - 9 |
Finland - 2 | Romania - 10 |
France - 58 | Slovakia - 3 |
Germany - 31 | Slovenia - 1 |
Greece - 7 | Spain – 9 |
Hungary - 5 | Sweden - 7 |
Ireland - 1 | Switzerland - 2 |
Italy - 17 |
|
The subject rankings list the following number of HEIs:
The institutional ranking includes 183 HEIs.
The EngiRank rankings are modelled on the basis on the Multi-Attribute Value Theory. According to this theory, firstly, it was assumed that it is possible to estimate the value or the total score of each HEI (as well as the disciplines under consideration) by taking into account the impact of the individual criteria as measured by the corresponding indicators. Secondly, it was assumed that if the criteria are not equally important, it is possible to make them comparable by weighting the corresponding indicators with appropriate coefficients. Thirdly, the additivity of the weighted criteria was assumed, which means that the final score is the linear combination of the partial scores. Finally, the indicators with an asymmetric distribution are generally transformed (square root or cube root) to reduce the skewness of the distribution.
The sub-scores for each indicator are calculated in relation to the leading institution. The score of 100 is assigned to the HEI with the highest indicator value, and the scores for the other institutions are calculated as a proportional distance to the leader. The partnership in a European university alliance is a specific binary indicator - a score of 100 is assigned to each of 118 universities listed in the institutional ranking that is a member of any European university alliance.
The sub-scores for all the indicators obtained by the HEI – both in the subject rankings and in the institutional ranking – are summed using appropriate weights. The HEIs are then ranked in descending order according to the weighted sum of the scores. The leading institution is given the final score of 100 and the following institutions are given scores equal to the ratio of their weighted sum of sub-scores to that of the leading institution (as a percentage). The position of HEIs in a ranking is determined by a discrimination threshold of 1 %pt. of the final score. This means that institutions with final scores that differ by less than 1 %pt. occupy the same position in the ranking.
[1] Where any institution has more than 8 accreditations in one discipline, an Accreditations indicator value of 100 was attributed to all institutions with at least 8 accreditations in that discipline.