Assumption and Methodology

The European Rankings of Engineering Programmes EngiRank fills a gap regarding current and trustworthy information on engineering education, as well as research and innovation, in European universities and other higher education institutions (HEIs)1  with strong technical profile.

 

Our primary concern in designing EngiRank was the highest reliability of the rankings. The geographical scope of EngiRank covers the European Union. One of the reasons for this coverage of the rankings is related to the right of EU citizens to study in other member states under the same conditions as nationals; the exercise of this right is additionally supported by the Erasmus+ mobility programmes. Intensified student mobility brings about demands for information on the quality of European HEIs. Furthermore, consortia of institutions from various member states can apply for research and innovation funding to the Horizon Europe programmes and the recently launched European Universities initiative develops long-term cooperation between the European HEIs. These actions level the playing field for HEIs within the European Union and make comparing those institutions more meaningful.

 

Another factor that was considered essential for the EngiRank credibility is the quality and reliability of data – the rankings are based only on trustworthy external databases containing information on European HEIs that is collected in a unified way, such as the bibliographic database Scopus, EPO Worldwide Patent Statistical Database (PATSTAT), information on participations in the European Commission’s initiatives (Community Research and Development Information Service – CORDIS, information on mobilities within Erasmus+ programmes, participations in the European universities alliances), databases of programmes accredited by quality assurance agencies. No information necessary to compile the rankings were obtained directly from HEIs.

 

Then, we decided to reduce diversification of HEIs included in the rankings in order to avoid comparing unmatchable institutions and increase relevance of the rankings.

 

The ENTRY CRITERIA include a qualitative condition:

 

• The institutions considered to be classified in the EngiRank had been checked against the European Engineering Education Database (EEED) run by ENGINEERS EUROPE, a federation of professional engineering associations from 33 European Higher Education Area (EHEA) countries;

 

and quantitative conditions:

 

• The HEIs with a significant share of research output in engineering and technology2  – at least 30% of publications from last 5 years (2018-2022) that are indexed in the Scopus database – were examined. However, the institutions with majority of publications in medical and health sciences or in social sciences between 2018 and 2022 were not considered as HEIs with strong technical profile and they are not listed in EngiRank;
• The threshold numbers of publications in main engineering disciplines from the last 5 full years that are indexed in the Scopus database are as follows:
  – chemical engineering – 200,
  – civil engineering – 100,
  – electrical engineering, electronic engineering, information engineering – 200,
  – environmental engineering – 100,
  – materials engineering – 250,
  – mechanical engineering – 200,
  – medical engineering – 100.

 

The subject ranking in each of the above disciplines includes HEIs satisfying the qualitative condition and both quantitative conditions: share of publications in engineering and technology not less than 30% and a number of publications in a given discipline not less than a threshold value3. The institutional ranking within EngiRank includes HEIs which are classified in at least three subject rankings.

 

Our particular concern in designing EngiRank was the degree of the institutions’ commitment to their economic and social missions. To reflect transfer and application of academic knowledge by HEIs we included the indicators measuring collaboration between academia and industry researchers, use of research output in successful patent applications, as well the very patent activity of HEIs, students internships and contribution to sustainable development goals – where appropriate – into the rankings.

 

EngiRank is a composition of different categories of indicators. We believe that the scale of institution’s activities matters: research and innovation exhibits increasing returns to scale, and the larger the magnitude of HEI’s operations, the more opportunities for students and academic staff. Thus indicators of size, measuring volume of research output, amount of research and innovation funding, number of patents or number of publications assigned to the selected sustainable development goals, play an important role in the rankings. Inclusion of the subject rankings scores into the institutional ranking reflects returns to scope and benefits of interdisciplinarity. Then we have the conventional efficiency indicators expressed in relative terms, such as citations per publication, share of publication in the top 10% journals, number of patent citations received on average by a publication, percentage of publications that are co-authored by industry researchers or foreign researchers. Introducing a dynamic indicator, change in citation impact, is a kind of novelty in the universe of HEIs’ rankings. Last, we included qualitative indicators representing engineering degree programmes accreditations and membership in a European university alliance.

The EngiRank rankings in the following disciplines:

 

• chemical engineering,
• electrical engineering, electronic engineering, information engineering,
• materials engineering,
• mechanical engineering,

 

are based on three criteria: Research (64%), Innovation (26%) and Teaching Quality (10%).

 

Then we addressed the potential and expected contribution of HEIs’ activities in the remaining disciplines to the particular sustainable development goals (SDGs):

 

• civil engineering – SDG 11: Sustainable cities and communities,
• environmental engineering – SDG 6: Clean water and sanitation,
• medical engineering – SDG 3: Good health and well-being.

 

Thus the additional criterion, Contribution to SDGs (5%), was added for each of the above disciplines, and simultaneously the weights of other criteria were cut: Research – to 60%, and Innovation – to 25%.

 

Every indicator in the subject rankings refers strictly to a given discipline. Some indicators in the institutional ranking and in the rankings by subject are based on the same metrics, differing only in the scope of publications or grants considered. Although such indicators from the rankings by subject are indirectly included in the institutional ranking (via the Engineering and Technology Capacity criterion) that shouldn’t be considered a duplication of the indicators. Note that some HEIs are not classified in all the subject rankings (in particular, approximately half of HEIs listed in the institutional ranking are classified in the medical engineering ranking). Besides, even in case of HEIs classified in all the rankings by subject there are publications and grants not covered by the subject rankings indicators, like the ones in general engineering; nuclear energy and engineering; control and systems engineering; safety, risk, reliability and quality; media technology; bioengineering; other miscellaneous engineering. They are counted in the institutional ranking only.

 

RESEARCH (60%/64%)

 

• Publications: number of publications from the years 2018-2022 in the relevant discipline in the Scopus database which are affiliated with the institution. Source: Scopus/SciVal (15%/16%)
• Publications in Top 10% Journals: percentage of the institution’s publications published in the top 10% journals by CiteScore in the relevant discipline in the years 2018-2022. Source: Scopus/SciVal (15%/16%)
• Citations: ratio of the number of citations received by the institution’s publications from the years 2018-2022 in the relevant discipline indexed in the Scopus database to the number of these publications. Source: Scopus/SciVal (15%/16%)
• Change of Impact: dynamic indicator calculated as the ratio of field-weighted citation impact (FWCI) for the institution’s publications from the years 2018-2022 to FWCI for the publications from the years 2013-2017 in the relevant discipline. Source: Scopus/SciVal (15%/16%)

 

CONTRIBUTION TO SDGs (0%/5%)

 

The single indicator criterion. The indicator measures institution’s contribution to selected SDGs in defined disciplines by the number of publications.

 

• SDG 3: Good health and well-being (medical engineering only): number of publications from 2018-2022 assigned simultaneously to the United Nations Sustainable Development Goal 3: Good health and well-being and to medical engineering (FORD category 2.6). Source: Scopus/Scival
• SDG6: Clean water and sanitation (environmental engineering only): number of publications from 2018-2022 assigned simultaneously to the United Nations Sustainable Development Goal 6: Clean water and sanitation and to environmental engineering (FORD category 2.7). Source: Scopus/Scival
• SDG11: Sustainable cities and communities (civil engineering only): number of publications from 2018-2022 assigned simultaneously to the United Nations Sustainable Development Goal 11: Sustainable cities and communities and to civil engineering (FORD category 2.1). Source: Scopus/Scival

 

The indicators’ weights in particular subject rankings are summarised in the table below.

 

INNOVATION (25%/26%)

 

The criterion composed of two indicators. The Scopus bibliographic database was the source for calculating the Academic-Corporate Collaboration indicator. The values for the Research and Innovation Funding are based on information from the CORDIS database.

 

• Academic-Corporate Collaboration: percentage of publications in the relevant discipline published between 2018 and 2022 that are co-authored by researchers affiliated to an institution outside academia. Source: Scopus/SciVal (15%/16%)
• Research and Innovation Funding: value of grants awarded to the university within the EU framework programmes funding research and innovation between 2018 and 2022 to finance research projects in the relevant discipline. Source: CORDIS database (10%)

 

TEACHING QUALITY (10%)

 

The single indicator criterion. The indicator is measured by the number of the degree programmes accredited by ENAEE authorised agencies or by ABET (more information on ENAEE and ABET in the frame below)

 

Accreditations: number of engineering degree programmes related to the relevant discipline accredited by the agency authorized by the European Network for Accreditation of Engineering Education (ENAEE) or by the Accreditation Board for Engineering and Technology (ABET) as of 30.06.2023. Source: databases of EUR-ACE labelled programmes (ENAEE authorised) and of ABET accredited programmes (10%)

 

The subject rankings list the following number of HEIs:

 

• chemical engineering – 167,
• civil engineering – 145,
• electrical engineering, electronic engineering, information engineering – 177,
• environmental engineering – 178,
• materials engineering – 191,
• mechanical engineering – 169,
• medical engineering – 91.

 

The institutional ranking includes 178 HEIs.

 

The EngiRank rankings are modelled on the basis on the Multi-Attribute Value Theory. According to the theory, it was assumed – firstly – that it is possible to estimate the value, or the aggregate score, of each HEI (as well as the disciplines under consideration) taking into account impacts of the individual criteria as measured by corresponding indicators. Secondly, it was assumed that if the criteria are not equally important, then bringing them to comparability is possible by weighting the corresponding indicators with appropriate coefficients. Thirdly, the additivity of weighted criteria was assumed meaning that the final score is the linear combination of partial scores. Moreover, the indicators with an asymmetric distribution are generally subject to the transformation (square root or cube root) to reduce skewness of the distribution.

 

The partial scores for every indicator are calculated with reference to the leading institution. The score of 100 is assigned to the HEI with the highest indicator value and for the other institutions a proportional distance to the leader is calculated. The partnership in a European university alliance is a specific binary indicator corresponding – each of 88 universities listed in the institutional ranking that is a member of any European university alliance is assigned a score of 100.

 

Partial scores for all the indicators obtained by the HEI – both in the subject rankings and in the institutional ranking – are added using appropriate weights. Then the HEIs are ranked according to the weighted sum of scores in the descending order. The leading institution is assigned the final score of 100, and the subsequent institutions receive scores that are equal to the ratio of their weighted sum of partial scores to the one for the leader (in percentage terms). Position of HEIs in a ranking is determined using a discrimination threshold of 1% of the final score. It means that institutions with final scores differing by less than 1% occupy the same position in the ranking.