THE Europe Teaching Rankings 2018: developing continental standards

Browse the full Times Higher Education Europe Teaching Rankings 2018 results

Teaching is one of the most – if not the most – important missions of a university. But it does not get the attention it deserves in global university rankings – in large part because of the lack of available and comparable data across national borders and the difficulty in designing effective metrics to measure teaching excellence.

So when we decided to launch the world’s first Europe Teaching Rankings, following the success of our US and Japan teaching-focused rankings last year (pages 32-33), it was a bold step. But it did not come without its challenges.

The first hurdle was to establish a definition of Europe and then to decide which countries we could include in the pilot. We knew that it would not be realistic to produce a ranking of the roughly 2,000 universities that are based in the more than 30 countries across the continent.

To narrow the focus and ensure that we were measuring comparable institutions, we decided to include only countries that belonged to both the Organisation for Economic Cooperation and Development and the European Higher Education Area and were considered to be a “developed economy” based on the United Nations country classifications. This allowed us to ensure consistency in the standards and calibre of higher education qualifications.

A core element of the Times Higher Education Europe Teaching Rankings is the student engagement survey, which was administered by the market research organisation Streetbees on behalf of THE.

Conducting research in different countries meant that we encountered diverse cultures and survey behaviour, so we needed to reduce systemic bias and monitor and manage the statistical and conceptual validity of the responses.

As a consequence, student survey feasibility became another country selection criterion. This narrowed our focus to countries of western and southern Europe, representing slightly more than 1,000 institutions.

Exploring the type and characteristics of these universities in more detail also highlighted the need for further inclusion criteria at an institutional level. We decided that any institution considered for participation must:

• offer courses and programmes leading to officially recognised higher education degrees, such as a bachelor’s and master’s or equivalent-level degree
• be focused on more than one subject area
• have at least 5,000 students enrolled at a bachelor’s or equivalent level
• receive a minimum of 50 valid student survey responses
• be registered in the Times Higher Education World University Rankings database.

Regarding data sources, we drew on direct institutional data through the THE data collection portal as well as 2014 data from the European Commission’s European Tertiary Education Register.

We modelled the methodology on that used to underpin the Wall Street Journal/THE US College Rankings and the THE Japan University Rankings . As such, we looked at four broad aspects of teaching:

• Resources – the raw material you have to support teaching in your university
• Engagement – what students say about how you’re teaching them
• Outcomes – what your teaching achieves for your students
• Environment – other factors, such as the diversity of your student and staff community, that can have an impact on teaching and learning.
• We conducted a survey of university representatives to examine the key indicators used for evaluating the teaching performance of universities in Europe, to get an understanding of the landscape of available data in each country, and to collect recommendations for potential indicators.

Suggestions included the research orientation of teaching, income from continuous professional development, the share of students employed within 12 months of graduation and student retention rates.

However, we knew that we needed to identify performance indicators that were applicable to higher education institutions across the different national systems and, most importantly, measurable using available data.

The final result was a list of 13 metrics (see pages 28-29) – that range from the number of staff per student (does a university have enough teachers to teach effectively?) to the gender balance of staff and students (does the institution provide a diverse and inclusive environment?) – that aims to show which European universities provide the best learning environment for students. 

Baerbel Eckelmann is lead rankings analyst, Times Higher Education.