Focus on Employability Boosts Universities' Success in the Teaching Excellence Framework

Universities' Teaching Excellence Framework (TEF) grades - designed to help students choose where to study - are being boosted for the institutions that highlight employability post-university and student outcomes in their TEF submission reports, according to a new study published in Educational Review.

The TEF is a measurement of teaching quality at UK universities. In 2017, participating universities received awards of gold, silver or bronze for their teaching excellence, designed to give information to prospective students. Each institution received was based upon a set of quantitative measures, as well as a 15-page 'provider submission' in which they describe teaching quality.

In the first study of its kind, Adam Matthews and Dr Ben Kotzee of the University of Birmingham used corpus linguistic methods to look for the key words and identifiable patterns in these provider submissions.

They discovered that universities using words such as "employment", "employability", "outcomes" and "research", had the largest positive rating outcome, which could impact on whether the university received a gold, silver or bronze TEF award.

"The TEF sits alongside the REF (Research Excellence Framework) as regulatory frameworks to measure quality in two of universities' primary activities: teaching and research. While the REF evaluates research and its impact, the TEF measures discourse about teaching from students and universities themselves. An evaluation of this discourse can tell us about the 'accepted' discourse of teaching excellence as well as the wider discourse around the purpose of higher education."

The research team, at Birmingham's School of Education, scrutinised the language used in the texts of all 228 statements (1,742,438 words in total) submitted by participating institutions to the 2017 TEF using innovative computational methods from the field of corpus linguistics.

Analysis of the words most likely to appear next to "employment" revealed that universities tended to use the same words as those found in the TEF guidelines. This suggests that, in successful written submissions, universities 'mirror' or 'repeat back' the approved language of those guidelines.

In-depth reading of a selection of statements indicated that successful submissions used quantitative evidence to demonstrate the success of initiatives which aimed to improve learning and teaching. Successful submissions also consciously 'bought into' the philosophy behind the TEF and showed their support for the initiative. Criticism of the Universities' Teaching Excellence Framework (TEF) grades - designed to help students choose where to study - are being boosted for the institutions that highlight employability post-university and student outcomes in their TEF submission reports, according to a new study published in Educational Review.

The TEF is a measurement of teaching quality at UK universities. In 2017, participating universities received awards of gold, silver or bronze for their teaching excellence, designed to give information to prospective students. Each institution received was based upon a set of quantitative measures, as well as a 15-page 'provider submission' in which they describe teaching quality.

In the first study of its kind, Adam Matthews and Dr Ben Kotzee of the University of Birmingham used corpus linguistic methods to look for the key words and identifiable patterns in these provider submissions.

They discovered that universities using words such as "employment", "employability", "outcomes" and "research", had the largest positive rating outcome, which could impact on whether the university received a gold, silver or bronze TEF award.

"The TEF sits alongside the REF (Research Excellence Framework) as regulatory frameworks to measure quality in two of universities' primary activities: teaching and research. While the REF evaluates research and its impact, the TEF measures discourse about teaching from students and universities themselves. An evaluation of this discourse can tell us about the 'accepted' discourse of teaching excellence as well as the wider discourse around the purpose of higher education."

The research team, at Birmingham's School of Education, scrutinised the language used in the texts of all 228 statements (1,742,438 words in total) submitted by participating institutions to the 2017 TEF using innovative computational methods from the field of corpus linguistics.

Analysis of the words most likely to appear next to "employment" revealed that universities tended to use the same words as those found in the TEF guidelines. This suggests that, in successful written submissions, universities 'mirror' or 'repeat back' the approved language of those guidelines.

In-depth reading of a selection of statements indicated that successful submissions used quantitative evidence to demonstrate the success of initiatives which aimed to improve learning and teaching. Successful submissions also consciously 'bought into' the philosophy behind the TEF and showed their support for the initiative. Criticism of the TEF as an exercise or questions regarding its accuracy was confined to unsuccessful submissions.

"Recognising what discourse is rewarded in the TEF has important implications for the accepted discourse of teaching excellence in UK higher education," Dr Kotzee added.

"In the TEF, 'teaching excellence' is constructed as students developing their employability and, eventually gaining employment. We therefore expect that, in future, the discourse around 'quality' learning and teaching in the UK will become ever more similar as institutions adopt the 'approved' discourse that sees teaching quality as a metric-driven progress towards greater employment outcomes for students. Should this become the dominant message, it will further drown out discourse around higher education for social good, personal development or equity."

TEF as an exercise or questions regarding its accuracy was confined to unsuccessful submissions.

"Recognising what discourse is rewarded in the TEF has important implications for the accepted discourse of teaching excellence in UK higher education," Dr Kotzee added.

"In the TEF, 'teaching excellence' is constructed as students developing their employability and, eventually gaining employment. We therefore expect that, in future, the discourse around 'quality' learning and teaching in the UK will become ever more similar as institutions adopt the 'approved' discourse that sees teaching quality as a metric-driven progress towards greater employment outcomes for students. Should this become the dominant message, it will further drown out discourse around higher education for social good, personal development or equity."