Our research centre does not just have an enormous amount of subject expertise when it comes to science and mathematics, it also has plenty of novel methodological expertise. This new project by PI Dr Christian Bokhove, funded by the Nuffield Foundation, is an example of this.

From 1 April 2022, the project “Inspecting the inspectorate: new insights into Ofsted inspections” will provide new insights into the reliability and consistency of Ofsted inspections.

Ofsted plays a crucial role in the education system as the school inspectorate. Parents, teachers and policy makers take significant interest in their judgements, the outcomes of which can have serious consequences. However, their consistency across inspectors and schools have been questioned, especially as inspection visits have shortened and teams reduced in size.

Few large-scale quantitative studies have investigated these concerns. Difficulties in assembling the relevant data is one possible explanation for the limited body of research on this important topic, since the structured data published by Ofsted does not contain the detailed information required. To address this research gap, the research team will be using cutting edge text mining and natural language processing techniques to extract the information available within Ofsted’s written inspection reports; thereby, allowing for a better understanding of the consistency and usefulness of Ofsted judgements.

The research will consist of two phases:

• Phase 1 will consider how Ofsted judgements vary between inspectors. It will describe Ofsted inspectors by certain characteristics and background (gender, team size, experience), and determine whether inspectors with certain characteristics are more likely to be assigned to certain schools. Then, this information will be used to explore how inspection outcomes vary across inspectors and by inspector/inspection team characteristics.

• Phase 2 will ask whether inspectors consistently apply Ofsted policy when making judgements. It will focus on a body of previous inspection reports and two Ofsted policy documents: Ofsted inspections – clarification for schools (Ofsted, 2014) and Education inspection framework (Ofsted, 2019). Analysis will apply ‘semantic similarity’ and ‘topic modelling’ techniques to look for differences in the content of inspection reports before and immediately after publication of these key policy documents.

Two academic papers will be published upon completion of the research. These will be accompanied by blogs, videos, press releases and infographics. A public-facing report will be released and a database containing all the information extracted from the inspection reports will be deposited with the UK Data Service. Lastly, two short training courses will be held for academic and government researchers to cover the methods and data resources generated during the project.