How to use external quality assurance information more efficiently

Reports produced by quality assurance agencies can be used in many ways, including in policy-making. As EUA’s Tia Loukkola explains, there is much room for bettering the accessibility and use of information generated by external quality assurance processes across Europe.

In an era of “fake news” and attempts to manipulate information provided to the public, evidence-based policy making is more important than ever. A Eurydice report, mapping the kind of evidence used in formulating educational policies, notes that quality assurance agencies in only a handful of European countries provide input in policy-making.

Nevertheless, reports resulting from external quality assurance activities across the continent surely contain plenty of information about higher education and there has long been an expectation that quality assurance reports are published in order to increase public trust in higher education through transparency. Already the 2005 Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) set out this principle and the need to publish full reports was further clarified in the 2015 ESG.

Why is the information generated by the quality assurance agencies not used more? To explore the answer, it is worth defining the two kinds of information produced by the agencies based on their external quality assurance activities. On the one hand, there is information on individual study programmes or higher education institutions included in external quality assurance reports. On the other hand, there is also information and analysis on wider trends or themes drawn from many individual reports.

A recent EUA-led study identified three aspects to be considered when discussing the information function of external quality assurance reports: accessibility, comparability and usefulness of the information. Colin Tück of EQAR discusses this in his Expert Voice article in further detail, but in terms of usefulness of information it is only fair to acknowledge the finding of the same study that the diversity of external quality assurance approaches and potential target audiences for the reports pose a challenge for agencies and the efficient use of the information in the reports.

There is much potential in the thematic or systematic analyses based on the individual reports. In fact, the ESG 2015 set out an expectation (standard 3.4) for quality assurance agencies to conduct such research. However, an analysis of EQAR Register Committee decisions published in 2018 shows that it is challenging for many agencies. Only setting up complaints and appeals procedures (standard 2.7) comes across as more challenging. The reason given by the EQAR Register Committee in its conclusion is that the standard “has become more demanding, requiring agencies to prepare and publish reports that describe and analyse the general findings of their external QA on a regular basis (ESG 2015), instead of an occasional undertaking (ESG 2005).”

But what does regularly mean? For example, experience shows that quality assurance agencies across Europe regularly review their quality assurance processes at the end of an evaluation, audit or review cycle. It would make sense to tie into this process a review of the results of these activities. It is also reasonable to expect an agency to allocate resources to analyse report information. This is especially true if a higher education system is conducting strategic discussions or a reform of an aspect that the external quality assurance reports cover.

The argument here is not that agencies should change focus or that they should become involved in national policy debates. Instead, they should remain independent in their operations and decision-making, as expected by the ESG standard 3.3. However, widely speaking, there is room to better utilise the wealth of information currently stored in their reports. One way to do this might even be to seek to collaborate with researchers in higher education policy, who can use the reports as a source for research data, as a recent DEQAR example shows.