Edited version of RHS response
- welcome a renewed emphasis on teaching quality, but are concerned that perverse incentives may be created by the focus on proxies that have little connection to actual teaching quality;
- want to see teachers at the heart of any assessment of teaching, shaping and defining good practice;
- believe that teaching cannot and must not be separated from research;
- believe that teaching methods and outcomes differ so much between disciplines that any assessment would have to be done on a discipline-by-discipline basis (which might carry dauntingly high regulatory and economic burdens);
- believe that the dual funding model for research must be protected.
The general principle of a ‘Teaching Excellence Framework’
The Royal Historical Society would like to see genuine incentives for HEIs to concentrate on teaching quality. Therefore, if the government introduces a TEF it should be one that encourages institutions to:
- create a culture in which pedagogy is valued, developed, and discussed among all members of academic staff, and quality teaching rewarded
- create programmes that are consciously designed to develop in students particular sets of intellectual (and, where appropriate, practical) skills
- recognise and develop the intrinsic relationship between research and teaching
We believe that to achieve these goals, the TEF must:
- recognise and value the whole educational experience of students – i.e. learning as well as teaching
- be based on programme-level (discipline specific) submissions
- be peer reviewed by respected academics in the same discipline, in much the same way as happens with the REF
- require submissions that will be largely narrative, rather than metrics-based
We can see the value in aiding student choice of appropriately benchmarked metrics, but we do not think they could ever be an effective proxy for teaching quality and doubt that they could form the basis of reliable comparison across institutions or disciplines.
It is one thing to seek to make information about teaching even more available and transparent to potential students and quite another to impose a target-led regulatory regime on HEIs, with all the well-known problems that will engender. As the government withdraws from direct public subsidy of teaching it must recognise that its regulatory role necessarily changes too. While the government can seek to give more leverage to students, since it is they who are now ultimately paying, it has ceded any direct leverage it may theoretically have had over how HEIs deliver teaching. Since the removal of the recruitment cap on undergraduate numbers, universities have noticeably been focusing more on teaching quality than was previously the case. The test of these proposals is whether it will enhance that tendency or retard it.
The Royal Historical Society’s case is that the only meaningful assessment of teaching quality is one that is rooted in the discipline-specific experience and judgement of those who participate in teaching and learning – the teachers as well as the students.
As the Green Paper concedes, it is palpably not up to the government to define what makes for effective teaching at HE level. It must therefore be up to teachers to determine how best to teach in their discipline at this level, within the resource constraints we face.
We are confident – on the basis of evidence – that a good history programme prepares students very effectively for the workplace, but we are certain that effective learning and teaching in History is not measurable by generic cross-disciplinary standards. Typically, humanities programmes will feature fewer contact hours than most science programmes, but will also feature much more small group teaching, a far greater emphasis on independent learning, and more concentration on the development of students’ writing and communication skills. There will be very different kinds of links with outside institutions and the nature of ‘employability’ will necessarily be different in a subject like ours than in some other disciplines. The QAA has already recognised this in establishing discipline-specific panels in order to draw up disciplinary benchmark standards. The TEF, if it is to have credibility, must do the same.
The RHS takes a lead in shaping and defining what effective teaching and learning means in our discipline. We therefore welcome the suggestion in the Green Paper that Learned Societies are among the ‘stakeholders’ with whom institutions would want to work. The RHS is already well positioned to play this role, being able to draw on a depth and range of expertise from many different institutions.
Metrics and how a TEF would work
We are pleased that the Green Paper acknowledges the difficulties of using metrics. We believe that the conclusions of Prof James Wilsdon’s review of the use of metrics in research assessment for HEFCE, The Metric Tide, apply with even greater force to teaching and learning.
By far the most problematic metric mentioned in the Green Paper is the destination of graduates. The difficulty here is that employment status and earnings are not related in any demonstrable or tangible way to teaching quality but reflect other factors including social class, the perceived status of the university attended, and secondary school type, as well as career choice. It may be of some value to students to have this information, but it should not be confused with a mechanism designed to drive up teaching quality.
In addition, the use of earnings/employment data, if they do not control for social origin, may have the perverse effect of incentivising socially selective admissions. We are sure you agree that government policy should not end up, however unintentionally, encouraging university admissions officers to have to weigh up the likely earning potential of applicants, something which would be dishearteningly easy for them to do using school and postcode data. Even where it is possible to control for some of these background factors, the demonstrable tendency of employers to use institutional reputation as a ‘screening’ mechanism in employment decisions suggests that a TEF based in any substantial degree on graduate earnings may only have the effect of fossilizing established hierarchies rather than inciting innovation and improvement.
Retention figures are of relevance but only so long as they are properly benchmarked against the background of the student body, since otherwise this would be a disincentive for institutions to recruit students from under-represented groups, including part-time and mature students, for whom the funding situation has already been deteriorating markedly in recent years.
Student satisfaction data is potentially of value, but again only so long as it is properly benchmarked. But, to state the obvious, a measure of student satisfaction is not a measure of teaching quality. Learning should be difficult and should require commitment on the part of the learner, and rigorous assessment means that by definition not everyone will succeed. These self-evident observations severely limit the ability of ‘satisfaction’ measures to capture teaching quality.
Of the metrics mentioned for later implementation we would strongly welcome a measure of the proportion of staff on permanent contracts. We would also urge that universities be incentivised to embed the relationship between research and teaching by being penalised for employing staff on teaching-only contracts. The distinctiveness of HE, and one of UK universities’ internationally recognised great strengths, is that students are taught by people who are also actively engaged in research. In our discipline (and no doubt in others) effective teaching cannot be divorced from research: we want to develop students who are engaged in research themselves and who are exposed to the people who are immersed in scholarship.
‘Teaching intensity’ is not clearly defined in the Green Paper, but in our discipline we believe it should mean levels of student engagement (i.e. evidence that students are actively participating in learning), insofar as that can be reduced to a metric. We are convinced of the value in our discipline of small group teaching and/or low staff/student ratios as a means to this end and believe that this is vastly more important than the number of contact hours. In our experience, drawn from across the spectrum of institutions teaching History, students consistently appreciate the individual attention and higher quality feedback that is only possible in a small-group setting.
On the whole, however, we suspect that the idea that metrics which may not be ‘robust’ can or should be ‘balanced’ with ‘institutional evidence’ is likely to lead to greater managerial attention to the former (which can be automated and rendered generic) than the latter (which relies on costly subject-specific and qualitative measures). Like other performance indicators this runs the risk of transferring institutions’ efforts from performance to indicator.
The precedent of the REF, where many of the distortions to academic effort have come from HEIs’ (mis)interpretation of the rules, is ominous. On this precedent one might expect university managers to be unnecessarily and distractingly interventionist not in ways that encourage diversity and experiment in teaching but rather in ways that are thought to mirror government ‘intentions’. It would be one of the most depressing unintended consequences of the TEF, albeit one very easy to imagine, if the regulatory constraints under which university teachers worked, stifling innovation and creativity, outweighed putative benefits in enhancing teaching quality.
Therefore, the TEF should not, indeed in our view cannot, impose measures that, however subtly or unintentionally, have the effect of directly engineering how teaching and learning happens in universities, whether that is through crude measures like ‘contact hours’ or anything else.
Social Mobility and Widening Participation in Higher Education
The Royal Historical Society is strongly supportive of efforts to ensure that class and ethnicity are not barriers to a good historical education with all the benefits that brings. Unless Access Agreements – together with evidence of progress toward achieving the goals set out in them — are a prerequisite for a TEF award, it is difficult to see what incentive HEIs would have for maintaining or extending the numbers of students from under-represented groups. This is especially true since in some respects the TEF, as outlined in this Green Paper, would otherwise incentivise universities to become more socially selective in order, for example, to ensure their graduates have higher earning potential.
There are few more important issues for universities, or the education system and society more widely, than ensuring fair access. Therefore we believe the government should use all available policy levers to ensure that universities make strenuous efforts in this area. However, we do not support externally imposed institution-level admissions targets for under-represented groups.
The challenge for HEIs and for us as a Learned Society is to work with schools and other organisations to increase participation and academic success among under-represented groups at all stages of the education system.
The challenge for government is to discriminate between those levers that can be pulled at the point of admission to university and those that can’t. It would be a mistake to place more policy emphasis on one specific stage than it can bear, and which would therefore be likely to incentivise game playing and produce perverse effects.
As the Green Paper points out, ‘prior educational attainment is the key factor in determining progression’. Disadvantage starts in infancy and deepens (unevenly) at different stages of the life course. Government policy on social mobility needs to link up communities, child-care, child support, schools, universities, careers, equal-employment policies and income inequality. At the point of entry to HE, a successful widening participation policy must give universities the ability to identify applicants who have demonstrated their intellectual ability while overcoming measurable disadvantages. Properly contextualised decisions about academic potential can only be made on the basis of as much data, about social class, ethnicity and school as possible. Therefore we support greater access to the kind of information currently held by UCAS.
A student-led market in HE
There is an important component largely missing from the Green Paper’s discussion of market exit: how to facilitate students transferring, during their degree programme, from one institution to another. Although there is now a rudimentary system of credit transfers that in theory allows students to move from one HEI to another without losing credit, in practice institutions often place barriers in the way. Incentivising universities to facilitate the transfer of students from a programme in one institution to a programme in another would create more fluidity in the market.
Naturally, it would also require an HE system that had robust mechanisms for ensuring peer-reviewed validation of programmes and the reinforcing of broad comparability among programmes through the external examining process.
Regulation, Research Funding and the ‘architecture’ of Higher Education
The higher education architecture is unnecessarily complex, but this is in large part because as government withdraws from direct administration it leaves behind complex regulatory frameworks. SLC, OFFA, QAA, HEA and HESA are all relatively recent products of this regulatory policy. The Office for Students (OfS) would be yet another one.
Not all of HEFCE’s current functions are appropriate for an OfS. Hiving off HEFCE’s research functions will, according to the Nurse Review, require a further complication of the research support structure. In addition to assessment and allocation of QR, HEFCE provides essential core funding to research institutes (such as the Institute of Historical Research, as part of the School of Advanced Study of the University of London, which provides crucial infrastructural support for our discipline). These research functions – not provided for in the rationale for government intervention in HE on p. 58 of the Green Paper– are crucial and not easily separable from ‘education’. For example, it is not clear from the Green Paper proposals where responsibility for PGT and PGR students will lie.
We would favour retention and indeed strengthening of a single higher-education regulator, such as the funding councils provide, with if anything transfer to it of some of the functions of the other, smaller quangos targeted for supersession.
Although little teaching grant remains for humanities disciplines such as ours, we are concerned about the proposal to allocate teaching grant from within BIS to ‘enable ministers to strengthen incentives for higher education provision that supports the needs of the economy’. This seems to suggest a degree of ‘manpower planning’ which governments have largely abjured since the 1960s and which is inconsistent with an HE system centred on students and student demand. It points to features of the Australian system, including differential fees and subject quotas, which have been heavily criticised by students and employers for unbalancing the normal functioning of supply and demand mechanisms.
Independence of Research and Academic Freedom
Changes to the institutional framework are presented as reducing complexity. But we are concerned that they are also likely to reduce the independence of academic judgement in teaching and research that has been the recipe for international success of the UK higher education system for decades. We are pleased to see an acknowledgement in the Green Paper of the Haldane Principle, but we note that successive governments have redefined the Haldane Principle when it suits them. Clause 68(3) of the Further and Higher Education Act 1992, still in force, provides that ‘Such terms and conditions may not be framed by reference to particular courses of study or programmes of research (including the contents of such courses or programmes and the manner in which they are taught, supervised or assessed) or to the criteria for the selection and appointment of academic staff and for the admission of students’. This, in fact, provides for a stronger protection of academic freedom and independence than does the Haldane Principle, as observed as recently as 2010 by BIS in its Strategic Plan. But it would be weakened by the Green Paper’s proposal to limit protection to ensure that ‘ministers and officials could not single out specific institutions’. We note that the Green Paper acknowledges the importance of ‘research which is directed within institutions’ (p. 70), also protected by the 1992 Act, but it does not extend the same significance to teaching.
We would expect to see the independence of arm’s length bodies safeguarded in legislation at least as strongly as at present, with stronger protections for what the Green Paper calls ‘research directed from within institutions’.
The dual system of research funding
The two streams of the dual funding system have different purposes and different statutory status. By redefining the Haldane Principle, successive governments have increased their ability to influence decisions made on the RC arm. At the same time the balance between the budgets of the two elements has shifted in favour of RC funding (for example, in the 1992 ‘dual support transfer’ and again in 1998 in order to fund the Arts and Humanities Research Council – in both cases from the QR to the RC stream). To protect both the value and the independence of QR funding, some longer-term guarantee ought to be provided fixing the balance between these two budgets, as indeed is recommended by the Nurse Review.
QR is essential to the independence of individual academics and their employing institutions. Without it, student fees would presumably have to be used to cross-subsidise research infrastructure, which would be difficult to justify, not least to students. Above all, QR ensures, as the Green Paper puts it, that research ‘may be purely curiosity-driven, respond to individual partnering opportunities, or may be in rapid reaction to advances in a given field’. These qualities are more important to the humanities than to the sciences, as our research does not cluster so readily or fruitfully around easily identifiable research frontiers or immediate applications. It is for this reason that, rightly, 85% of government research funding in the humanities is provided by QR and only 15% by RCUK. As a means of distributing QR, REF has its critics, yet so far no one has come up an alternative that would be credible and consistent while also identifying and rewarding quality in all its many manifestations. In fields such as ours where quality is very widely distributed across a range of institutions, it also enables individual achievers to gain recognition independent of the reputation of their institutions and also to small pots of funding that have disproportionately significant effects in stimulating creative research across the system. This plurality is a sign of health in UK HE that is rightly envied abroad.
If REF is to measure research quality wherever it is, we need to find ways to reduce the extensive (and expensive) game-playing in which institutions engage, modelling the exercise many times in advance and erecting elaborate selection processes that divide the academic community and distract from the research enterprise itself. One option would be to require the submission of all research-contracted staff, but this requirement suffers from the fatal flaw that it would encourage the proliferation of teaching-only contracts and the bifurcation of HE into research and teaching streams, which, as the Green Paper indicates, would be to the detriment of both.