Education Policy

Royal Historical Society responds to TEF Review

The Royal Historical Society recently submitted a response to Dame Shirley Pearce’s independent review of the Teaching Excellence and Student Outcomes Framework (TEF). Our full response is now available in full on our website here. A post with more on the TEF Review, and a short summary of the main points in our document, is on our blog. 

 

New School History Curriculum Briefing Pack

On 27 March 2018, the Society hosted a day-long event on ‘The New School History Curriculum and the Transition to Higher Education’. This conference formed part of the 150th anniversary celebrations of the Royal Historical Society, and it was organised by the Society in conjunction with the Historical Association.

Recent changes to the specifications and curricula available at both GCSE and A Level have resulted in significant alterations to the historical education that students receive in English schools from years 9 to 13 (with many schools starting GCSE tuition three years before the examinations). With the first students taking the new GCSE in summer 2018, and the first cohort through the new linear A Levels having arrived at university in autumn 2017, it was a good moment in March 2018 to take stock of these developments and their implications foir the teaching of history in Higher Education.

The conference brought together over 80 delegates representing various key constituencies: representatives from all the Examination Boards in England and Wales (the Scottish SQA had to withdraw at the last moment), HEI heads of department and admission tutors, secondary teachers and members of the museum and archives sector, plus officers and councillors from the Society and the HA.

It proved to be a very stimulating day, both in terms of the presentations and discussion, and also the contacts many of us made with experts in different constituencies, and we agreed to make the extensive briefing pack available online.

The pack contains 10 items. (1) is the programme of events and speakers; (2) – (4) gives the essential background: DfE and Ofqual 2014 Subject Requirements for A and AS Levels and GCSE plus Assessment Objectives. (5) – (8) list the syllabi of the examination boards for England and Wales, including the WJEC’s new GCSE for schools in England – edquas. (9) summarises planned changes to Higher History in Scotland. Finally, we include (10) the 2017 HA’s survey of History in schools, a valuable analysis based on responses from 287 schools. We are grateful to the various authors of these briefings and reports for permission to upload them.

Prof. Ken Fincham
Vice-President (Education)

1. Event Programme and Biographies of Speakers

2. Ofqual GCE (A and AS Level) History Subject Requirements 2014

3. DfE GCSE History Subject Requirements 2014

4. Ofqual GCSE History Assessment Objectives 2014

5. AQA syllabus at A and AS Levels and GCSE and sample questions

6. OCR syllabus at A Level, and the relative popularity of modules; and GCSE syllabus

7. Pearson syllabus at A Level and GCSE

8. WJEC A Level and GCSE syllabus and eduqas (WJEC in England) GCSE for schools in England

9. SQA planned changes to Higher History from 2018-19

10. HA Survey of History in Schools in England 2017

 

Subject-level TEF Consultation

The Department for Education has invited responses to its technical consultation to inform the development of the Teaching Excellence and Student Outcomes Framework (TEF) at subject level. The Society has submitted a response to the consultation that you can read here.

 

RHS sponsors Historical Association Quality Mark

In recognition of the excellence and high standards that the Historical Associaton’s Quality Mark (QM) can bring to a school we are delighted to announce that to help celebrate our 150th anniversary, the Royal Historical Society is to provide sponsored bursaries to ensure more schools can take part. The QM supports the development of excellent history provision by the teacher, the department and as part of the whole school offer to young people. You can read more about the scheme here.

The Royal Historical Society will be provide sponsored bursaries in 2018 for up to ten secondary schools to participate in all stages of the Quality Mark programme. Conditions of the sponsorship mean that the sponsored schools will be state funded, non-selective schools drawn from across the country. To register your interest in the the Royal Historical Society Quality Mark Bursaries, please contact Mel Jones (melanie.jones[AT]history.org.uk) for more details, or download and complete the application form from the HA website. Applications close on July 16 2018.

 

HEFCE Recruitment for TEF Panel Members

HEFCE has opened recruitment for approximately 100 panel member roles on the TEF Subject Pilot panels and a small number of roles on TEF Year Three for applicants with HE in further education colleges and/or alternative providers.  There are roles for students, academics, widening participation experts, employment experts and employer and PSRB representatives to review submissions and decide on the assessment outcomes.

We would encourage you to review the available roles to see if you or a colleague may be interested. We would also be grateful if you could disseminate details to colleagues across your provider.

The role specifications and applications forms for the available roles can be found on the TEF recruitment portal. Please note that the deadline for receipt of applications for all roles is midday on Friday 6 October.

Should you have any questions regarding the roles, please contact TEF@hefce.ac.uk.

 

Education Policy Committee Overview

Ken Fincham is Chair of the Education Policy Committee. He writes:

The RHS is committed to monitoring and supporting the teaching of history in higher education and secondary schools. The Educational Policy Committee was founded in 2003 to bring together expertise in and outside Council in order to promote history teaching through offering expert advice to government and NGOs as well as sponsoring conferences and teaching-related initiatives. Accordingly, on the Committee sit not just elected councillors with a particular interest in teaching but also a number of co-opted members from History UK (representing History in HE), the Historical Association and secondary schools, and they bring to our discussions a wealth of experience and insights. Among our co-opted members are the former Chief HMI for History at Ofsted, the former discipline lead for History at the Higher Education Academy, and a current academy-school teacher and editor of Teaching History. Regular Committee meetings are supplemented by biannual visits to history departments where we see history teaching on the ground, sometimes in rather challenging circumstances, which helps inform our discussions and shape our priorities.

History in HEIs

The Committee takes a broad purview of history at university by examining trends in admissions, curriculum design and delivery for both undergraduates and taught postgraduates. We are interested in history teaching in all its various settings, whether in a single-subject department, a multi-disciplinary humanities or humanities and social science unit, or historians teaching in other departments such as languages. Representatives of the RHS speak at teaching events and sponsor a number of workshops and conferences, most recently the ‘New to History teaching’, a one-day event in September 2017 at the IHR. We often provide expert advice to public bodies: in 2013-14 we contributed to the revision of the History Benchmarking document produced by the Quality Assurance Agency, and in 2017 gave evidence to the British Academy’s flagship project on skills in Arts, Humanities and Social Sciences. Currently the most pressing matter is the rapid development of the Teaching Excellent Framework. In January 2016 the Society submitted its response to the Government Green Paper on HE, ‘Fulfilling our Potential: Teaching Excellence, Social Mobility and Student Choice’, which was endorsed by the Economic History Society, History UK, the British Agricultural History Society, and the East Midlands Centre for History Teaching and Learning. Since then, the Society has maintained a watching brief over the rapid evolution of the TEF, and will in due course take up the invitation to offer views on the pilot subject-level metrics in 2017-19 and to the Independent Review of TEF in 2018-19. Another important role of the Society is to provide guidelines and resources for the fellowship and the wider discipline. We see it as a priority in the next few years to consider the implications of the digital revolution as it affects the teaching of history, and the evidence for better and best teaching practice across UK, and to share our thoughts and findings through a teaching resource portal on our website.

History in Schools

The RHS plays an important role in discussions about teaching in history, working in partnership with the Historical Association. In 2014-15 we took a leading role in discussions on reshaping history teaching from the National Curriculum via GCSE to A level. We are committed to ensuring that history in schools offers the best possible preparation for further study of the subject while also providing a rounded and fulfilling historical education for those who do not continue to study history at a higher level. Over the past decade we have built up excellent relations to the School Examination Boards, offering them advice and support as required, and convening a meeting most years to facilitate the exchange of views between their representatives as well as to asess trends in uptake, choice within specifications and performance. The recent revision of GCSE and A Level make these meetings particularly timely and informative.

The Move from School to University

Given the remit of the Society, and our close working relationship with the Historical Association, we are well-placed to help bridge the gap between history at school and university. We welcome, and are actively supporting, the Historical Association’s recent initiatives to forge closer ties with HEIs. We keep a close eye over the provision of teacher training places, and intervened in 2015-16 to protest to government over the threatened reduction in their number. A conference is being planned for March 2018, under the auspices of Professor Arthur Burns, Chair of the Committee in 2013-16, to bring together teachers of history in schools and universities with represenmtatives of the Examination Boards to review the consequences of recent curriculum changes and to smooth the transition from A Level to first year university history.

Kenneth Fincham is Professor of Early Modern History at the University of Kent. A former RHS Hon. Secretary and then Councillor, he is an experienced academic editor and has a long-standing interest in curriculum and in building bridges between secondary and tertiary teaching. In the 1980s he taught for two years in a secondary school, and since then has been involved in ‘A’ Level history as an examiner, setter and now reviser.

 

Response to Stern Review of the REF

Lord Stern’s review of the Research Excellence Framework (REF) issued a call for evidence in January 2016 (read the full call here). The review will have important implications for the scholarly community and for history as a discipline, dealing with the mechanisms for allocating QR (quality-related research funding) and the shape of future REF exercises.

The Royal Historical Society has provided a robust and thoughtful response to Stern’s call, challenging the notion that metrics can be used to measure research quality in the arts and humanities and pointing to the consequences of the present system for our discipline, whilst reflecting on the positive changes to research culture engendered by the REF. Read the Society’s full response below, or download a PDF version.

 

Response to Stern Review of the REF

 

  • What changes to existing processes could more efficiently or more accurately assess the outputs, impacts and contexts of research in order to allocate QR? Should the definition of impact be broadened or refined? Is there scope for more or different use of metrics in any areas?

It is essential that an exercise such as the REF commands wide support within the academic community and that its conclusions are respected.  This is currently a clear strength and would be compromised by wider use of metrics, which simply do not work across the board. A main finding of The Metric Tide is that, in contrast to peer review, academics are sceptical of metrics, which are particularly problematic when assessing outputs in the Arts and Humanities.  In terms of historical scholarship, there are no current measures which provide reliable data, and this is not likely to change given the broad range of types of publications in which scholars publish quality research, including book chapters, websites, and datasets.  History has no established rank order of periodicals and impact factors—as in Humanities more generally—mean very little e.g. http://guides.temple.edu/c.php?g=78121&p=509794

There are two additional difficulties.  The first is that, for historians, books are of primary importance in disseminating research.  This was demonstrated in REF2014 where ‘books and parts of books’ were most likely to receive scores of 4*. There is no way of evaluating this type of output other than through peer review.  In a discipline where so many outputs are submitted in book form, either as monographs or as chapters in edited volumes, metrics thus pose a particular problem.  Second, the download half-life of journal articles in History—and Humanities articles more generally—is very much longer than it is for the Sciences.  This is insufficiently recognized. The point is made in the British Academy report on Open Access which nevertheless severely underestimated this half-life as the report did not include downloads from heavily used archive sites such as JSTOR.  The RHS estimates that the true download half-life of a History article is at least 12 years.

The RHS would therefore argue strongly that the quality of scholarship in History, as in Humanities more generally, is not quantifiable by metrics and its full value and impact become apparent over a significantly longer term than a REF cycle.  Greater use of metrics in place of peer review would not only fail to capture the nature and quality of world-leading scholarship but is also likely to have a distorting effect on the methods by which historical scholarship is disseminated.  As peer review offers the flexibility to assess research in new, minority or unfashionable fields, any downgrading, or substitution by metrics, is also likely to distort subject matter by encouraging publications on well-worn or voguish topics.

The RHS thus remains strongly committed to peer review, which is widely and routinely used to assess research quality in, for example, employment, promotion and publication decisions.  It is, in fact, the only expert device we have to assess quality.  After consulting REF2014 panel members, the RHS is confident that the workload in terms of the peer review of outputs was manageable, and the process very conscientiously carried out.  Panel membership attracts outstanding academics who benefit from the opportunity to survey the field and in whom their peers have confidence. Maintaining this calibre, and this level of participation, is essential to the REF process.

The assessment of impact was new to REF2014 and here some doubt has been expressed, both over the volume of material to review and the fact that inevitably peer reviewers had less experience of evaluating impact.  Academic assessors are trained to assess intellectual quality rather than impact. There was also less time and information available for review.

This issue is likely to diminish as the accumulated experience of impact grows within the research community.  The RHS moreover believes that impact has benefitted the historical profession as it has underlined the deep public interest in History and the relevance of our research to various fields, including education, digitization, and policy.  Although the definition of impact for REF2014 excluded the kind of broad expertise of a historical field that is evident in much public engagement work—and which should be reflected in how the underpinning research is defined and understood—it is valued by many.

However, research undertaken by the RHS demonstrates that, in terms of authorship, ICSs, are not representative of the wider research community.  The ‘impact case study’ is an artificially constructed exercise, but the fact that 75% of identified PIs were men and just under 65% of PIs were professors is of real concern.  One simple way of making ICSs more representative of the historical profession would be to make impact portable.  There is an obvious logical inconsistency in having outputs transferable and impact not as both rest on underlying research usually undertaken over a number of years.  The RHS is clear that it is highly discriminatory against ECRs not to allow them to transfer impact from one institution to another, or to include that based on unpublished research in a PhD thesis. This makes it almost inevitable that institutions will rely on case studies contributed by people at mature stages in their careers.

A further consideration is how the requirement that departments submit one impact case study plus one other for up to 10 researchers has affected very small research clusters, for example, in universities where a department or school might only have 2 or 3 historians.  The RHS is concerned that various REF measures put this kind of unit at risk (see the remarks on environment below).

 

 

  • If REF is mainly a tool to allocate QR at institutional level, what is the benefit of organising an exercise over as many Units of Assessment as in REF 2014, or in having returns linking outputs to particular investigators? Would there be advantages in reporting on some dimensions of the REF (e.g. impact and/or environment) at a more aggregate or institutional level?

The benefits of organizing returns by unit of assessment are most apparent in the evaluation of outputs, where subject-specific specialists are clearly best placed to conduct peer reviews. History is represented within the great majority of universities as well as in other cultural institutions.  Large disciplines need their own UoA; the volume of outputs is substantial and the variety of expertise already contained within the discipline is broad.

In History, as more generally, REF owes its credibility as an assessment exercise to its expert review panels.  It is clear from our consultations that the History panel worked very effectively, with a shared understanding of criteria and quality.  In contrast, colleagues on panels that covered a range of disciplines found the task of assessment could be more difficult and even conflictive.  Departments within these broader panels, for example Languages, experienced more uncertainty preparing for REF.  REF ‘scores’ based on amalgamated disciplines may also be misleading in terms of individual departments or schools and we would certainly resist History’s incorporation into a wider UoA.  The international strength of historical research in the UK is reflected in the large number of high performing units, which has been confirmed in all previous REF exercises. We believe it is important to showcase this; any move to amalgamation would occlude the proportion of world-leading historical research for which British universities are responsible.

It is hard, if not impossible, to see how outputs could not be linked to individual researchers in History.   This means that, while allocated scores—at least in terms of QR—go to institutions—and so, in a sense, it is the headline institutional score that matters—it is not clear how this could be obtained without expert peer review, which has to take place at the level of the individual outputs or ICS.  There is also a further point, in that the granular detail of REF feeds into, for example, university guides, admissions league tables, and wider research rankings and here the disciplinary picture is crucial.  This a particular concern for small, strong units within less-research intensive universities; these are not uncommon in History.

In broad terms, the RHS believes that the current arrangement for outputs (four, with differentially weighted monographs), impact and environment is manageable and effective with outputs as the main weighting. There is some feeling that environment should not weigh more heavily in the process.  While, in a ‘bundling’ category such as environment, some form of metric evaluation is conceivable—research income and PGR numbers are two of the very few measures that can be aggregated across all subjects and both relate to environment—we see real difficulties with evaluating environment simply by metrics.  REF is designed to recognize and support essential research activities, including a rich academic culture represented by seminars, workshops, conferences etc, participation and leadership in learned societies, editorial work, peer review and collaboration across institutions.  Not to assess these vital academic functions would be to undermine them.

 

Wider use of metrics would also raise real issues of equity even at UoA level.  It is clear from the RHS’s analysis of REF2014 that research income and PGR numbers were crucial to success in terms of research environment. Every university in the top 22, bar one, graduated at least 1 PhD per FTE over the REF cycle and 10 more than 1.5; the best predictor of rank in research environment was the number of PhDs per staff FTE between 2008 and 2013.  Given the concentration of AHRC funding for doctoral study in a small number of consortia, in which Russell Group institutions predominate, this makes it almost impossible for small units in less-research intensive universities to do well in terms of environment no matter how strong their collective research endeavours.  The RHS views this with real concern.

 

 

  • What use is made of the information gathered through REF in decision making and strategic planning in your organisation? What information could be more useful? Does REF information duplicate or take priority over other management information?

This is not primarily a question for representative bodies such as learned societies.  Indeed, as REF information is provided at aggregate levels, and only every seven years or so, it is hard to see it as a significant source of management information.

The RHS believes that there is a useful purpose to having research, particularly research outputs, evaluated by independent external assessors.  However, if, as we believe and as is set out in question 2, REF is a tool to allocate QR then it should be used for this purpose.   It should not be for government to suggest or draw on other uses of REF by individual universities and the RHS would resist any move to embed REF as a performance management tool.

 

  • What data should REF collect to be of greater support to Government and research funders in driving research excellence and productivity?

The RHS is sceptical as to whether data in and of itself can be used to drive research excellence.  We are strongly committed to research excellence, and to research publication in all its various guises, but see academic freedom, research time, the availability of funding and an atmosphere of creativity and communication as being far more pertinent to those working in Humanities as guarantees of research excellence and productivity. However, there are areas where data gathering as part of REF would usefully highlight areas of concern, for example historical research conducted in other languages and linguistic skills that need developing and/or support.

It is also the case that the suggestion in the recent Green Paper that measures of casualisation might serve as a measure of teaching quality could equally be applied to REF.  The close link between teaching and research is one of the great strengths of the British university system. This is reflected in the high proportion of staff on full academic contracts, which should be protected.  Collecting such data would also provide a clear picture of the employment position and foreground an issue that is of particular importance to ECRs and the future shape of the profession.

 

  • How might the REF be further refined or used by Government to incentivise constructive and creative behaviours such as promoting interdisciplinary research, collaboration between universities, and/or collaboration between universities and other public or private sector bodies?

This is in some ways a curious question: how can REF be used in support collaboration between universities when by its very nature it makes them compete?  However, the RHS is confident that collaborative mechanisms—which are often informal—are strong and that historians collaborate with as much or more vigour as ever.  There are also examples of strong regional partnerships that encourage and fund research and doctoral studentship collaborations, for example the White Rose.  These are, however, not directly related to REF.

Interdisciplinary research is stimulating and valuable but it is also important to defend multi-disciplinary collaborations and the single-discipline scholarship on which these rest.  Promoting interdisciplinary research is not in itself a guarantee of improvements in research quality and the RHS would encourage the review to address this directly.

In terms of REF, impact has clearly encouraged collaborations with other private and public sector bodies and there may be scope to adjust output criteria to acknowledge this.  The arrangements for cross-referencing interdisciplinary work to other panels and interdisciplinary specialists appear to have worked effectively, another strength of the flexibility provided by peer review.

 

  • In your view how does the REF process influence, positively or negatively, the choices of individual researchers and / or higher education institutions? What are the reasons for this and what are the effects? How do such effects of the REF compare with effects of other drivers in the system (e.g. success for individuals in international career markets, or for universities in global rankings)? What suggestions would you have to restrict gaming the system?

There is widespread concern over the potential for departments to score highly in REF by only submitting a small proportion of their staff.  There is a division of opinion among historians as to whether this should be addressed through a 100% return of eligible staff, given that a possible response would be to increase the use of teaching contracts and so alter the proportion of staff employed on full academic contracts (see the response to Qu. 4 above).

However, there is a clear desire to prevent—or discourage—institutions from gaming the system.  Not only can this have detrimental consequences for those omitted for strategic reasons but it can also lead to bad management decisions, as the 2/3 borderline is by far the most difficult to predict in internal assessment exercises. The RHS would therefore strongly favour restoring the previous system of requiring UoAs to identify the proportion of staff submitted.

It is hard to see how the other identified drivers affect historians or, indeed, most individual academics.  Only a handful of scholars operate strategically in ‘international career markets’ while ‘global rankings’ are of little relevance below institutional level and then only to a select group of universities.

 

  • In your view how does the REF process influence the development of academic disciplines or impact upon other areas of scholarly activity relative to other factors?  What changes would create or sustain positive influences in the future?

There is no doubt that REF has affected scholarly behaviour in History and that it will continue to do so.  The introduction of impact, discussed above, has encouraged the academic community to be more outward facing and has also diversified the understanding of academic merit. Promotion criteria, for example, now commonly reflect the importance of external engagement.  Less happily, the pressure for outputs has downgraded the status of the book in several disciplines—especially, but not exclusively, in the social sciences—where such publications used to be common.  The differential weighting of monographs has prevented this in History and other Humanities subjects, and the RHS sees this as essential both to prevent distorting the research process and to reflect the research and scholarship that goes into producing such a substantial piece of work.

There is some suggestion that triple or quadruple weighting of monographs, and other very substantial outputs, would be beneficial, removing what may be seen as a perverse incentive to produce lower-quality article-length outputs to meet the output requirement.  A single-authored 80-100,000-word monograph—the norm in our discipline—represents greater productivity than that required in other fields where team based research is the standard mode.  Against this is the point that book authors seldom write nothing else and the greater complication it would give to panel deliberations.  The RHS notes that the panel accepted 97% of double-weighting requests for REF2014, but this might change if the weighting rises.

One final concern is that REF pressure has downgraded the value of producing synthetic books and articles, which have an interpretative function and are primarily used for teaching purposes, including postgraduate teaching and training.  Though pedagogically important, these are unlikely to be included in TEF and so run a real risk of falling between two stools.  It would be helpful to have the value of these outputs recognised.

 

  • How can the REF better address the future plans of institutions and how they will utilise QR funding obtained through the exercise?

This is not a question that a learned society is well suited to answer but we are quite sure that our Fellows would resist adding bureaucratic requirements rather than taking them away.  There is also a concern that the ‘strategy’ sections of the environment and impact statements were the least valuable parts of all the submissions: they are brief, rhetorical, and impossible for the panel to check.

 

  • Are there additional issues you would like to bring to the attention of the Review?

The changes to the last REF were very profound and the RHS believes that there is a strong case for little or even minimal change to allow the present system to bed down, particularly in terms of the new emphasis on impact.  Institutions are already undertaking extensive planning for a future REF and there is no doubt that radical adjustment would be disruptive and costly.  As stated above, REF affects both individual and institutional choices; as research planning should look to the medium and long term, this is all the more reason not to keep moving the goalposts.

 

 

RHS Letter on Initial Teacher Training

Many of you will be aware that current policies on ITT are having a worrying affect on the future provision of History in schools. At the end of last year the RHS and the Historical Association wrote a joint letter to government ministers and their teams outlining our concerns. We have now received a reply from Nick Gibb, Minister of State for Schools. The response is detailed and covers the issues of ITT provision as a whole. However, it does not address the concerns the subject community have about history. We would like to have some assurances and understanding about our subject, it’s not that we think we’re special, it’s just that each subject is different. So we have sent another letter (and some very specific detail) and requested a meeting.

Here is Nick Gibb’s response, our letter of reply, and the detailed reply outlining all our concerns and points on behalf of the subject community.