Professor Thelwall will lead the UKRI Metascience Sandpit grant to assess whether it is reasonable to give large language models any role in the peer review of grant proposals.
This new research project will predict reviewer scores for grant proposals using local copies of large language models within a highly secure computing environment. The overall goal is to reduce the burden of peer review for human reviewers and to speed up the peer review process.
Professor Thelwall said: 鈥淲e expect large language models to have some ability to assess the quality of grant review proposals and we are keen to find out how they compare to expert reviewers.
We doubt that they will replace humans yet but hope that they can still play a role in supporting the overall process, such as by guiding decisions when the experts disagree or when not enough reviewers can be found.鈥
Professor Thelwall attended a four day intensive sandpit event in February, led by the UK Metascience Unit. This event brought together a diverse group of participants from a range of disciplines and backgrounds to develop projects to address challenges using UKRI data.
The 12-month project begins in July 2025 and also includes partners The Open University and The University of Salford.