Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
Simon Waldman
Simon Waldman
@simon_on_energy@fediscience.org  ·  activity timestamp 2 weeks ago

EDIT: This Nature news piece is based on a study that is only reported in a glossy report by Frontiers that has a clear agenda ("only half of researchers use AI for peer review? Clearly we need to help the rest"). I'm not saying it's wrong, but... I have limited trust. https://www.frontiersin.org/documents/unlocking-ai-potential.pdf

Nature : "More than half of researchers now use AI for peer review"

https://www.nature.com/articles/d41586-025-04066-5

"GPT-5 could mimic the structure of a peer-review report and use polished language, but that it failed to produce constructive feedback and made factual errors."

Yep, that matches the recent review I had, on the basis of which my manuscript was rejected. Vague criticisms that sound bad but are not actionable.

Bigger picture: Nobody has time to do peer review, so many reviews are shoddy. Now shoddy reviews can have AI help, but they're still shoddy reviews.
AI's making it worse, but the fundemental issues here are around workloads.

#academicchatter

  • Copy link
  • Flag this post
  • Block
David Weir :v_enby:
David Weir :v_enby:
@davidjamesweir@mementomori.social replied  ·  activity timestamp 2 weeks ago

@simon_on_energy UKRI now requires one to promise not to use generative AI in any way when agreeing to review UK-based proposals, which is a good step forward. But of course, it's hard to know how many will comply.

I guess it is based on this policy:
https://www.ukri.org/publications/generative-artificial-intelligence-in-application-and-assessment-policy/

Generative artificial intelligence in application and assessment policy

This policy sets out UKRI’s position on the use of generative artificial intelligence (AI) during the funding application and assessment process
  • Copy link
  • Flag this comment
  • Block

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.1-alpha.41 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Members
  • Code of Conduct