Whoah, CHI is introducing a new reviewing phase where ACs read AI-generated summaries of papers in order to desk-reject 40-50% of submissions: https://chi2027.acm.org/2026/08/17/how-the-chi-2027-papers-review-process-will-work/
4
Replies
1
Boost
@tonofcrates
I don't envy the CHI Papers chairs - this is a really challenging situation.
That said, the conference is moving ever more into a direction I don't feel comfortable with. I'm happy that I'm not involved in any CHI submission this year. Looking on from the sidelines.
@RaphaelWimmer @tonofcrates Agreed - I understand the motivation, but the process is completely opaque.
From TFA: "To help ACs manage the volume, an AI-assisted tool will assess the paper against a rubric based on ACM criteria (Originality, Correctness, Novelty, Importance, and Clarity of Exposition) to produce a preliminary report for each paper."
At the very least, I would want to know which exact LLM they are using and what the prompt is, otherwise, they might just as well flip a coin. 😑
@floe @RaphaelWimmer @tonofcrates I wouldn’t trust that process because I’ve personally found LLMs to be very inconsistent at any kind of “evaluation” task like that. Third is also bound to introduce a heavy bias against any unconventional paper, regardless if it differs in language, structure or topic from the rest.
@jaseg @RaphaelWimmer @tonofcrates Absolutely! What I would really like to do, given the model and prompt, is to run some papers from previous years through the pipeline (maybe 3 times each) and collect some data on how they would fare ...
