1 Review

I was initially excited about this paper because it helps me consider the potential of PLMs and LLMs in novelty evaluation, an understudied aspect of peer review. But as I read the paper, I encountered awkward wording, poor prompt engineering, and a focus on just methodological novelty. I sought a larger, more comprehensive study and the opportunity to utilize more reliable ground truth data than conference peer reviews. A bit more care in thinking through the study design would have been valuable. The authors, to their credit, were on point in identifying key limitations (data leakage should have been mentioned) and contrasted PLMs with LLMs. That sort of contrast is rare. I also appreciated their ablation study, though I would have liked them to conjecture broader implications for the design of future models from that experience. Future studies in this area would be welcome and more polished writing. This paper was repetitive and could have used the help of several copyeditors.