Guest Post – Artificial Intelligence in Scholarly Publishing: Are We Moving Faster Than the Evidence?

Editors’ note: Today’s post is by Sarah Hands, Chief Operating Officer at PA EDitorial.

From Adoption to Evaluation

Over the past couple of years, publishers have invested heavily in implementing artificial intelligence (AI) tools. Now a more important phase begins: understanding what early adoption is teaching us. By sharing experiences, evidence, and lessons from successes and failures, the industry can move beyond isolated experiments toward a more informed approach to AI.

The pressure to move quickly is understandable. Publishers are looking for ways to handle growing submission volumes and research integrity challenges. Vendors do not want to miss the market, and editorial teams do not want to appear resistant to innovation. Yet, perhaps the most valuable thing we can do now is make space to evaluate what we’re learning.

“Stop but Don’t Stop”

I remember being struck by Jake Okechukwu Effoduh’s 2024 keynote address to the Association of Learned and Professional Society Publishers (ALPSP) conference, particularly his call to use the opportunity AI presents to stop and examine our industry’s principles and practices without abandoning our commitment to scholarly publishing. I am not questioning whether AI should have a role in editorial workflows. Rather, as publishers gain real-world experience, we can ask better questions: Which tools are delivering meaningful improvements? Where are they introducing new risks or reinforcing existing biases? And how do we ensure that human expertise continues to guide the decisions that matter most?

Is Adoption Happening Faster Than the Evidence?

At times, AI adoption seems to be outpacing our ability to evaluate what it is achieving.

Tools are being introduced because they appear to offer clear efficiencies, because other publishers are already using them, or simply because organizations feel they need to keep pace. But enabling a tool is only the beginning. The more important questions are what happens next: Does it deliver the expected benefit? Where does it work well? Where does it introduce new risks or problems? And what changes further down the workflow?

There are several ways this gap can emerge:

• Adoption can become competitive rather than evidence-led. When other publishers implement AI, pressure can build to follow before there is enough evidence that a tool is solving a genuine problem rather than adding another layer to an existing workflow.
• Success can be measured too narrowly. Speed and productivity are relatively easy to quantify, but they don’t necessarily tell us whether a process has improved or whether quality has been maintained.
• Wider consequences can be harder to see. Even when a tool delivers an apparent efficiency, it may create work elsewhere in the workflow, reduce opportunities for human judgment, or introduce risks that are not immediately visible.

The answer isn’t necessarily to slow down adoption, but to get better at learning from it. As we implement AI, we need to ask what the evidence is telling us and share those lessons more widely.

We know AI can help. But do we know enough about how, where, and under what conditions it helps most effectively?

This is where industry-wide collaboration can make a real difference. Publishers do not need to repeat the same experiments in isolation. As in research, there is value in making null, inconclusive, and negative results visible rather than sharing only successes. Shared evidence of what has worked, what hasn’t, and where results have been inconclusive could help us avoid repeating the same experiments and build a clearer picture of where AI genuinely adds value, where human oversight remains essential, and what effective implementation looks like. Cross-industry work on AI standards and interoperability offers one place to begin.

From “Can It?” to “When Should We Trust It?”

One challenge in evaluating AI in publishing is that we can establish whether a tool can perform a task much more quickly than we can understand when we should trust it to do so.

Rather than simply asking, “Can AI screen manuscripts?” we need to understand where it performs well, where it fails, whether performance varies across disciplines, article types, or writing styles, and what level of human oversight is needed. An AI-generated flag isn’t necessarily a finding; it is a signal requiring context and interpretation. When AI and human judgment disagree, we also need to know how that disagreement should be handled.

These questions are much harder to answer than whether a tool can be installed and switched on. They require evidence over time and a willingness to examine where the technology doesn’t perform as expected.

Bias Doesn’t Disappear — It Can Become Harder to See

One less visible risk of introducing AI into editorial workflows isn’t that AI makes the wrong decision; it’s that we become less likely to question the decision because AI made it.

As AI becomes embedded in editorial processes, editors will increasingly encounter scores, flags, and recommendations alongside their own judgment. These can be valuable signals, but there is a risk that they begin to feel more objective or authoritative than they are. Editors may instinctively question a colleague’s judgment, but will they be equally willing to question a system-generated flag?

This is automation bias: the tendency to give undue weight to an automated system’s recommendation. The danger is not simply that AI gets something wrong; it’s that the human response becomes, “The system has flagged it, so there must be something there,” rather than asking what the flag means and whether it makes sense in context.  But the opposite is also possible. Editors may distrust an AI-generated signal precisely because it comes from a machine, or interpret it through assumptions they already hold about a manuscript. Human judgment is not free from bias either.  Therefore, the introduction of AI does not simply add a new source of bias to an otherwise neutral process; it can interact with existing biases in ways that may be difficult to see.

That matters particularly in research integrity, where a flag should be treated only as a signal requiring interpretation. Human judgment remains essential precisely because context, nuance, and experience can tell us when something that looks unusual may actually be legitimate — or when something that looks reassuring deserves a closer look.

The question, then, is not simply, “Does the tool work?” It is, “What happens to human judgment when we put the tool into the workflow?” The issue is not whether AI replaces human bias with machine objectivity, but how the two interact. Workflows may not have been perfect before, but there is little value in technology that makes us less likely to question, investigate, or think critically.

As AI becomes more prevalent, perhaps the test of a good workflow isn’t how little human intervention it requires, but whether it helps people make better decisions.

Slowing Down Isn’t Resisting Innovation

Publishers are under pressure to do more with the resources they have. A 2026 Organization Science report, for example, found that submissions to the journal had risen by 42% since ChatGPT’s release and described the resulting strain on editors and reviewers. AI may offer genuine efficiencies, so it is entirely understandable that publishers are looking to it for solutions.

Slowing down doesn’t mean resisting innovation. It means giving ourselves enough space to evaluate tools properly, measure outcomes, and monitor unintended consequences. Most importantly, it means asking whether a workflow has genuinely improved — not simply whether we have made it faster.

This means moving deliberately, not hesitantly: adopting AI with the curiosity to experiment, the discipline to evaluate, and the willingness to share what we learn.

I’m Not a Dinosaur, But…

I can’t help but think of Ian Malcolm in Jurassic Park, who said:

“Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.”

Dinosaurs may seem an odd and counterintuitive way to end a discussion about AI in scholarly publishing, but Ian Malcolm’s observation strikes me as the right one. We are no longer relying solely on predictions about what AI might do. We now have real-world experience and emerging evidence with which to test our assumptions, learn from what is working, and be honest about what is not. The challenge is to ask both questions: ‘Can AI do this?’ and ‘Should it?’

The Opportunity Ahead

If the answer to both questions above is yes, a third question follows: “How will we know whether AI made things better?” Without agreed measures and a willingness to share disappointing or inconclusive results, adoption will keep outrunning the evidence.

Perhaps the opportunity now is to bring the same curiosity and critical thinking to AI in practice that we have brought to its possibilities.  As these technologies become embedded in real workflows, we have a chance to become equally curious about what they actually do in practice, where they genuinely add value, where they fall short, and what new risks or unintended consequences they introduce. In which case we need to decide what we need to measure to know that these tools have made scholarly publishing better. And to address that, we need to return to what we are ultimately trying to achieve. Throughput, speed, and productivity may be useful measures, but they are not necessarily measures of improvement in themselves. Processing more manuscripts, more quickly, does not automatically mean we are doing a better job.  Better might mean more accurate decisions, fewer false positives or negatives, less downstream work created for editors, greater confidence in the process, or better outcomes for authors and the integrity of the scholarly record. I would like to think that the goal is bigger than efficiency. Rather, it is to protect and enhance the scholarly record, supporting rigorous research, maintaining quality and integrity, and enabling good decisions throughout the publishing process.

If we can keep that purpose at the center, and become as interested in the evidence of AI’s impact as we have been in its potential, we can move beyond asking simply what can AI do? towards the, perhaps even more important question of where does AI genuinely make scholarly publishing better?

注意转载授权 · 阅读原文