PRIME: Predictive Routing of Interventions by Modeling Effects for Scientific Ideation

   Xie Yu    Zhexuan Cui    Yuke Li    Zirui Song    Fan Zhang    Qian Jiang    Jingpu Yang    Xiuying Chen    Preslav Nakov    Zhuohan Xie\corresponding

Abstract

Feedback-driven iteration is widely used to improve LLM-generated scientific ideas, but existing systems either leave revision decisions implicit in generation, align revisions with feedback-localized weaknesses, or restrict action selection to system-specific operations. This is problematic because problems, hypotheses, and solutions are interdependent: the location where a weakness manifests does not itself determine the revision scope that will produce a better successor. We term this phenomenon diagnosis–intervention mismatch. We introduce Predictive Routing of Interventions by Modeling Effects, PRIME, a plug-and-play feedback-to-revision decision layer for existing scientific ideators. PRIME operates in three stages: (1) Structural Attribution interprets feedback over the problem–hypothesis–solution structure without fixing a revision target; (2) Counterfactual Routing constructs alternative scope-conditioned interventions and compares their projected successor utility and risks through lightweight rollouts; and (3) Intervention Execution compiles the selected intervention into an ordered generator-facing plan with semantic actions, preservation constraints, and dependency requirements for successor generation. We validate diagnosis intervention mismatch on 112 human-annotated idea–feedback pairs. Across four benchmarks, two ideators, and fixed- and open-problem settings, PRIME improves successor quality and routing reliability over strong baselines; removing counterfactual routing reduces quality from 3.463 to 1.960 and increases route regret from 0.426 to 1.929, confirming cross-scope effect modeling as the primary source of its gains.

¹School of Artificial Intelligence, Wuhan University ²MBZUAI

³Northeastern University ⁴Zhongguancun Academy

jingpuyang290@gmail.com, fengxian.ji@mbzuai.ac.ae, fan.zhang@mbzuai.ac.ae, zhuohan.xie@mbzuai.ac.ae

Introduction

With the widespread adoption of large language models in scientific research automation, LLM-based scientific idea generation has emerged as an important research direction (Radensky et al. 2026; Luo et al. 2025; Shahhosseini et al. 2025; Hasib et al. 2025; Reddy and Shojaee 2025; Radensky et al. 2024; Chen 2025; Gelles et al. 2024; Xiong et al. 2024b; Mishra et al. 2026; Kon et al. 2025; Liu et al. 2025a). Within this area, feedback-conditioned scientific idea revision supports the generation of successor ideas based on evaluations of a current idea. For example, in idea refinement, feedback is used to address the limitations of the current idea and improve its hypotheses, methodology, and validation plan (Deshwal 2025; Madaan et al. 2023; Alkan et al. 2025; Liu et al. 2025b; Keya et al. 2025); in idea evolution, feedback can guide the system in adjusting its research trajectory and exploring alternative hypotheses, solutions, or problem formulations (Yuksekgonul et al. 2026; Zhang 2025; Kulkarni et al. 2025; Nigam et al. 2024). Despite their different settings, these scenarios share the same core objective: translating feedback into concrete updates to successor ideas. Reliable feedback also depends on judge quality: recent evidence shows that LLM judges can favor presentation style over scientific substance when evaluating research ideas (Ji et al. 2026). At a later stage of scientific automation, LabGuard grounds natural-language laboratory rules into runtime guards for embodied laboratory agents (Yang et al. 2026); PRIME addresses the earlier step of revising ideas in response to feedback.

Based on how feedback is translated into subsequent revision actions, existing systems can be broadly categorized into three types. The first is holistic rewriting, exemplified by Self-Refine (Madaan et al. 2023), in which the current idea, together with an overall score, natural-language critique, or reviewer feedback, is provided to a generative model to produce a complete successor version. In this approach, both what should be revised and the appropriate scope of revision remain implicit in the rewriting process. The second is localized corrective revision, exemplified by ResearchAgent (Baek et al. 2025) and EvoIdeator (Sauter et al. 2026), in which feedback is first grounded in specific text spans, evaluation dimensions, or idea components, after which localized corrections are generated for the diagnosed weaknesses. The third is workflow-coupled revision, exemplified by Deep Ideation (Zhao et al. 2025), VirSci (Su et al. 2025a), and EIG (Dong et al. 2026), in which the next action is selected from a predefined set of revision operations according to the current idea’s evaluation scores, defect categories, or structured state, using either predefined rules or a learned controller. These three categories respectively leave revision decisions implicit in holistic rewriting, bind them directly to the weaknesses localized by feedback, or constrain them to an operation space predefined by a particular system.

Table 1 situates these revision mechanisms within the broader landscape of scientific ideation methods. The first category leaves the revision decision implicit in generation, while the second commonly treats the diagnosed weakness as the direct target of revision (Madaan et al. 2023; Baek et al. 2025; Sauter et al. 2026). The third makes action selection explicit but couples the decision to a system-specific state representation and a predefined operation space (Dong et al. 2026). None of these mechanisms provides a general decision layer for translating feedback into revision scopes that can be executed by existing idea generators. This limitation is particularly important in scientific ideation, where the problem, hypothesis, and solution are interdependent (Xiong et al. 2024a; Pu et al. 2025; Dong et al. 2026). The same observable weakness may arise from causes at different levels, while interventions at different levels may propagate through these dependencies in different ways. Taken together, these methods leave a fundamental feedback-to-revision gap unresolved: "where a weakness is diagnosed does not by itself determine where revision should be applied".

Bridging this gap presents three key challenges. First, the locus where feedback manifests may not be the component that actually causes the problem. The same feedback may correspond to multiple plausible intervention loci, making the revision scope difficult to determine uniquely (Xiong et al. 2024a; Pu et al. 2025; Dong et al. 2026). Second, the appropriate intervention scope cannot be directly observed before revision. For a given idea, only the successor produced by the selected revision is typically observed, while exhaustively executing all alternatives is prohibitively expensive (Hu et al. 2024; Wang et al. 2026; Sauter et al. 2026; Li et al. 2026b). Third, translating the intervention scope into actionable instructions that guide the generator to revise target components, update dependencies, and preserve valid content (Baek et al. 2025; Pu et al. 2025; Dong et al. 2026).

We annotate 112 human-annotated idea–feedback pairs, and evaluate six baselines, revealing a diagnosis–intervention mismatch in scientific ideation: feedback may target a different scope from the one that produces a higher-quality successor idea. To bridge this gap, as shown in Fig. 1, we introduce Predictive Routing of Interventions by Modeling Effects, PRIME, a plug-and-play feedback-to-revision decision layer that can be inserted between an existing feedback provider and idea generator. PRIME consists of three components: (i) Structural Attribution, which interprets feedback over the research problem, scientific hypothesis, solution, and their dependencies to distinguish where a weakness manifests from its potential structural sources; (ii) Counterfactual Routing, which constructs candidate interventions covering the feedback-localized component, its potential upstream causes, cross-component dependencies, and problem-reframing directions, and uses lightweight successor rollouts to evaluate their feedback resolution, prospective utility, and structural risks before selecting an intervention scope; and (iii) Intervention Execution, which translates the selected intervention into an ordered, generator-facing revision plan specifying the target content, concrete semantic operations, preservation constraints, and required dependency updates. Through these three components, PRIME invokes the original idea generator to produce a complete successor idea. We evaluate PRIME on four benchmarks. Compared with direct feedback updating, PRIME-Refinement and PRIME-Evolution improve 14/16 and 15/16 metrics, respectively, and jointly achieve the best results on 10/16 metrics, showing that effective revision requires deciding not only how to address feedback, but also where to intervene.

Our main contributions and findings are: (i) We formulate the diagnosis–intervention mismatch in scientific ideation, where a feedback-localized weakness does not uniquely determine the revision scope, and validate it through a controlled audit of 114 human-annotated idea–feedback pairs; (ii) We propose PRIME, a plug-and-play feedback-to-revision decision layer that performs revision planning through Structural Attribution, Counterfactual Routing, and Intervention Execution without replacing the underlying ideator; (iii) Across four benchmarks, two ideators, and both refinement and evolution settings, PRIME jointly achieves the best results on 20/32 metric ideator combinations; in the intervention audit, PRIME-Evolution further achieves the highest successor utility of 64.16 and the lowest route regret of 3.93.

Related Works

Iterative Scientific Ideation and Direct Revision.

As shown in Fig. 2(a,b), Iterative scientific ideation systems improve candidate ideas through repeated critique, review, refinement, or evolution. General LLM self-improvement methods, such as Self-Refine and Reflexion, show that verbal feedback can drive iterative revision (Shinn et al. 2023). In scientific ideation, ResearchAgent, IdeaSynth, AI Scientist, Co-Scientist, Chain of Ideas, Many Heads, and SciDeator employ reviewing agents, literature-grounded critiques, rubric-based judgments, or multi-agent interactions to improve generated proposals (Baek et al. 2025; Lu et al. 2024; Gottweis et al. 2026; Li et al. 2025; Su et al. 2025b). These systems demonstrate the value of evaluation-driven iteration, but they use evaluation as a direct revision signal, for example, by rewriting the entire proposal, revising its weakest aspect, or continuing along the current trajectory. PRIME explicitly models the diagnosis-to-intervention decision instead of treating the diagnosed locus as the revision target.

Structured State and Graph-Based Edit Control.

Structured ideation methods reduce the opacity of text-only revision by exposing intermediate idea states. SciPIP, KG-CoI, Graph2Idea, and IdeaSynth ground ideation in paper graphs, knowledge graphs, graph-structured contexts, or decomposed idea facets (Wang et al. 2024b; Xiong et al. 2024a; Li et al. 2026a; Pu et al. 2025). Most closely related to our setting, EIG represents an evolving proposal as an idea graph and learns edit-and-commit operations over localized graph structures (Dong et al. 2026). This line of work shows that structured weakness localization and editable state representations are useful. EIG controls localized edits within its idea-graph workflow, whereas PRIME selects an intervention scope without equating the diagnosed locus with the revision target.

Search and Evolution over Idea Spaces.

Search- and evolution-based methods broaden scientific ideation by exploring multiple candidate ideas, branches, or trajectories. Nova, Chain of Ideas, AI Scientist, Co-Scientist, FlowPIE, EvoIdeator, and EvoGens(Jin et al. 2026; Jiang et al. 2025; Yamada et al. 2025; Chen et al. 2026b; Du et al. 2026; Liu et al. 2026; Yao et al. 2023) improve idea generation through planning, search, population evolution, checklist-guided optimization, or test-time exploration (Hu et al. 2024; Lu et al. 2024; Gottweis et al. 2026; Wang et al. 2026; Sauter et al. 2026; Li et al. 2026b). Their selection targets are typically generated ideas, branches, population members, or trajectories. These methods do not explicitly ask whether, given the same feedback-localized weakness, the next update should take the form of a local refinement, an upstream rationale revision, or a broader problem-level evolution. PRIME instead selects a semantic intervention scope before successor generation, turning feedback into an explicit revision decision for the existing ideator.

Problem Formulation

We denote the research context, scientific idea generator, and current idea by C, G, and It, respectively. Given evaluation criteria Γ, an evaluator produces feedback ft=J⁡(It,C,Γ). Rather than using ft directly as an edit instruction, we study how to select a revision by jointly considering the feedback and the structural context of the current idea.

The feasible intervention set is 𝒜t=𝒜⁡(It,ft,C). Each a∈𝒜t specifies a revision scope s⁡(a) and a corresponding revision, inducing a potential successor It+1a∼G(⋅∣C,It,ft,a). The feedback-to-revision decision is

where U measures feedback resolution and scientific quality, while ℛ captures revision risks such as loss of valid content, dependency inconsistency, and task drift. Conditioning the intervention set on both It and ft captures that the same feedback may admit different revisions under different idea structures, while the expectation reflects that their successor effects are unobserved before execution.

The feedback-localized locus is ℓt=Loc⁡(It,ft), and a diagnosis-aligned intervention atdiag satisfies s⁡(atdiag)=ℓt. Diagnosis–intervention mismatch occurs when an intervention with s⁡(a)≠ℓt achieves a higher objective value than atdiag. Thus, the diagnosed locus informs intervention selection but does not by itself determine the revision scope.

PRIME

Overview

PRIME approximates the feedback-to-revision decision in Equation. 1 through three stages. Structural Attribution interprets feedback over the problem, hypothesis, solution, and their dependencies without fixing a revision target; Counterfactual Routing compares alternative intervention scopes using their projected successor effects; and Intervention Execution translates the selected intervention into an ordered revision plan for the original generator. PRIME therefore operates as a plug-in decision layer without modifying the feedback provider or idea generator.

Structural Attribution

Feedback identifies an observable weakness but does not uniquely reveal its structural source. Structural Attribution therefore interprets ft over the structure of It without selecting an intervention scope. PRIME normalizes ft into f¯t, which contains an improvement goal, supporting evidence, preservation requirements, and revision constraints. The current idea is then represented as

where Pt, Ht, and St denote the research problem, scientific hypothesis, and solution, respectively. The dependencies require the hypothesis to address the problem and the solution to operationalize or test the hypothesis.

PRIME interprets f¯t through six structural views 𝒰={P,H,S,P​H,H​S,P​H​S}. The first three views examine individual components, P​H and H​S examine adjacent dependencies, and P​H​S examines the complete reasoning chain. For each u∈𝒰, PRIME produces one attribution record containing supporting evidence, the observed failure, a plausible structural explanation, the corresponding repair requirement, downstream consequences, severity, and confidence. The module output is

The records in 𝒵t provide diagnostic evidence rather than revision actions: they describe which structural conditions are deficient without prescribing which components should be modified. Consequently, one attribution may support multiple intervention scopes in Counterfactual Routing.

Counterfactual Routing

Structural Attribution provides evidence for revision but does not determine which intervention will produce a better successor. Counterfactual Routing therefore constructs candidate interventions and compares their projected effects before selecting one for execution.

Candidate Intervention Construction.

An intervention separates where to revise from how to revise. Its scope satisfies s⁡(a)∈ℳ={P,H,S,P​H,H​S,P​H​S}, where each scope is an edit mask. The P mask covers the problem framing, scope, bottleneck, and objective; H covers the hypothesis and its mechanism; and S covers the solution, its mechanism, and validation. Composite masks permit joint revisions to the corresponding components, while fields outside the selected mask are preserved. Dependencies are represented through explicit requirements and risks rather than automatic dependency closure.

The revision within a scope is specified by an ordered action plan Πta composed of Add, Remove, Revise, and Replace operations. Each step identifies its target component and field, concrete instruction, success condition, and dependencies on preceding steps. Add introduces missing content, Remove deletes unsupported content, Revise modifies an existing commitment while retaining its role, and Replace substitutes it with a new commitment.

For interventions containing P, the problem operation is Update or Switch. Update changes the problem framing while preserving its identity, whereas Switch moves to another context-grounded problem and is available only in evolution settings. Interventions not containing P use None.

Problem-Reference Conditioning.

Interventions involving P are conditioned on a persistent Problem Pool 𝒫t, which maintains the active problem identity and grounded alternative references. A P-Update uses the active reference as an identity anchor, whereas a P-Switch uses an alternative reference as its target. The pool only supplies reference conditions for candidate construction and does not rank interventions. PRIME-Refinement uses a fixed reference policy and disables P-Switch; PRIME-Evolution uses an open policy and additionally constructs P-Switch candidates from grounded alternatives in 𝒫t. Both variants otherwise use the same attribution, routing, and execution procedure. The open policy permits but does not force evolution: only a selected P-Switch realizes an evolution step.

Using the structural representation Xt, normalized feedback f¯t, and attribution 𝒵t, PRIME constructs

Each candidate additionally cites supporting attributions 𝒵ta⊆𝒵t and carries preservation requirements 𝒦ta and dependency requirements Λta. The supporting attribution need not have the same label as s⁡(a): evidence from one structural view may support a local, upstream, or joint intervention.

Counterfactual Evaluation and Selection.

For each candidate a∈𝒜~t, PRIME generates a lightweight structured successor

The rollout applies the declared actions while copying fields outside their targets unchanged. It approximates the potential successor It+1a in Equation 1, but is not the final idea produced by G.

A routing evaluator estimates the prospective utility U^t​(a) and revision risk ℛ^t​(a) of each rollout. The former captures feedback resolution and scientific quality, while the latter captures unsupported changes, infeasibility, dependency breakage, task drift, overcomplexity, and uncertainty. Candidates violating predefined scope, preservation, risk, or problem-reference constraints are removed. PRIME then selects

If no candidate passes the strict validity constraints, PRIME selects the highest-ranked executable candidate. Counterfactual Routing outputs the selected intervention a^t and its projected successor X~t+1a^t for Intervention Execution.

Intervention Execution

The selected intervention must be translated into instructions that the original generator can execute without reconsidering its scope. PRIME therefore compiles a^t and its projected successor into a generator-facing specification and invokes G once:

The execution specification χt+1 contains the selected scope s⁡(a^t), ordered action plan Πta^t, preservation requirements 𝒦ta^t, and dependency requirements Λta^t. The projected successor provides a semantic target for these actions rather than a draft to be copied.

The generator is instructed to execute the selected actions in order, preserve content outside s⁡(a^t), satisfy the specified dependencies, and return a complete successor idea in the same format as It. The compiled specification instructs G not to introduce additional revision targets from the raw feedback or to modify content outside the selected scope. Because PRIME communicates with G through χt+1 without modifying its parameters, it can be combined with different instruction-conditioned idea generators.

Experiments

Experimental Setup

Baselines.

We compare PRIME with six feedback-to-revision strategies. (1) Flat Feedback directly injects a single natural-language improvement direction into the subsequent generation task. (2) Problem-First H/S fixes the initial problem and revises only the hypothesis and solution. (3) Problem Regeneration updates only the problem used to guide the next iteration. (4) Direct Feedback Decomposition generates an ordered, free-form list of revisions and directly executes the highest-ranked item. (5) Feedback Reasoning separates feedback generation from revision planning and serves as a simplified PRIME variant without counterfactual cross-scope comparison. (6) Graph-Based grounds feedback and localized edits in a structured problem–hypothesis–solution graph, while its reasoning variant explicitly selects a graph edit before generation. For each baseline, we evaluate two models: Direct Prompting and SciMon (Wang et al. 2024a).

Benchmark.

We evaluate PRIME on four scientific idea benchmarks: AI Idea Bench (Qiu et al. 2025), IdeaBench (Guo et al. 2025), LiveIdeaBench (Ruan et al. 2024), and MLR-Bench (Chen et al. 2026a). These benchmarks follow two evaluation protocols. The first assesses idea reconstruction by comparing generated ideas with target ideas from real papers. AI Idea Bench uses I2I, I2T, and MCQ to evaluate idea matching, topic alignment, and identification of key research content. IdeaBench further uses the novelty score NIS and feasibility score FIS to assess innovation relative to existing work and practical implementability. The second category follows a paper-review-style protocol, with multiple LLM judges directly assessing overall idea quality. LiveIdeaBench evaluates originality, feasibility, and clarity, focusing on whether an idea forms a complete, coherent, and worthwhile research plan. MLR-Bench evaluates clarity, novelty, feasibility, and significance to determine whether an idea approaches the quality expected of a machine learning research paper. For fair comparison, all methods use the same scientific context, reference materials, feedback provider, Qwen3.5-4B generator, 20 iterations, and a generation budget of 2,048 tokens on each benchmark. For all experiments, we compute the score for each idea generated across all iterations and then report the average score.

Main Results

The main experiment examines whether explicitly modeling the feedback-to-revision decision improves the quality of iteratively generated successor ideas across ideators and revision settings. Table 2 compares five general update policies with PRIME-Refinement and PRIME-Evolution under Direct Prompting and SciMON across four benchmarks. The Graph-Based block further tests whether PRIME can be integrated with a structured ideator. Using random seed 101, we sample 50 tasks from each benchmark and perform 20 feedback-to-revision iterations for every task under each policy–ideator configuration. All methods share the same Qwen3.5-4B base model, research contexts, feedback provider, generation budget, and benchmark-specific evaluation protocol. Across the five general policies and two PRIME variants, PRIME achieves the best results on 10/16 metrics under both Direct Prompting and SciMON. Compared with Flat Feedback, PRIME-Refinement and PRIME-Evolution improve 14/16 and 15/16 metrics under Direct Prompting, and 11/16 and 13/16 under SciMON, respectively. PRIME-Refinement leads all three AI Idea Bench metrics under Direct Prompting, while PRIME-Evolution achieves the highest MLR-Bench overall scores under both ideators (7.260 and 7.120). Although results on LiveIdeaBench are mixed, adding PRIME-Refinement to the Graph-Based ideator improves all LiveIdeaBench and MLR-Bench metrics, increasing the MLR-Bench overall score from 6.589 to 7.060. These results support PRIME as an effective plug-in layer for feedback-to-revision routing.

Statistical Significance and Multi-Judge Consistency. To ensure robust evaluation and reduce judge-specific bias, we randomly sample 50 generated ideas from each of the eight result groups, yielding 400 sampled ideas in total. Each idea is independently scored by three LLM judges, resulting in 1,200 evaluation records. We report exact agreement, EA=N−1∑i𝟏[yi(a)=yi(b)]; adjacent agreement, AA=N−1∑i𝟏[|yi(a)−yi(b)|≤1]; and the Pearson correlation between judges. The resulting agreement scores, EA=68.7%, AA=78.9%, and Pearson r=0.72, indicate strong inter-judge consistency.

Diagnostic Intervention Mismatch

We test whether the locus identified by feedback reliably determines the intervention scope that produces a better successor idea. From 114 human-annotated idea–feedback pairs, we retain 112 complete instances from IdeaBench, LiveIdeaBench, and MLR-Bench, comprising 42, 42, and 28 instances, respectively. For each instance, we hold the idea, feedback, context, generator, and generation budget fixed, and vary only the intervention scope over {P,H,S,P​H,H​S,P​H​S}, yielding 672 scope-conditioned outcomes. Human annotators identify the diagnostic locus using only the original idea and feedback, without observing successors or policy decisions, while blind evaluators independently assess the resulting successors. A selection split defines the ϵ-near-optimal intervention set, and a separate evaluation split measures held-out mismatch and regret. Panel (a) audits whether diagnosis identifies a near-optimal intervention, while Panel (b) applies seven update policies to the same outcome bank to evaluate feedback-to-intervention routing.

Table 3 shows a substantial diagnosis–intervention mismatch. At the primary threshold ϵ=2, the human-ground-truth diagnosis falls outside the near-optimal intervention set in 65.2% of instances; the mismatch remains 40.2% when all admissible diagnostic loci are considered and replicates on the held-out split in 43.8% of instances. Directly following the diagnosed locus incurs an average held-out regret of 3.79 points (95% CI: [2.74,4.93], p=.0001), with similar mismatch rates across model and individual-human diagnoses (61.6%–71.2%). For routing, PRIME-Evolution achieves the highest Target Hit@2 (43.8%), Benefit@2 (65.2%), and successor utility (64.16), while obtaining the lowest RepErr@2 (16.1%), Harm@2 (2.7%), and evaluation regret (3.93). PRIME-Refinement does not exhibit the same advantage, suggesting that restricting problem-level alternatives can be limiting when broader intervention is beneficial. Overall, the results show that feedback localization alone is insufficient and that comparing the projected effects of alternative intervention scopes improves revision routing.

Base-Model and Ideators Generalization

We evaluate whether PRIME generalizes across both base models and ideation workflows through two controlled substitutions on AI Idea Bench. First, we replace the primary base model with Llama while retaining Direct Prompting as the ideator. Second, we retain the primary base model but replace Direct Prompting with CoI-Agent. All other settings, including the update policies, feedback provider, generation budget, and evaluation protocol, remain unchanged. Table 4 reports the results. Under the Llama substitution, PRIME obtains the best I2I, I2T, and IMCQ scores across its refinement and evolution variants. Under the CoI-Agent substitution, PRIME-Evolution achieves the best IMCQ and a near-best I2T score, although Flat Feedback remains stronger on I2I. These results show that PRIME’s gains transfer strongly across base models, while its benefits under ideator substitution are metric-dependent.

Ablation

We conduct ablation studies to examine the key design choices of PRIME by removing counterfactual routing, upstream intervention candidates, problem-level candidates, and the risk penalty from the full model. As reported in Table 5, removing counterfactual routing replaces successor-effect comparison with diagnosis- based intervention selection, removing upstream or problem-level candidates restricts the intervention space, and removing the risk penalty selects interventions solely according to their predicted utility. Counterfactual routing is the most influential component: removing it decreases Quality from 3.463 to 1.960, increases Regret from 0.426 to 1.929, and decreases Consistency from 3.173 to 1.773. Excluding upstream and problem-level candidates also consistently reduces successor quality and increases routing regret. In contrast, removing the risk penalty results in only a minor performance degradation, suggesting that projected successor utility dominates routing decisions on this benchmark, while risk modeling primarily provides a modest regularization effect.

Conclusion

In this paper, we identify reference-frame ambiguity as a key challenge in context-driven scientific idea generation, where ideas may target different problems and flat comparison may conflate problem selection with solution quality. We propose PRIME, a plug-and-play framework that recovers latent problem frames, organizes ideas in a dynamic problem tree, and provides problem-conditioned evaluation and feedback. Our results show that feedback localization alone is insufficient for selecting revision scope; by making both problem positioning and intervention routing explicit, PRIME enables more reliable comparison, actionable feedback, and controllable iterative improvement for scientific ideation.

References

• Alkan et al. (2025) A. K. Alkan, S. Sourav, M. Jablonska, S. Astarita, R. Chakrabarty, N. Garuda, P. Khetarpal, M. Pióro, D. Tanoglidis, K. G. Iyer, M. S. Polimera, M. J. Smith, T. Ghosal, M. Huertas-Company, S. Kruk, K. Schawinski, and I. Ciucă A survey on hypothesis generation for scientific discovery in the era of large language models. External Links: 2504.05496, Link Cited by: Introduction.
• Baek et al. (2025) J. Baek, S. K. Jauhar, S. Cucerzan, and S. J. Hwang ResearchAgent: iterative research idea generation over scientific literature with large language models. In Proceedings of NAACL 2025, Cited by: Introduction, Introduction, Introduction, Iterative Scientific Ideation and Direct Revision..
• Chen et al. (2026a) H. Chen, M. Xiong, Y. Lu, W. Han, A. Deng, Y. He, J. Wu, Y. Li, Y. Liu, and B. Hooi Mlr-bench: evaluating ai agents on open-ended machine learning research. Advances in Neural Information Processing Systems 38. Cited by: Benchmark..
• Chen et al. (2026b) J. Chen, B. D. Mishra, J. Nam, R. Meng, T. Pfister, and J. Yoon MARS: modular agent with reflective search for automated ai research. arXiv preprint arXiv:2602.02660. Cited by: Search and Evolution over Idea Spaces..
• Chen (2025) W. Chen Automated, interpretable, and scalable scientific machine learning. Proceedings of the AAAI Conference on Artificial Intelligence 39 (27), pp. 28708–28708. External Links: Document, ISSN 2159-5399, Link Cited by: Introduction.
• Deshwal (2025) A. Deshwal Adaptive experimental design to accelerate scientific discovery and engineering design. Proceedings of the AAAI Conference on Artificial Intelligence 39 (27), pp. 28709–28709. External Links: Document, ISSN 2159-5399, Link Cited by: Introduction.
• Dong et al. (2026) J. Dong, B. Li, and W. Lin Evolving idea graphs with learnable edits-and-commits for multi-agent scientific ideation. External Links: 2605.04922, Link Cited by: Introduction, Introduction, Introduction, Structured State and Graph-Based Edit Control..
• Du et al. (2026) Y. Du, X. Yang, Z. Zhou, W. Liu, Z. Lei, Z. Chen, F. Liu, H. Wu, Y. Cai, Z. Liu, et al. DataMaster: data-centric autonomous ai research. arXiv preprint arXiv:2605.10906. Cited by: Search and Evolution over Idea Spaces..
• Gelles et al. (2024) R. Gelles, V. Kinoshita, M. Musser, and J. Dunham Resource democratization: is compute the binding constraint on ai research?. Proceedings of the AAAI Conference on Artificial Intelligence 38 (18), pp. 19840–19848. External Links: Document, ISSN 2159-5399, Link Cited by: Introduction.
• Gottweis et al. (2026) J. Gottweis, W. Weng, A. Daryin, T. Tu, P. Sirkovic, A. Myaskovsky, G. Glowaty, F. Weissenberger, A. Orlandi, D. Popovici, A. Palepu, K. Rong, R. Tanno, K. Saab, F. Zhang, J. Blum, A. Carroll, K. Kulkarni, N. Tomašev, D. Zverinski, I. Rendulic, E. Vedadi, F. Hasler, L. Rimanic, M. Boia, I. Budiselic, B. Feinstein, M. Bellaiche, T. Sheffer, J. Freyberg, J. Ratcliff, O. Bertolli, K. Chou, A. Hassidim, B. Gokturk, A. Vahdat, Y. Guan, V. Dhillon, E. D. Vaishnav, B. Lee, T. R. D. Costa, J. R. Penadés, G. Peltz, Y. Matias, J. Manyika, D. Hassabis, Y. Xu, P. Kohli, A. Pawlosky, A. Karthikesalingam, and V. Natarajan Accelerating scientific discovery with co-scientist. Nature. External Links: ISSN 1476-4687, Link, Document Cited by: Iterative Scientific Ideation and Direct Revision., Search and Evolution over Idea Spaces..
• Guo et al. (2025) S. Guo, A. H. Shariatmadari, G. Xiong, A. Huang, M. Kim, C. M. Williams, S. Bekiranov, and A. Zhang Ideabench: benchmarking large language models for research idea generation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 5888–5899. Cited by: Benchmark..
• Hasib et al. (2025) M. M. Hasib, S. Jo, H. Sinha, J. Song, A. Das, Z. Liu, H. Galloway, H. Huang, K. Zhang, S. Gao, et al. A process-centric survey of ai for scientific discovery through the exhyte framework. Cited by: Introduction.
• Hu et al. (2024) X. Hu, H. Fu, J. Wang, Y. Wang, Z. Li, R. Xu, Y. Lu, Y. Jin, L. Pan, and Z. Lan Nova: an iterative planning and search approach to enhance novelty and diversity of llm generated ideas. arXiv preprint arXiv:2410.14255. Cited by: Introduction, Search and Evolution over Idea Spaces..
• Ji et al. (2026) F. Ji, Y. Li, J. Yang, J. Wu, F. Zhang, Z. Cui, Y. Xie, M. Peng, Q. Xie, X. Chen, and Z. Xie Style wins, substance loses: a diagnosis of llm-as-judge in idea generation. External Links: 2608.01666, Link Cited by: Introduction.
• Jiang et al. (2025) Z. Jiang, D. Schmidt, D. Srikanth, D. Xu, I. Kaplan, D. Jacenko, and Y. Wu Aide: ai-driven exploration in the space of code. arXiv preprint arXiv:2502.13138. Cited by: Search and Evolution over Idea Spaces..
• Jin et al. (2026) J. Jin, Y. Hu, K. Qiu, Q. Dai, C. Luo, G. Dong, X. Li, T. Zhao, X. Ma, G. Zhang, et al. Toward generalist autonomous research via hypothesis-tree refinement. arXiv preprint arXiv:2606.11926. Cited by: Search and Evolution over Idea Spaces..
• Keya et al. (2025) F. Keya, G. Rabby, P. Mitra, S. Vahdati, S. Auer, and Y. Jaradeh SCI-idea: context-aware scientific ideation using token and sentence embeddings. External Links: 2503.19257, Link Cited by: Introduction.
• Kon et al. (2025) P. T. J. Kon, J. Liu, X. Zhu, Q. Ding, J. Peng, J. Xing, Y. Huang, Y. Qiu, J. Srinivasa, M. Lee, et al. Exp-bench: can ai conduct ai research experiments?. arXiv preprint arXiv:2505.24785. Cited by: Introduction.
• Kulkarni et al. (2025) A. Kulkarni, F. Alotaibi, X. Zeng, L. Wu, T. Zeng, B. M. Yao, M. Liu, S. Zhang, L. Huang, and D. Zhou Scientific hypothesis generation and validation: methods, datasets, and future directions. arXiv preprint arXiv:2505.04651. Cited by: Introduction.
• Li et al. (2025) L. Li, W. Xu, J. Guo, R. Zhao, X. Li, Y. Yuan, B. Zhang, Y. Jiang, Y. Xin, R. Dang, Y. Rong, D. Zhao, T. Feng, and L. Bing Chain of ideas: revolutionizing research via novel idea development with llm agents. In Findings of the Association for Computational Linguistics: EMNLP 2025, Cited by: Iterative Scientific Ideation and Direct Revision..
• Li et al. (2026a) X. Li, H. Tu, and X. Han Graph2Idea:retrieval-augmented scientific idea generation with graph-structured contexts. External Links: 2606.09105, Link Cited by: Structured State and Graph-Based Edit Control..
• Li et al. (2026b) X. Li, H. Tu, X. Li, K. Zhao, X. Han, and Z. Liu EvoGens: a population-based heuristic search framework for scientific idea generation. arXiv preprint arXiv:2605.30961. Cited by: Introduction, Search and Evolution over Idea Spaces..
• Liu et al. (2025a) H. Liu, S. Huang, J. Hu, Y. Zhou, and C. Tan Hypobench: towards systematic and principled benchmarking for hypothesis generation. arXiv preprint arXiv:2504.11524. Cited by: Introduction.
• Liu et al. (2025b) Y. Liu, P. Sharma, M. Oswal, H. Xia, and Y. Huang PersonaFlow: designing llm-simulated expert perspectives for enhanced research ideation. In Proceedings of the ACM Conference, Cited by: Introduction.
• Liu et al. (2026) Z. Liu, Z. Shi, Y. Sang, B. He, M. Lin, T. Wei, D. Wang, B. Dumoulin, W. Jin, and H. Lu Adaptive auto-harness: sustained self-improvement for agentic system deployment on open-ended task streams. arXiv preprint arXiv:2606.01770. Cited by: Search and Evolution over Idea Spaces..
• Lu et al. (2024) C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha The ai scientist: towards fully automated open-ended scientific discovery. External Links: 2408.06292, Link Cited by: Iterative Scientific Ideation and Direct Revision., Search and Evolution over Idea Spaces..
• Luo et al. (2025) Z. Luo, Z. Yang, Z. Xu, W. Yang, and X. Du LLM4SR: a survey on large language models for scientific research. External Links: 2501.04306, Link Cited by: Introduction.
• Madaan et al. (2023) A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. Self-refine: iterative refinement with self-feedback. Advances in neural information processing systems 36, pp. 46534–46594. Cited by: Introduction, Introduction, Introduction.
• Mishra et al. (2026) S. Mishra, G. Sahu, M. Pedersoli, L. Charlin, J. Dolz, and C. Pal AInstein: can llms solve research problems from parametric memory alone?. External Links: 2510.05432, Link Cited by: Introduction.
• Nigam et al. (2024) H. Nigam, M. Patwardhan, L. Vig, and G. Shroff Acceleron: a tool to accelerate research ideation. arXiv (Cornell University). External Links: Document Cited by: Introduction.
• Pu et al. (2025) K. Pu, K. J. K. Feng, T. Grossman, T. Hope, B. Dalvi Mishra, M. Latzke, J. Bragg, J. C. Chang, and P. Siangliulue IdeaSynth: iterative research idea development through evolving and composing idea facets with literature-grounded feedback. In Proceedings of the CHI Conference on Human Factors in Computing Systems, Cited by: Introduction, Introduction, Structured State and Graph-Based Edit Control..
• Qiu et al. (2025) Y. Qiu, H. Zhang, Z. Xu, M. Li, D. Song, Z. Wang, and K. Zhang Ai idea bench 2025: ai research idea generation benchmark. arXiv preprint arXiv:2504.14191. Cited by: Benchmark..
• Radensky et al. (2024) M. Radensky, S. Shahid, R. Fok, P. Siangliulue, T. Hope, and D. S. Weld Scideator: human-llm compound system for scientific ideation through facet recombination and novelty evaluation. arXiv preprint arXiv:2409.14634. Cited by: Introduction.
• Radensky et al. (2026) M. Radensky, S. Shahid, R. Fok, P. Siangliulue, T. Hope, and D. S. Weld Human-llm compound system for scientific ideation through facet recombination and novelty evaluation. External Links: 2409.14634, Link Cited by: Introduction.
• Reddy and Shojaee (2025) C. K. Reddy and P. Shojaee Towards scientific discovery with generative ai: progress, opportunities, and challenges. Proceedings of the AAAI Conference on Artificial Intelligence 39 (27), pp. 28601–28609. External Links: Document, ISSN 2159-5399, Link Cited by: Introduction.
• Ruan et al. (2024) K. Ruan, X. Wang, J. Hong, P. Wang, Y. Liu, and H. Sun LiveIdeaBench: evaluating llms’ divergent thinking for scientific idea generation with minimal context. arXiv preprint arXiv:2412.17596. Cited by: Benchmark..
• Sauter et al. (2026) A. Sauter, Y. Zhao, J. Urbani, W. Hu, Z. Meng, L. Zhou, X. Yan, and Y. Lyu EvoIdeator: evolving scientific ideas through checklist-grounded reinforcement learning. arXiv preprint arXiv:2603.21728. Cited by: Introduction, Introduction, Introduction, Search and Evolution over Idea Spaces..
• Shahhosseini et al. (2025) F. Shahhosseini, A. Marioriyad, A. Momen, M. S. Baghshah, M. H. Rohban, and S. H. Javanmard Large language models for scientific idea generation: a creativity-centered survey. arXiv preprint arXiv:2511.07448. Cited by: Introduction.
• Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: language agents with verbal reinforcement learning. Advances in neural information processing systems 36, pp. 8634–8652. Cited by: Iterative Scientific Ideation and Direct Revision..
• Su et al. (2025a) H. Su, R. Chen, S. Tang, Z. Yin, X. Zheng, J. Li, B. Qi, Q. Wu, H. Li, W. Ouyang, et al. Many heads are better than one: improved scientific idea generation by a llm-based multi-agent system. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 28201–28240. Cited by: Introduction.
• Su et al. (2025b) H. Su, R. Chen, S. Tang, Z. Yin, X. Zheng, J. Li, B. Qi, Q. Wu, H. Li, W. Ouyang, P. Torr, B. Zhou, and N. Dong Many heads are better than one: improved scientific idea generation by a llm-based multi-agent system. External Links: 2410.09403, Link Cited by: Iterative Scientific Ideation and Direct Revision..
• Wang et al. (2024a) Q. Wang, D. Downey, H. Ji, and T. Hope SciMON: scientific inspiration machines optimized for novelty. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, pp. 279–299. External Links: Document Cited by: Baselines..
• Wang et al. (2026) Q. Wang, H. Wang, L. Chen, Z. Yang, G. Chen, H. Alinejad- Rokny, H. Li, Y. Lin, and M. Yang FlowPIE: test-time scientific idea evolution with flow-guided literature exploration. arXiv preprint arXiv:2603.29557. Cited by: Introduction, Search and Evolution over Idea Spaces..
• Wang et al. (2024b) W. Wang, L. Gu, L. Zhang, Y. Luo, Y. Dai, C. Shen, L. Xie, B. Lin, X. He, and J. Ye SciPIP: an llm-based scientific paper idea proposer. arXiv preprint arXiv:2410.23166. Cited by: Structured State and Graph-Based Edit Control..
• Xiong et al. (2024a) G. Xiong, E. Xie, A. H. Shariatmadari, S. Guo, S. Bekiranov, and A. Zhang Improving scientific hypothesis generation with knowledge grounded large language models. arXiv preprint arXiv:2411.02382. Cited by: Introduction, Introduction, Structured State and Graph-Based Edit Control..
• Xiong et al. (2024b) G. Xiong, E. Xie, A. H. Shariatmadari, S. Guo, S. Bekiranov, and A. Zhang Improving scientific hypothesis generation with knowledge grounded large language models. External Links: 2411.02382, Link Cited by: Introduction.
• Yamada et al. (2025) Y. Yamada, R. T. Lange, C. Lu, S. Hu, C. Lu, J. Foerster, J. Clune, and D. Ha The ai scientist-v2: workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066. Cited by: Search and Evolution over Idea Spaces..
• Yang et al. (2026) J. Yang, F. Ji, Z. Lai, Z. Cui, G. Ouyang, Q. Jiang, F. Zhang, M. Peng, Q. Xie, P. Nakov, and Z. Xie LabGuard: grounding natural-language laboratory rules into runtime guards for embodied laboratory agents. External Links: 2606.31045, Link Cited by: Introduction.
• Yao et al. (2023) S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan Tree of thoughts: deliberate problem solving with large language models. Advances in neural information processing systems 36, pp. 11809–11822. Cited by: Search and Evolution over Idea Spaces..
• Yuksekgonul et al. (2026) M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, et al. Learning to discover at test time. arXiv preprint arXiv:2601.16175. Cited by: Introduction.
• Zhang (2025) Q. Zhang Knowledge-driven scientific large language models. Proceedings of the AAAI Conference on Artificial Intelligence 39 (27), pp. 28736–28736. External Links: Document, ISSN 2159-5399, Link Cited by: Introduction.
• Zhao et al. (2025) K. Zhao, W. Lin, Q. Zheng, F. Xu, and Y. Li Deep ideation: designing llm agents to generate novel research ideas on scientific concept network. arXiv preprint arXiv:2511.02238. Cited by: Introduction.

注意转载授权 · 阅读原文