Blind Variation, Selective Retention: Improving Creative Idea Generation in AI Scientists

   Jiaru Zou Affiliation: Stanford University^*Co-first Authors    Mete Erdogan Affiliation: Stanford University^*Co-first Authors    Haotian Ye Affiliation: Stanford University^*Co-first Authors    Pan Lu Affiliation: Stanford University^*Co-first Authors    James Zou Affiliation: Stanford University^*Co-first Authors

Abstract

Scientific discovery requires ideas that are original, useful, and non-obvious. Yet AI scientists are typically given as much task information as possible during idea generation, which can anchor generation toward familiar approaches. Inspired by the cognitive framework of Blind Variation and Selective Retention (BVSR), we propose a discovery framework that separates these two roles of information: it generates proposals under deliberately limited task information to broaden exploration, then learns from proposals previously verified on the same task and their outcomes to retain promising candidates for costly empirical evaluation. We evaluate BVSR across five proposer models and 12 scientific tasks. Limiting task information substantially increases proposal originality and non-obviousness, and produces a larger share of highly creative and high-utility proposals than existing state-of-the-art prompting-based baselines. Learning from prior proposal outcomes on the task further improves candidate selection, discarding half of the candidates while retaining most top-performing proposals. When incorporated into existing AI-scientist workflows, BVSR achieves stronger discoveries across gene expression analysis, drug response prediction, anomaly detection, scaling law discovery, and brain activity prediction. These results suggest that effective AI scientists should use information differently across stages of discovery: constrain task-specific information during exploration while exploiting accumulated empirical experience on the task.

1 Introduction

As large language models (LLMs) have made substantial gains in reasoning and long-horizon task execution, a growing body of work explores deploying AI agents for scientific discovery [24, 11]. Existing studies demonstrate that AI agents can outperform the best human-designed solutions across a wide range of tasks, including mathematical proof [14], mathematical discovery [33], optimization [29], and a subset of scientific problems [57]. However, strong performance alone does not establish the creativity of the proposed methods. Prior work has identified limitations in the novelty of AI-generated proposals, including a tendency to explore incremental modifications or extensions of existing approaches [54, 15]. A key challenge is therefore to develop AI agents that can reliably generate proposals that are creative. Addressing this challenge requires both assessing creativity rigorously and developing methods to improve it.

Since the 1950s, psychologists have systematically studied how creativity should be judged in humans [44, 34]. Their work identifies three defining properties of creative ideas: novelty, effectiveness, and non-obviousness [39]. These properties distinguish creativity from performance alone: an idea must work, but its originality and whether its success could have been anticipated also matter. This perspective is captured by a quantitative definition [40]:

where p is the probability of generating an idea, u is its tested utility, and v represents prior knowledge of its effectiveness. Originality captures whether an idea is uncommon rather than something that many others would propose. Utility reflects whether the idea actually works in practice. Non-obviousness concerns whether its effectiveness is known in advance. In this paper, we adopt this definition to assess the creativity of AI-generated proposals.

To improve the creativity of AI agents, prior work has relied mainly on prompt engineering, retrieval augmentation, and a variety of search methods: instructing the agent to be novel [37], supplying it with literature as inspiration [50], or iteratively refining and selecting among many candidates according to search heuristics [20]. Each of these approaches, however, has serious limitations. Prompt engineering yields diminishing returns on frontier models [49]. Literature is a valuable reference, but the literature an agent retrieves for itself suffers from the same lack of diversity as its own proposals [38, 13]. Heuristic search methods rely on a fine-grained signal, whereas most scientific problems offer at best a coarse metric of how well a finished proposal works [52].

Motivated by these limitations, we draw on the theory of Blind Variation (BV) and Selective Retention (SR) to guide how AI agents generate and select proposals [5]. This theory describes creative discovery as a process involving two complementary stages. The first is blind variation: scientists generate diverse ideas without foreknowledge of which will succeed, sometimes drawing inspiration from experiences unrelated to the immediate problem [42, 9]. The second is selective retention: based on prior experience, they discard ideas that are clearly unreasonable and retain plausible ones for further investigation. Similarly, we find that in AI agents, providing too much task information during proposal generation causes the model to anchor on familiar approaches. As illustrated in Figure 1, our method on blind variation first limits the task information available during generation. We then use selective retention to learn, as a scientist does by working on a problem, from proposals already verified on that task and their empirical outcomes to identify promising candidates for further evaluation.

We first show that limiting task information during proposal generation can improve creativity. We provide an intuition for its effect on originality under information-induced anchoring, and validate its benefits empirically across five models and 12 scientific tasks. Across these settings, BV outperforms state-of-the-art prompting-based baselines in originality and non-obviousness, while generating a larger share of highly creative and useful proposals. We then develop SR to learn from previously verified proposals and their empirical outcomes, using a pairwise objective that separates high-utility proposals from the rest. SR retains top proposals more effectively than prompting-based methods, in-context learning, and supervised fine-tuning. Finally, we integrate BVSR into SimpleTES [56], an existing common adopted AI scientist workflow, where it largely improves discovery performance over the original baselines.

2 Preliminaries

An AI scientist attempts to solve an automated scientific discovery task T by producing n candidate methods. Each candidate c is produced in two stages. An ideation stage reads a problem context I (possibly only part of the full problem description S) and returns an idea together with a reference paper supporting it. An implementation stage then reads S and the idea and writes the code. Both stages are driven by instruction prompts. We may vary the prompts and the context I. The task provides an evaluator that runs a candidate’s code and returns a score sT​(c).

Measuring creativity. Following Equation 1, each candidate is scored as creativity⁡(c)=(1−p)​u​(1−v), with the three terms computed as follows.

Originality 1−p. p⁡(c) is the probability that the ideation stage, run again under the same context I and prompt, produces the idea of c:

where c′ is an independent candidate from the same generator and c′≡c denotes that the two express the same idea. In practice, we use an LLM to partition the n candidates into groups expressing the same idea and estimate p⁡(c) by |g⁡(c)|/n, where g⁡(c) is the group containing c. Because 1−p tends to be closer to 1 when n is large, we place originality on a log scale and define it as 1−log⁡|g⁡(c)|/log⁡n. Here, originality refers specifically to generative rarity, following the creativity literature, rather than novelty relative to the scientific literature.

Utility u. We run the candidate’s code under the task evaluator and normalize its improvement over the task’s baseline score sTbase by the best improvement achieved by any candidate on T in our experiments, sTbest:

Non-obviousness 1−v. v⁡(c) measures how much evidence existed at proposal time that the idea would work. An LLM judge reads S and the idea, and outputs v⁡(c), where v=1 means prior experience strongly suggests the mechanism will succeed, and v=0 means its success cannot readily be inferred from existing knowledge.

We assess the reliability of LLM judgments using PhD-level human experts. Human validation of both semantic grouping used for originality and non-obviousness is provided in Section 5.4.

Information level. We vary the information provided in context I and quantify how much it reveals about the problem description S using 𝒱-usable information [53, 7], a model-dependent analogue of mutual information:

where H𝒱​(S) and H𝒱​(S∣I) are the unconditional and conditional 𝒱-entropies of S, respectively. They measure the uncertainty about S for predictors in 𝒱 without and with context I. In practice, we estimate these quantities using the negative log-likelihood of the same problem description S under a language model, first with an empty context and then with I, using base-2 logarithms to express both values in bits. Their difference measures how much easier S becomes to predict when I is provided. See Appendix A for details.

3 Blind Variation: Limited Information Induces More Creativity

In this section, we show that providing a model with too much task information during proposal generation steers it toward common, less novel approaches. This substantially reduces the originality and non-obviousness of its proposals and, in turn, limits the utility of its top proposals. As illustrated in Figure 1(a), blind variation limits task information during idea generation to encourage exploration beyond familiar approaches.

3.1 Motivation

Recall that during proposal generation, the model reads a problem context I and returns an idea along with a reference paper. We study how proposal creativity varies with the amount of information contained in I. While additional task information can strengthen task relevance, it also aligns independent generations around the same task-specific signal and reduces exploration. We characterize this effect in the following proposition.

Proposition 3.1 (Information–Exploration Tradeoff).

Under Assumption C.1, the expected originality of generated proposals is non-increasing with task information:

Under a fixed generation budget, if higher-utility ideas are less associated with the shared task signal, increasing task information decreases the expected utility of the best generated proposal.

The assumptions and derivation are provided in Appendix C.1.

3.2 Experimental Setup

To test this empirically, we use Claude Opus 5 to summarize the full problem statement S, producing contexts I with varying amounts of information for the same task. Appendix E shows example contexts. We then evaluate proposal creativity under each context. Further details on the models, benchmarks, and settings are provided in Appendix B. For each task, each model, and each information level, we generate 112 proposals.

Benchmarks. We evaluate 12 tasks across four scientific domains. (1) Biology. We use four single-cell analysis tasks [25], namely cell-cell communication, label projection, perturbation prediction, and spatially variable genes. (2) Astrodynamics. We use the Cassini [30] and Voyager 2 [18] trajectory design problems. (3) Machine Learning. We use unsupervised anomaly detection and post-hoc probability calibration from MLS Bench [27], along with scaling-law discovery in pretraining and U-shaped / double-descent regime from SLDBench [22]. (4) Neuroscience. We use neural activity forecasting from ZAPBench [26] with prediction horizons of 16 and 32.

Models. We evaluate five mainstream LLMs to assess whether our findings generalize across model families. These include DeepSeek-V4-Flash, DeepSeek-V4-Pro [6], GLM-5.3 Flash, GLM-5.2 [10], and Kimi-K3 [48]. All models have reasoning enabled and access to OpenAlex [31] for literature search.

3.3 Findings

Figure 2 shows how creativity, utility, originality, and non-obviousness change as the amount of task information increases. Each dot represents the proposals generated by one model for one benchmark at one information level.

Since AI scientist tasks place particular value on highly creative and highly useful methods, we first examine how top creativity and top utility vary with information level. We focus on proposals ranked in the top 1% within each benchmark, according to creativity and utility, respectively. The vertical position of each dot indicates a relative frequency, computed as the fraction of its proposals that fall into this top 1%, divided by 1%. We find that models generally produce a larger share of these top-ranked proposals at lower information levels.

We next examine whether originality and non-obviousness follow a similar pattern. We plot mean originality and mean non-obviousness against information level and find that both decline substantially as the amount of task information increases. Across proposals generated by five models on 12 tasks, these results provide systematic evidence that increasing task information reduces the relative frequency of highly creative proposals, supporting the conclusion that limited information can induce greater creativity. We call this approach of improving model creativity by limiting the information provided Blind Variation (BV).

3.4 Baseline Comparison

Prior work improves AI-scientist creativity by modifying prompt instructions. We compare against three such baselines: analogical reasoning, cross-domain search, and no-domain search [37]. We further add one control: BV with the full task description, which separates the effect of limited information from that of our prompt. All methods share the same proposer models, candidate budget, and reasoning configuration. For each model and task, we generate 112 proposals and report in Table 1 the top creativity and top utility, defined here as the mean creativity and utility of the top 1% of proposals, together with mean originality and mean non-obviousness.

BV achieves the highest creativity, utility, and non-obviousness. On the same tasks, the full task description costs the same prompt a large part of its advantage over the baselines: creativity falls from 0.530 to 0.462 and non-obviousness from 0.713 to 0.609, indicating that limiting task information is a major source of BV’s gains.

4 Selective Retention: Learning from Scientific Experience

Blind variation deliberately reduces task-specific guidance during idea generation, but scientific discovery should not ignore what is learned by working on the task. Just as scientists develop judgment about a problem by testing ideas on it, we formulate selective retention as learning from experience on the task itself. Each proposal previously verified on the task provides a pair consisting of a candidate and its empirical outcome. From these proposal–outcome pairs, we learn a scoring function that decides which new candidates for that task should be retained for expensive execution. As illustrated in Figure 1(b), selective retention (SR) uses this learned judgment to reduce the number of candidates requiring empirical verification while preserving promising proposals.

4.1 Motivation

Verification can be time-consuming in many scientific tasks: wet-lab experiments may take months, and machine learning experiments may require substantial GPU computation [43]. A natural approach is to prompt an LLM to inspect candidates. However, as our comparison below shows, these prompting-based approaches offer limited preservation of top candidates. This motivates learning a screener rather than relying exclusively on prompted judgments. The learning objective must also reflect the purpose of screening. Pointwise supervised fine-tuning can train a model to predict each candidate’s performance, but its regression loss does not explicitly prioritize the preservation of top candidates. Selective retention has a more targeted requirement: high-value candidates should remain above the discard region. Precise prediction and ordering are secondary. We therefore focus on separating the top proposals from the remaining ones. We first argue that identifying clearly ineffective methods is much easier than identifying the most effective ones.

Proposition 4.1 (Selective Retention Advantage).

Under the mild Utility-Gap Assumption C.2, if ΔSR​(d)>Δbest, then a comparison across the selective-retention boundary is more accurate than distinguishing the best candidate from its closest competitor. Formally, we have

We then argue SR reduces the verification time required to achieve the same performance, provided that its probability of discarding top proposals is significantly lower than that of random discarding.

Proposition 4.2 (Safe Discard Threshold).

Let d∈[0,1] denote the overall discard ratio and βq​(d) the expected fraction of top-q proposals discarded by SR. For a tolerated loss ϵ, define

Then dq⋆​(ϵ) is the largest discard ratio that limits the expected loss of top-q proposals to at most ϵ, and therefore minimizes the number of proposals requiring verification under this constraint.

The assumptions and derivations for both propositions are provided in Appendices C.2 and C.3.

4.2 Learning to Select Top Proposals

We implement selective retention using a language model with a scalar scoring head. Given a task specification S and a candidate x, the model produces a score s=fθ​(S,x). For a training run, let 𝒯 contain its top-5% utility candidates. We include all candidates in 𝒯 and uniformly sample candidates ℬ from the remainder. We apply a pairwise logistic loss [3], restricted to comparisons between these two groups:

Each comparison can be interpreted as a unit of scientific experience: given two previous proposals, the model learns which one was more worthy of empirical investment. Aggregating many such comparisons trains the selector to exploit past outcomes when screening its future candidates.

Experimental Setup. To evaluate the effectiveness of our method, we use 5,783 proposals generated by the five proposer models on the 12 tasks in Section 3. Our training dataset consists of a separate set of 40,733 proposals on the same tasks. Implementation details are provided in Appendix B.4. To assess whether the model can identify low-quality proposals, we ask it to rank the candidates and discard those ranked lowest. We vary the discard ratio and measure the fraction of top candidates kept. Higher curves indicate better preservation. All screeners receive the same task specification and candidate implementation. We compare SR with four zero-shot prompting-based judges: Pointwise judge assigns absolute scores, Setwise judge repeatedly identifies weak candidates in groups, Listwise judge aggregates sliding-window rankings in the style of RankGPT [45], and Tournament judge uses six rounds of Swiss-system pairwise comparisons [38]. The in-context learning (ICL) judge also sees four similar verified proposals and their outcomes from the SR training set. We also compare with SFT using the same backbone, trained with mean-squared error.

Findings. Figure 3 shows that learned SR preserves top candidates much more effectively than prompting-based screeners. The in-context learning baseline, despite seeing verified outcomes of similar proposals, performs no significantly better than zero-shot judges. SFT improves preservation over many prompt-based methods. Nevertheless, it remains less effective than SR. This result is consistent with the motivation that accurate prediction throughout the distribution is a different objective from keeping exceptional candidates out of the discard set. At a discard ratio of 50%, SR preserves 79% of the top-5% proposals by creativity and 82% of the top-5% proposals by utility. We therefore use this SR model to discard the lowest-ranked 50% of proposals in later experiments.

5 Applications

5.1 Cross-field Discovery

Beyond the gains of BV and SR individually, we evaluate whether combining information-limited generation with experience-based selection improves end-to-end scientific discovery. We evaluate its effectiveness by integrating it into SimpleTES [56]. We compare SimpleTES with and without BVSR on five tasks introduced earlier: gene expression analysis (spatially variable genes), drug response prediction (perturbation prediction), anomaly detection, scaling law discovery, and brain activity prediction (ZAPBench horizon 16). We use Kimi-K3 for both configurations. Following the SimpleTES setup, we run four parallel search trajectories, each generating 40 proposals per iteration, and run for 10 iterations. Figure 4 plots the best score against the cumulative number of empirically verified proposals. BVSR achieves higher final scores on all five tasks and exceeds the original workflow’s final score with fewer verification steps. These results show that BVSR can improve discovery performance when incorporated into existing AI scientist workflows.

5.2 Case Study

We describe three example best programs found by BVSR in Table 2. Each surpasses the best program found by SimpleTES under the same budget and the strongest existing baselines on their tasks.

Spatially variable genes via the nudged elastic band. The task is to rank genes by how strongly their expression is organized in space on a tissue section. The program borrows the nudged elastic band method from transition-state theory [17], which finds a minimum-energy path by relaxing a chain of images under two projected forces: the true force perpendicular to the path and a spring force along it. It takes the heat-flow trajectory of each gene’s expression field on the tissue graph, from the observed field to the uniform one, and computes at every image the component of a robust energy gradient perpendicular to the path. For pure noise, the heat-flow path is already the minimum-energy path, and this component vanishes; spatial patterns bend the path, so the perpendicular force grows with pattern strength and becomes the gene’s score. It reaches 0.918, against 0.906 for SimpleTES and 0.853 for SPARK-X, the best method on OpenProblems.

Anomaly detection via finite-momentum pairing. The task asks for an anomaly detector with at most 10,000 parameters, scored on four tabular datasets. The program transplants the nonuniform (FFLO) superconducting state [21], in which two species with mismatched Fermi surfaces still pair by giving each pair a nonzero momentum, so that the order parameter oscillates in space. Its two “species” are two scores that often disagree: marginal tail evidence from empirical distribution functions, and geometric evidence from a robust covariance and a Gaussian mixture of the normal mass. Instead of averaging them, it pairs them with an oscillating modulation along the direction in which they disagree most, relaxes the recovered pairing signal on a nearest-neighbour graph and adds it to the combined score. With no trained network it reaches 0.679, against 0.629 for SimpleTES and 0.491 for Isolation Forest.

Scaling law via assignment-game prices. The task is to find a formula with at most 26 parameters for the final training loss as a function of learning rate, batch size, data size and model size. The program draws on the assignment game of Shapley and Shubik [36], whose optimal matching is a linear program with competitive prices as dual variables. It solves one fixed, entropy-regularized assignment problem between two grids in the (log learning rate, log batch size) plane and uses the resulting price field as a fixed surface that reshapes the loss basin near the optimal learning rate and batch size. The law adds power laws in data and model size and a convex bowl around a scale-dependent optimum, and fits only two parameters. It reaches a 0.967 held-out R2.

5.3 Ablation Study

To assess the individual contributions of BV and SR, we compare four configurations: the original SimpleTES, SimpleTES with BV alone, SimpleTES with SR alone, and the full BVSR. We evaluate all four configurations on gene expression analysis and scaling law discovery, and track the best task score against verification steps. Figure 5.3 shows that BV and SR each improve the final score over SimpleTES, and that combining them works best on both tasks. On gene expression analysis, BV mainly raises the final score and SR mainly speeds up discovery, while BVSR achieves a larger gain than either. On scaling law discovery, BVSR again reaches a higher score earlier than BV or SR. Because BVSR improves upon the stronger component on each task, the two components are complementary.

Figure 5: Ablation study. Best task score versus verification steps for four SimpleTES configurations. Higher is better.

5.4 Human Validation

We conduct two human studies to validate the LLM judgments underlying originality and non-obviousness. The former relies on grouping semantically equivalent proposals to estimate their frequencies, while the latter assesses whether prior knowledge suggests that a proposal will work. Detailed protocols are provided in Appendix F.

Originality. Five PhD-level researchers each annotate 30 proposal pairs, yielding 150 pairs in total. The sample is balanced between pairs grouped together and pairs assigned to different groups by the LLM, and covers multiple tasks, proposer models, and generation methods. Evaluators judge whether the proposals express the same underlying mechanism. 84% of pairs grouped together by the LLM are also judged equivalent by humans (95% CI [0.74,0.92]), while 73% of pairs assigned to different groups are judged distinct (95% CI [0.61,0.82]). These results support the semantic-equivalence judgments used to estimate originality.

Non-obviousness. Five PhD-level researchers evaluate 150 high-quality proposals from BV and Analogical Reasoning (AR) under blinded conditions. All sampled proposals achieve verified scores above the best baseline for their respective tasks. Evaluators see the task name and summaries of the source paper and proposed method. The summaries are rewritten in a uniform style, and method identifiers are removed. Evaluators assess how readily a proposal’s effectiveness could be inferred from the paper and their own knowledge of the literature, using a 1–5 scale with higher scores indicating greater non-obviousness. Human ratings correlate with the automatic non-obviousness score 1−v^ at r=0.56 (95% CI [0.45,0.67]). BV proposals receive a mean rating of 3.64, compared with 2.97 for AR, a difference of 0.67 points (95% CI [0.33,0.99], p=0.0006). These results support the automatic non-obviousness score and show that BV’s advantage persists under expert judgment.

5.5 Domains of Inspiration

During proposal generation, models search for papers as sources of inspiration. To examine which scientific domains contribute more to top proposals, we sample 20,000 proposals and group them by the domains of their reference papers. For creativity and utility separately, we compute the fraction of proposals from each domain that enter their respective task’s top 1%, divided by the corresponding fraction across all proposals. A value above 1 indicates an above-average likelihood of producing a top proposal. Figure 6 shows that mathematics has the highest normalized fraction for top creativity (1.33), while biology / medicine and mathematics lead in top utility (1.25 and 1.09).

6 Related Work

AI Scientists. Recent work increasingly connects language models to scientific workflows. Coscientist integrates research planning with laboratory automation, while Agent Laboratory and Virtual Lab organize research through specialized agents and human guidance [2, 35, 46]. The AI Scientist extends automation across ideation, experimentation, and manuscript preparation [24]. AlphaEvolve and SimpleTES organize proposal generation and execution [29, 56]. These systems provide capable discovery workflows, but their effectiveness also depends on the candidates they explore. Our work studies how task information shapes proposal generation and how empirical outcomes from previous proposals can guide which new candidates are selected for verification.

Creativity in Humans. Associative theories explain creativity through connections between previously remote concepts [28, 8]. Studies show that familiar categories and example solutions can constrain subsequent ideas [51, 16]. Research on incubation further suggests that temporarily setting a problem aside can improve creative problem solving [41]. Idea selection presents a separate challenge: people often favor feasible and desirable candidates at the expense of originality [32]. Together with Campbell’s blind-variation-and-selective-retention theory [5], these findings help explain the relationship between information and creativity. We investigate whether reducing task information during ideation can encourage exploration in AI scientists.

Creativity in AI. Computational creativity distinguishes combinational, exploratory, and transformational processes [1]. Recent work evaluates LLM creativity through divergent-thinking tasks and the diversity of repeated generations [19, 59]. Beyond text generation, Creative Agents incorporates imagination into embodied planning for open-ended construction tasks [4]. Methods for diversifying outputs include verbalized sampling [58], which elicits distributions over responses, and analogical reasoning [37], which transfers relational structures across scientific domains. Our approach adds task information level as a distinct control for creative generation.

7 Conclusion

We studied how information should be used at different stages of automated scientific discovery. Our results suggest that more information is not uniformly beneficial. During idea generation, detailed task information can anchor language models toward familiar mechanisms and reduce the probability of generating creative proposals. Blind Variation mitigates this effect by limiting task information. After generation, empirical experience earned on the task can guide which new candidates deserve expensive verification. Selective Retention learns this decision from proposal–outcome data. Together, these findings suggest a broader design principle for AI scientists: separate the information used for exploration from the experience used for selection.

References

• [1] M. A. Boden (1998) Creativity and artificial intelligence. Artificial Intelligence 103 (1), pp. 347–356. Note: Artificial Intelligence 40 years later External Links: ISSN 0004-3702, Document, Link Cited by: §6.
• [2] D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes (2023) Autonomous chemical research with large language models. Nature 624 (7992), pp. 570–578. External Links: ISSN 1476-4687, Document, Link Cited by: §6.
• [3] C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender (2005) Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning, ICML ’05, New York, NY, USA, pp. 89–96. External Links: ISBN 1595931805, Link, Document Cited by: §4.2.
• [4] P. Cai, C. Zhang, Y. Fu, H. Yuan, and Z. Lu (2025) Creative agents: empowering agents with imagination for creative tasks. In The 41st Conference on Uncertainty in Artificial Intelligence, External Links: Link Cited by: §6.
• [5] D. Campbell (1960) Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological Review 67, pp. 380–400. External Links: Document Cited by: §1, §6.
• [6] DeepSeek-AI, A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, C. Lu, C. Zhao, C. Deng, C. Hou, C. Xu, C. Shao, C. Ruan, C. Sun, D. Dai, D. Guo, D. Yang, D. Chen, D. Li, D. Ji, E. Li, F. Wei, F. Lin, F. Yuan, F. Xia, F. Dai, G. Hao, G. Chen, G. Cao, G. Meng, G. Li, H. Yu, H. Zhang, H. Xu, H. Li, H. Liang, H. Zhang, H. Luo, H. Wei, H. Yuan, H. Zhang, H. Luo, H. Chen, H. Ji, H. Zhang, H. Ding, H. Tang, H. Cao, H. Gao, H. Qu, H. Zeng, J. Yang, J. Zhu, J. Luo, J. Song, J. Yu, J. Huang, J. Cai, J. Liang, J. Zhou, J. Ye, J. Li, J. Xu, J. Hu, J. Yang, J. Chen, J. Yan, J. Chen, J. Zhou, J. Xiang, J. Yuan, J. Cheng, J. Zhou, J. Zhu, J. Yu, J. Sun, J. Ran, J. Jiang, J. Qiu, J. Li, J. Zheng, J. Song, K. Dong, K. Gao, K. Guan, K. Zhou, K. Huang, K. Yu, L. Wang, L. Zhang, L. Wang, L. Xia, L. Zhang, L. Zhao, L. Guo, L. Luo, L. Ma, L. Zhu, L. Wang, L. Cai, L. Zhang, L. Chen, M. Di, M. Xu, M. Mei, M. Wang, M. Zhang, M. Zhang, M. Tang, M. Li, M. Zhou, M. Han, N. Wang, P. Huang, P. Wang, P. Cong, P. Wang, P. Zhang, Q. Wang, Q. Zhu, Q. Li, Q. Chen, Q. Du, Q. Jiang, R. Tian, R. Xu, R. Lu, R. Xu, R. Ge, R. Zhang, R. Pan, R. Wang, R. Chen, R. Yin, R. Xu, R. Shen, R. Zhang, R. Chen, S. Liu, S. Lu, S. Sun, S. Zhou, S. Chen, S. Cai, S. Nie, S. Wu, S. Chen, S. Hu, S. Liu, S. Hu, S. Ma, S. Wang, S. Yu, S. Zhou, S. Pan, S. Yu, S. Zhou, T. Ni, T. Yun, T. Jin, T. Pei, T. Ye, T. Lin, T. Ji, T. Cui, T. Yue, T. Yu, T. Wang, W. Zhang, W. Xiao, W. Zeng, W. An, W. Zhao, W. Liu, W. Liang, W. Pang, W. Luo, W. Yao, W. Gao, W. Yang, W. Huang, W. Hou, W. Zhang, W. Ma, X. Gao, X. He, X. Wang, X. Wang, X. Bi, X. Liu, X. Wang, X. Chen, X. Zhang, X. Nie, X. Sun, X. Wang, X. Cheng, X. Liu, X. Xie, X. Liu, X. Liu, X. Yu, X. Li, X. Yang, X. Zhang, X. Chen, X. Wang, X. Su, X. Chen, X. Lin, X. Fu, Y. Yan, Y. Wang, Y. Ma, Y. Luo, Y. Zhang, Y. Xu, Y. Ma, Y. Huang, Y. Li, Y. Li, Y. Xu, Y. Zhao, Y. Sun, Y. Wang, Y. Qian, Y. Shao, Y. Yu, Y. Zhang, Y. Ding, Y. Shi, Y. Wu, Y. Xiong, Y. Ma, Y. He, Y. Tang, Y. Zhou, Y. Luo, Y. Zhong, Y. Piao, Y. Wang, Y. Zhang, Y. Chen, Y. Tan, Y. Wei, Y. Ma, Y. Liu, Y. Yang, Y. Guo, Y. Wu, Y. Wu, Y. Li, Y. Cheng, Y. Ou, Y. Xu, Y. Li, Y. Wang, Y. Yang, Y. Xu, Y. Wu, Y. Meng, Y. Zou, Y. Zha, Y. Xiong, Y. Chen, Y. Lin, Y. Cao, Y. Wang, Y. Zhang, Y. Yan, Y. Lin, Y. Gu, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. Zhou, Y. Huang, Z. Wu, Z. Wang, Z. Zhao, Z. Ren, Z. Zhang, Z. Sha, Z. Fu, Z. Ju, Z. Xu, Z. Xie, Z. Zhang, Z. Gao, Z. Hao, Z. Gou, Z. Ma, Z. Yan, Z. Shao, Z. Huang, Z. Chen, Z. Wu, Z. Ren, Z. Wu, Z. Li, Z. Zhang, Z. Xu, Z. Wang, Z. Qu, Z. Gu, Z. Zhu, Z. Li, Z. Zhang, Z. Xie, Z. Gao, Z. Wan, Z. Pan, and Z. Yao (2026) DeepSeek-v4: towards highly efficient million-token context intelligence. External Links: 2606.19348, Link Cited by: §3.2.
• [7] K. Ethayarajh, Y. Choi, and S. Swayamdipta (2022) Understanding dataset difficulty with 𝒱-usable information. In Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, pp. 5988–6008. External Links: Link Cited by: §2.
• [8] R. A. Finke, T. B. Ward, and S. M. Smith (1996) Creative cognition: theory, research, and applications. External Links: Link Cited by: §6.
• [9] S. L. Gable, E. A. Hopper, and J. W. Schooler (2019) When the muses strike: creative ideas of physicists and writers routinely occur during mind wandering. Psychological Science 30 (3), pp. 396–404. Note: PMID: 30653407 External Links: Document, Link, https://doi.org/10.1177/0956797618820626 Cited by: §1.
• [10] GLM-5-Team, :, A. Zeng, X. Lv, Z. Hou, Z. Du, Q. Zheng, B. Chen, D. Yin, C. Ge, C. Huang, C. Xie, C. Zhu, C. Yin, C. Wang, G. Pan, H. Zeng, H. Zhang, H. Wang, H. Chen, J. Zhang, J. Jiao, J. Guo, J. Wang, J. Du, J. Wu, K. Wang, L. Li, L. Fan, L. Zhong, M. Liu, M. Zhao, P. Du, Q. Dong, R. Lu, Shuang-Li, S. Cao, S. Liu, T. Jiang, X. Chen, X. Zhang, X. Huang, X. Dong, Y. Xu, Y. Wei, Y. An, Y. Niu, Y. Zhu, Y. Wen, Y. Cen, Y. Bai, Z. Qiao, Z. Wang, Z. Wang, Z. Zhu, Z. Liu, Z. Li, B. Wang, B. Wen, C. Huang, C. Cai, C. Yu, C. Li, C. Hu, C. Zhang, D. Zhang, D. Lin, D. Yang, D. Wang, D. Ai, E. Zhu, F. Yi, F. Chen, G. Wen, H. Sun, H. Zhao, H. Hu, H. Zhang, H. Liu, H. Zhang, H. Peng, H. Tai, H. Zhang, H. Liu, H. Wang, H. Yan, H. Ge, H. Liu, H. Chu, J. Zhao, J. Wang, J. Zhao, J. Ren, J. Wang, J. Zhang, J. Gui, J. Zhao, J. Li, J. An, J. Li, J. Yuan, J. Du, J. Liu, J. Zhi, J. Duan, K. Zhou, K. Wei, K. Wang, K. Luo, L. Zhang, L. Sha, L. Xu, L. Wu, L. Ding, L. Chen, M. Li, N. Lin, P. Ta, Q. Zou, R. Song, R. Yang, S. Tu, S. Yang, S. Wu, S. Zhang, S. Li, S. Li, S. Fan, W. Qin, W. Tian, W. Zhang, W. Yu, W. Liang, X. Kuang, X. Cheng, X. Li, X. Yan, X. Hu, X. Ling, X. Fan, X. Xia, X. Zhang, X. Zhang, X. Pan, X. Zou, X. Zhang, Y. Liu, Y. Wu, Y. Li, Y. Wang, Y. Zhu, Y. Tan, Y. Zhou, Y. Pan, Y. Zhang, Y. Su, Y. Geng, Y. Yan, Y. Tan, Y. Bi, Y. Shen, Y. Yang, Y. Li, Y. Liu, Y. Wang, Y. Li, Y. Wu, Y. Zhang, Y. Duan, Y. Zhang, Z. Liu, Z. Jiang, Z. Yan, Z. Zhang, Z. Wei, Z. Chen, Z. Feng, Z. Yao, Z. Chai, Z. Wang, Z. Zhang, B. Xu, M. Huang, H. Wang, J. Li, Y. Dong, and J. Tang (2026) GLM-5: from vibe coding to agentic engineering. External Links: 2602.15763, Link Cited by: §3.2.
• [11] J. Gottweis, W. Weng, A. Daryin, T. Tu, P. Sirkovic, A. Myaskovsky, G. Glowaty, F. Weissenberger, A. Orlandi, D. Popovici, A. Palepu, K. Rong, R. Tanno, K. Saab, F. Zhang, J. Blum, A. Carroll, K. Kulkarni, N. Tomašev, D. Zverinski, I. Rendulic, E. Vedadi, F. Hasler, L. Rimanic, M. Boia, I. Budiselic, B. Feinstein, M. Bellaiche, T. Sheffer, J. Freyberg, J. Ratcliff, O. Bertolli, K. Chou, A. Hassidim, B. Gokturk, A. Vahdat, Y. Guan, V. Dhillon, E. D. Vaishnav, B. Lee, T. R. D. Costa, J. R. Penadés, G. Peltz, Y. Matias, J. Manyika, D. Hassabis, Y. Xu, P. Kohli, A. Pawlosky, A. Karthikesalingam, and V. Natarajan (2026) Accelerating scientific discovery with co-scientist. Nature 655 (8122), pp. 487–496. External Links: ISSN 1476-4687, Document, Link Cited by: §1.
• [12] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru, B. Roziere, B. Biron, B. Tang, B. Chern, C. Caucheteux, C. Nayak, C. Bi, C. Marra, C. McConnell, C. Keller, C. Touret, C. Wu, C. Wong, C. C. Ferrer, C. Nikolaidis, D. Allonsius, D. Song, D. Pintz, D. Livshits, D. Wyatt, D. Esiobu, D. Choudhary, D. Mahajan, D. Garcia-Olano, D. Perino, D. Hupkes, E. Lakomkin, E. AlBadawy, E. Lobanova, E. Dinan, E. M. Smith, F. Radenovic, F. Guzmán, F. Zhang, G. Synnaeve, G. Lee, G. L. Anderson, G. Thattai, G. Nail, G. Mialon, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. Zarov, I. A. Ibarra, I. Kloumann, I. Misra, I. Evtimov, J. Zhang, J. Copet, J. Lee, J. Geffert, J. Vranes, J. Park, J. Mahadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong, J. Lee, J. Fu, J. Chi, J. Huang, J. Liu, J. Wang, J. Yu, J. Bitton, J. Spisak, J. Park, J. Rocca, J. Johnstun, J. Saxe, J. Jia, K. V. Alwala, K. Prasad, K. Upasani, K. Plawiak, K. Li, K. Heafield, K. Stone, K. El-Arini, K. Iyer, K. Malik, K. Chiu, K. Bhalla, K. Lakhotia, L. Rantala-Yeary, L. van der Maaten, L. Chen, L. Tan, L. Jenkins, L. Martin, L. Madaan, L. Malo, L. Blecher, L. Landzaat, L. de Oliveira, M. Muzzi, M. Pasupuleti, M. Singh, M. Paluri, M. Kardas, M. Tsimpoukelli, M. Oldham, M. Rita, M. Pavlova, M. Kambadur, M. Lewis, M. Si, M. K. Singh, M. Hassan, N. Goyal, N. Torabi, N. Bashlykov, N. Bogoychev, N. Chatterji, N. Zhang, O. Duchenne, O. Çelebi, P. Alrassy, P. Zhang, P. Li, P. Vasic, P. Weng, P. Bhargava, P. Dubal, P. Krishnan, P. S. Koura, P. Xu, Q. He, Q. Dong, R. Srinivasan, R. Ganapathy, R. Calderer, R. S. Cabral, R. Stojnic, R. Raileanu, R. Maheswari, R. Girdhar, R. Patel, R. Sauvestre, R. Polidoro, R. Sumbaly, R. Taylor, R. Silva, R. Hou, R. Wang, S. Hosseini, S. Chennabasappa, S. Singh, S. Bell, S. S. Kim, S. Edunov, S. Nie, S. Narang, S. Raparthy, S. Shen, S. Wan, S. Bhosale, S. Zhang, S. Vandenhende, S. Batra, S. Whitman, S. Sootla, S. Collot, S. Gururangan, S. Borodinsky, T. Herman, T. Fowler, T. Sheasha, T. Georgiou, T. Scialom, T. Speckbacher, T. Mihaylov, T. Xiao, U. Karn, V. Goswami, V. Gupta, V. Ramanathan, V. Kerkez, V. Gonguet, V. Do, V. Vogeti, V. Albiero, V. Petrovic, W. Chu, W. Xiong, W. Fu, W. Meers, X. Martinet, X. Wang, X. Wang, X. E. Tan, X. Xia, X. Xie, X. Jia, X. Wang, Y. Goldschlag, Y. Gaur, Y. Babaei, Y. Wen, Y. Song, Y. Zhang, Y. Li, Y. Mao, Z. D. Coudert, Z. Yan, Z. Chen, Z. Papakipos, A. Singh, A. Srivastava, A. Jain, A. Kelsey, A. Shajnfeld, A. Gangidi, A. Victoria, A. Goldstand, A. Menon, A. Sharma, A. Boesenberg, A. Baevski, A. Feinstein, A. Kallet, A. Sangani, A. Teo, A. Yunus, A. Lupu, A. Alvarado, A. Caples, A. Gu, A. Ho, A. Poulton, A. Ryan, A. Ramchandani, A. Dong, A. Franco, A. Goyal, A. Saraf, A. Chowdhury, A. Gabriel, A. Bharambe, A. Eisenman, A. Yazdan, B. James, B. Maurer, B. Leonhardi, B. Huang, B. Loyd, B. D. Paola, B. Paranjape, B. Liu, B. Wu, B. Ni, B. Hancock, B. Wasti, B. Spence, B. Stojkovic, B. Gamido, B. Montalvo, C. Parker, C. Burton, C. Mejia, C. Liu, C. Wang, C. Kim, C. Zhou, C. Hu, C. Chu, C. Cai, C. Tindal, C. Feichtenhofer, C. Gao, D. Civin, D. Beaty, D. Kreymer, D. Li, D. Adkins, D. Xu, D. Testuggine, D. David, D. Parikh, D. Liskovich, D. Foss, D. Wang, D. Le, D. Holland, E. Dowling, E. Jamil, E. Montgomery, E. Presani, E. Hahn, E. Wood, E. Le, E. Brinkman, E. Arcaute, E. Dunbar, E. Smothers, F. Sun, F. Kreuk, F. Tian, F. Kokkinos, F. Ozgenel, F. Caggioni, F. Kanayet, F. Seide, G. M. Florez, G. Schwarz, G. Badeer, G. Swee, G. Halpern, G. Herman, G. Sizov, Guangyi, Zhang, G. Lakshminarayanan, H. Inan, H. Shojanazeri, H. Zou, H. Wang, H. Zha, H. Habeeb, H. Rudolph, H. Suk, H. Aspegren, H. Goldman, H. Zhan, I. Damlaj, I. Molybog, I. Tufanov, I. Leontiadis, I. Veliche, I. Gat, J. Weissman, J. Geboski, J. Kohli, J. Lam, J. Asher, J. Gaya, J. Marcus, J. Tang, J. Chan, J. Zhen, J. Reizenstein, J. Teboul, J. Zhong, J. Jin, J. Yang, J. Cummings, J. Carvill, J. Shepard, J. McPhie, J. Torres, J. Ginsburg, J. Wang, K. Wu, K. H. U, K. Saxena, K. Khandelwal, K. Zand, K. Matosich, K. Veeraraghavan, K. Michelena, K. Li, K. Jagadeesh, K. Huang, K. Chawla, K. Huang, L. Chen, L. Garg, L. A, L. Silva, L. Bell, L. Zhang, L. Guo, L. Yu, L. Moshkovich, L. Wehrstedt, M. Khabsa, M. Avalani, M. Bhatt, M. Mankus, M. Hasson, M. Lennie, M. Reso, M. Groshev, M. Naumov, M. Lathi, M. Keneally, M. Liu, M. L. Seltzer, M. Valko, M. Restrepo, M. Patel, M. Vyatskov, M. Samvelyan, M. Clark, M. Macey, M. Wang, M. J. Hermoso, M. Metanat, M. Rastegari, M. Bansal, N. Santhanam, N. Parks, N. White, N. Bawa, N. Singhal, N. Egebo, N. Usunier, N. Mehta, N. P. Laptev, N. Dong, N. Cheng, O. Chernoguz, O. Hart, O. Salpekar, O. Kalinli, P. Kent, P. Parekh, P. Saab, P. Balaji, P. Rittner, P. Bontrager, P. Roux, P. Dollar, P. Zvyagina, P. Ratanchandani, P. Yuvraj, Q. Liang, R. Alao, R. Rodriguez, R. Ayub, R. Murthy, R. Nayani, R. Mitra, R. Parthasarathy, R. Li, R. Hogan, R. Battey, R. Wang, R. Howes, R. Rinott, S. Mehta, S. Siby, S. J. Bondu, S. Datta, S. Chugh, S. Hunt, S. Dhillon, S. Sidorov, S. Pan, S. Mahajan, S. Verma, S. Yamamoto, S. Ramaswamy, S. Lindsay, S. Lindsay, S. Feng, S. Lin, S. C. Zha, S. Patil, S. Shankar, S. Zhang, S. Zhang, S. Wang, S. Agarwal, S. Sajuyigbe, S. Chintala, S. Max, S. Chen, S. Kehoe, S. Satterfield, S. Govindaprasad, S. Gupta, S. Deng, S. Cho, S. Virk, S. Subramanian, S. Choudhury, S. Goldman, T. Remez, T. Glaser, T. Best, T. Koehler, T. Robinson, T. Li, T. Zhang, T. Matthews, T. Chou, T. Shaked, V. Vontimitta, V. Ajayi, V. Montanez, V. Mohan, V. S. Kumar, V. Mangla, V. Ionescu, V. Poenaru, V. T. Mihailescu, V. Ivanov, W. Li, W. Wang, W. Jiang, W. Bouaziz, W. Constable, X. Tang, X. Wu, X. Wang, X. Wu, X. Gao, Y. Kleinman, Y. Chen, Y. Hu, Y. Jia, Y. Qi, Y. Li, Y. Zhang, Y. Zhang, Y. Adi, Y. Nam, Yu, Wang, Y. Zhao, Y. Hao, Y. Qian, Y. Li, Y. He, Z. Rait, Z. DeVito, Z. Rosnbrick, Z. Wen, Z. Yang, Z. Zhao, and Z. Ma (2024) The llama 3 herd of models. External Links: 2407.21783, Link Cited by: Appendix A, Appendix A.
• [13] X. Hu, H. Fu, J. Wang, Y. Wang, Z. Li, R. Xu, Y. Lu, Y. Jin, L. Pan, and Z. Lan (2025) NOVA: an iterative planning framework for enhancing scientific innovation with large language models. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, pp. 21330–21359. External Links: Link, Document, ISBN 979-8-89176-256-5 Cited by: §1.
• [14] T. Hubert, R. Mehta, L. Sartran, M. Z. Horváth, G. Žužić, E. Wieser, A. Huang, J. Schrittwieser, Y. Schroecker, H. Masoom, O. Bertolli, T. Zahavy, A. Mandhane, J. Yung, I. Beloshapka, B. Ibarz, V. Veeriah, L. Yu, O. Nash, P. Lezeau, S. Mercuri, C. Sönne, B. Mehta, A. Davies, D. Zheng, F. Pedregosa, Y. Li, I. von Glehn, M. Rowland, S. Albanie, A. Velingker, S. Schmitt, E. Lockhart, E. Hughes, H. Michalewski, N. Sonnerat, D. Hassabis, P. Kohli, and D. Silver (2026) Olympiad-level formal mathematical reasoning with reinforcement learning. Nature 651 (8106), pp. 607–613. External Links: ISSN 1476-4687, Document, Link Cited by: §1.
• [15] P. Jansen, O. Tafjord, M. Radensky, P. Siangliulue, T. Hope, B. Dalvi Mishra, B. P. Majumder, D. S. Weld, and P. Clark (2025) CodeScientist: end-to-end semi-automated scientific discovery with code-based experimentation. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, pp. 13370–13467. External Links: Link, Document, ISBN 979-8-89176-256-5 Cited by: §1.
• [16] D. G. Jansson and S. M. Smith (1991) Design fixation. Design Studies 12 (1), pp. 3–11. External Links: ISSN 0142-694X, Document, Link Cited by: §6.
• [17] H. Jónsson, G. Mills, and K. W. Jacobsen (1998) Nudged elastic band method for finding minimum energy paths of transitions. External Links: Link Cited by: §5.2.
• [18] C. E. Kohlhase and P. A. Penzo (1977) Voyager mission description. Space Science Reviews 21 (2), pp. 77–101. External Links: Document, Link Cited by: §3.2.
• [19] M. Koivisto and S. Grassini (2023) Best humans still outperform artificial intelligence in a creative divergent thinking task. Scientific Reports 13 (1), pp. 13601. External Links: ISSN 2045-2322, Document, Link Cited by: §6.
• [20] R. T. Lange, Y. Imajuku, and E. Cetin (2025) ShinkaEvolve: towards open-ended and sample-efficient program evolution. External Links: 2509.19349, Link Cited by: §1.
• [21] A. I. Larkin and Yu. N. Ovchinnikov (1964) Nonuniform state of superconductors. External Links: Link Cited by: §5.2.
• [22] H. Lin, H. Ye, W. Feng, Q. Huang, Y. Li, H. Lim, Z. Li, X. Wang, J. Ma, Y. Liang, and J. Zou (2026) Can language models discover scaling laws?. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §B.2, §3.2.
• [23] F. T. Liu, K. M. Ting, and Z. Zhou (2008) Isolation forest. 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. External Links: Link Cited by: Table 2.
• [24] C. Lu, C. Lu, R. T. Lange, Y. Yamada, S. Hu, J. Foerster, D. Ha, and J. Clune (2026) Towards end-to-end automation of ai research. Nature 651 (8107), pp. 914–919. External Links: ISSN 1476-4687, Document, Link Cited by: §1, §6.
• [25] M. D. Luecken, S. Gigante, D. B. Burkhardt, R. Cannoodt, D. C. Strobl, N. S. Markov, L. Zappia, G. Palla, W. Lewis, D. Dimitrov, M. E. Vinyard, D. S. Magruder, M. F. Mueller, A. Andersson, E. Dann, Q. Qin, D. J. Otto, M. Klein, O. B. Botvinnik, L. Deconinck, K. Waldrant, S. N. Yasa, A. Szałata, A. Benz, Z. Li, B. Rieck, C. Ahlmann-Eltze, E. da Veiga Beltrame, C. Bravo González-Blas, A. T. Chen, B. DeMeo, C. Ergen, S. Floc’hlay, A. Gayoso, S. Hicks, Y. Ji, V. Kleshchevnikov, G. La Manno, M. G. Lombardo, R. Lopez, D. Righelli, H. Sarkar, V. Svensson, A. Tong, G. Xing, C. Xu, J. M. Bloom, A. O. Pisco, J. Saez-Rodriguez, D. Wulsin, L. Pinello, Y. Saeys, F. J. Theis, S. Krishnaswamy, and O. P. J. Members (2025) Defining and benchmarking open problems in single-cell analysis. Nature Biotechnology 43 (7), pp. 1035–1040. External Links: ISSN 1546-1696, Document, Link Cited by: §B.2, §3.2.
• [26] J. Lueckmann, A. Immer, A. B. Chen, P. H. Li, M. D. Petkova, N. A. Iyer, L. W. Hesselink, A. Dev, G. Ihrke, W. Park, A. Petruncio, A. Weigel, W. Korff, F. Engert, J. Lichtman, M. Ahrens, M. Januszewski, and V. Jain (2025) ZAPBench: a benchmark for whole-brain activity prediction in zebrafish. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §B.2, §3.2.
• [27] B. Lyu, Y. Yang, S. Huang, J. Zhang, Q. Xu, X. Li, X. Han, Y. Zhang, H. Zhang, R. Huang, K. Yang, Z. Chen, W. Guo, J. Yang, X. Ai, W. Chai, Y. Cao, Z. Yang, K. Wang, D. Jiang, H. Gao, S. Tang, C. Shi, S. S. Du, M. Simchowitz, J. Jiao, D. Song, and C. Jin (2026) MLS-bench: a holistic and rigorous assessment of ai systems on building better ai. External Links: 2605.08678, Link Cited by: §B.2, §3.2.
• [28] S. Mednick (1962) The associative basis of the creative process. Psychological Review 69, pp. 220–232. External Links: Document Cited by: §6.
• [29] A. Novikov, N. Vũ, M. Eisenberger, E. Dupont, P. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. R. Ruiz, A. Mehrabian, M. P. Kumar, A. See, S. Chaudhuri, G. Holland, A. Davies, S. Nowozin, P. Kohli, and M. Balog (2025) AlphaEvolve: a coding agent for scientific and algorithmic discovery. External Links: 2506.13131, Link Cited by: §1, §6.
• [30] F. Peralta and S. Flanagan (1995) Cassini interplanetary trajectory design. Control Engineering Practice 3 (11), pp. 1603–1610. External Links: Document, Link Cited by: §3.2.
• [31] J. Priem, H. Piwowar, and R. Orr (2022) OpenAlex: a fully-open index of scholarly works, authors, venues, institutions, and concepts. External Links: 2205.01833, Link Cited by: §3.2.
• [32] E. Rietzschel, B. Nijstad, and W. Stroebe (2009) The selection of creative ideas after individual idea generation: choosing between creativity and impact. British journal of psychology (London, England : 1953) 101, pp. 47–68. External Links: Document Cited by: §6.
• [33] B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, et al. (2024) Mathematical discoveries from program search with large language models. Nature 625 (7995), pp. 468–475. Cited by: §1.
• [34] M. A. Runco and G. J. Jaeger (2012) The standard definition of creativity. Creativity Research Journal 24 (1), pp. 92–96. External Links: Document, Link, https://doi.org/10.1080/10400419.2012.650092 Cited by: §1.
• [35] S. Schmidgall, Y. Su, Z. Wang, X. Sun, J. Wu, X. Yu, J. Liu, M. Moor, Z. Liu, and E. Barsoum (2025) Agent laboratory: using LLM agents as research assistants. In Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, pp. 5977–6043. External Links: Link, Document, ISBN 979-8-89176-335-7 Cited by: §6.
• [36] L. S. Shapley and M. Shubik (1971) The assignment game i: the core. International Journal of Game Theory 1, pp. 111–130. External Links: Link Cited by: §5.2.
• [37] A. Shen, S. Druckmann, and J. Zou (2026) Unlocking llm creativity in science through analogical reasoning. External Links: 2605.11258, Link Cited by: §B.3, §1, §3.4, §6.
• [38] C. Si, D. Yang, and T. Hashimoto (2025) Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §4.2.
• [39] D. K. Simonton (2012) Taking the u.s. patent office criteria seriously: a quantitative three-criterion creativity definition and its implications. Creativity Research Journal 24 (2-3), pp. 97–106. External Links: Document, Link, https://doi.org/10.1080/10400419.2012.676974 Cited by: §1.
• [40] D. K. Simonton (2018) Defining creativity: don’t we also need to define what is not creative?. The Journal of Creative Behavior 52 (1), pp. 80–90. External Links: Document, Link, https://onlinelibrary.wiley.com/doi/pdf/10.1002/jocb.137 Cited by: §1.
• [41] U. N. Sio and T. Ormerod (2009) Does incubation enhance problem solving? a meta-analytic review. Psychological Bulletin 135, pp. 94–120. External Links: Document Cited by: §6.
• [42] S. M. Smith, T. B. Ward, and J. S. Schumacher (1993) Constraining effects of examples in a creative generation task. Memory & Cognition 21 (6), pp. 837–845. External Links: ISSN 1532-5946, Document, Link Cited by: §1.
• [43] J. Snoek, H. Larochelle, and R. Adams (2012) Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, F. Pereira, C.J. Burges, L. Bottou, and K. Weinberger (Eds.), Vol. 25, pp. . External Links: Link Cited by: §4.1.
• [44] M. I. Stein (1953) Creativity and culture. The Journal of Psychology 36 (2), pp. 311–322. External Links: Document, Link, https://doi.org/10.1080/00223980.1953.9712897 Cited by: §1.
• [45] W. Sun, L. Yan, X. Ma, S. Wang, P. Ren, Z. Chen, D. Yin, and Z. Ren (2023) Is ChatGPT good at search? investigating large language models as re-ranking agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, pp. 14918–14937. External Links: Link, Document Cited by: §B.4, §4.2.
• [46] K. Swanson, W. Wu, N. L. Bulaong, J. E. Pak, and J. Zou (2025) The virtual lab of ai agents designs new sars-cov-2 nanobodies. Nature 646 (8085), pp. 716–723. External Links: ISSN 1476-4687, Document, Link Cited by: §6.
• [47] G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, L. Rouillard, T. Mesnard, G. Cideron, J. Grill, S. Ramos, E. Yvinec, M. Casbon, E. Pot, I. Penchev, G. Liu, F. Visin, K. Kenealy, L. Beyer, X. Zhai, A. Tsitsulin, R. Busa-Fekete, A. Feng, N. Sachdeva, B. Coleman, Y. Gao, B. Mustafa, I. Barr, E. Parisotto, D. Tian, M. Eyal, C. Cherry, J. Peter, D. Sinopalnikov, S. Bhupatiraju, R. Agarwal, M. Kazemi, D. Malkin, R. Kumar, D. Vilar, I. Brusilovsky, J. Luo, A. Steiner, A. Friesen, A. Sharma, A. Sharma, A. M. Gilady, A. Goedeckemeyer, A. Saade, A. Feng, A. Kolesnikov, A. Bendebury, A. Abdagic, A. Vadi, A. György, A. S. Pinto, A. Das, A. Bapna, A. Miech, A. Yang, A. Paterson, A. Shenoy, A. Chakrabarti, B. Piot, B. Wu, B. Shahriari, B. Petrini, C. Chen, C. L. Lan, C. A. Choquette-Choo, C. Carey, C. Brick, D. Deutsch, D. Eisenbud, D. Cattle, D. Cheng, D. Paparas, D. S. Sreepathihalli, D. Reid, D. Tran, D. Zelle, E. Noland, E. Huizenga, E. Kharitonov, F. Liu, G. Amirkhanyan, G. Cameron, H. Hashemi, H. Klimczak-Plucińska, H. Singh, H. Mehta, H. T. Lehri, H. Hazimeh, I. Ballantyne, I. Szpektor, I. Nardini, J. Pouget-Abadie, J. Chan, J. Stanton, J. Wieting, J. Lai, J. Orbay, J. Fernandez, J. Newlan, J. Ji, J. Singh, K. Black, K. Yu, K. Hui, K. Vodrahalli, K. Greff, L. Qiu, M. Valentine, M. Coelho, M. Ritter, M. Hoffman, M. Watson, M. Chaturvedi, M. Moynihan, M. Ma, N. Babar, N. Noy, N. Byrd, N. Roy, N. Momchev, N. Chauhan, N. Sachdeva, O. Bunyan, P. Botarda, P. Caron, P. K. Rubenstein, P. Culliton, P. Schmid, P. G. Sessa, P. Xu, P. Stanczyk, P. Tafti, R. Shivanna, R. Wu, R. Pan, R. Rokni, R. Willoughby, R. Vallu, R. Mullins, S. Jerome, S. Smoot, S. Girgin, S. Iqbal, S. Reddy, S. Sheth, S. Põder, S. Bhatnagar, S. R. Panyam, S. Eiger, S. Zhang, T. Liu, T. Yacovone, T. Liechty, U. Kalra, U. Evci, V. Misra, V. Roseberry, V. Feinberg, V. Kolesnikov, W. Han, W. Kwon, X. Chen, Y. Chow, Y. Zhu, Z. Wei, Z. Egyed, V. Cotruta, M. Giang, P. Kirk, A. Rao, K. Black, N. Babar, J. Lo, E. Moreira, L. G. Martins, O. Sanseviero, L. Gonzalez, Z. Gleicher, T. Warkentin, V. Mirrokni, E. Senter, E. Collins, J. Barral, Z. Ghahramani, R. Hadsell, Y. Matias, D. Sculley, S. Petrov, N. Fiedel, N. Shazeer, O. Vinyals, J. Dean, D. Hassabis, K. Kavukcuoglu, C. Farabet, E. Buchatskaya, J. Alayrac, R. Anil, Dmitry, Lepikhin, S. Borgeaud, O. Bachem, A. Joulin, A. Andreev, C. Hardin, R. Dadashi, and L. Hussenot (2025) Gemma 3 technical report. External Links: 2503.19786, Link Cited by: Appendix A, Appendix A.
• [48] K. Team, T. Bai, Y. Bai, Y. Bao, M. C., J. Cai, X. Cai, P. Cao, Y. Cao, Z. Chai, Y. Charles, H. S. Che, G. Chen, G. Chen, G. Chen, H. Chen, J. Chen, J. Chen, J. Chen, K. Chen, P. Chen, R. Chen, W. Chen, X. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Z. Chen, D. Cheng, Y. Cheng, J. Cui, J. Cui, A. Dai, J. Deng, H. Ding, R. Ding, S. Ding, M. Dong, M. Dong, Y. Dong, Y. Dong, A. Du, C. Du, D. Du, J. Du, Y. Du, Y. Fan, J. Feng, Q. Feng, Y. Feng, K. Fu, Q. Fu, F. Gao, H. Gao, J. Gao, T. Gao, W. Gao, S. Geng, J. Gong, L. Gong, S. Gong, X. Gong, Q. Gu, Y. Gu, S. Guan, H. Guo, S. Guo, X. Guo, Z. Guo, B. Hao, W. Hao, X. Hao, D. He, H. He, L. He, Q. He, W. He, X. He, X. He, Y. He, Y. He, C. Hong, T. Hong, H. Hu, J. Hu, R. Hu, W. Hu, Y. Hu, Z. Hu, L. Hua, J. Huang, K. Huang, R. Huang, S. Huang, W. Huang, Y. Huang, Z. Huang, Z. Huang, Y. Hui, C. Jia, Y. Jiang, Z. Jiang, Z. Jiang, W. Jin, X. Jin, Y. Jing, H. Kong, G. Lai, A. Li, C. Li, C. Li, C. Li, F. Li, G. Li, H. Li, J. Li, J. Li, L. Li, L. Li, L. Li, W. Li, W. Li, X. Li, Y. Li, Y. Li, Y. Li, Y. Li, Z. Li, Z. Li, Z. Li, Z. Li, Z. Li, J. Lin, X. Lin, Y. Lin, Z. Lin, Z. Lin, B. Liu, B. Liu, C. Liu, L. Liu, S. Liu, S. Liu, S. Liu, T. Liu, W. Liu, Y. Liu, Y. Liu, Y. Liu, Y. Liu, Z. Liu, Z. Liu, E. Lu, H. Lu, L. Lu, T. Lu, Z. Lu, A. Luo, G. Luo, J. Luo, Y. Luo, B. Lyu, W. Lyu, S. Mao, Y. Mei, X. Men, M. Ni, Y. Niu, S. Pan, S. Peng, Z. Qi, R. Qin, Z. Qin, Z. Qin, H. Qiu, J. Qiu, J. Qiu, B. Qu, Y. Qu, Z. Shang, Y. Shao, H. Shen, J. Shi, J. Shi, L. Shi, S. Shi, W. Siu, P. Song, X. Song, J. Su, Y. Su, Z. Su, L. Sui, J. Sun, J. Sun, S. Sun, S. Sun, T. Sun, Y. Sun, Y. Tai, C. Tang, H. Tang, S. Tang, Z. Tang, C. Tian, R. Tian, Y. Tian, W. Tu, C. Wang, C. Wang, C. Wang, D. Wang, F. Wang, H. Wang, H. Wang, H. Wang, H. Wang, H. Wang, H. Wang, J. Wang, J. Wang, J. Wang, J. Wang, L. Wang, S. Wang, S. Wang, S. Wang, S. Wang, S. Wang, T. Wang, W. Wang, X. Wang, X. Wang, X. Wang, X. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, C. Wei, M. Wei, S. Wei, Z. Wen, F. Wu, H. Wu, R. Wu, W. Wu, X. Wu, Y. Wu, Y. Wu, Y. Wu, Z. Wu, X. Xian, C. Xiang, Y. Xiang, B. Xiao, C. Xiao, X. Xiao, J. Xie, X. Xie, Y. Xie, Z. Xie, B. Xing, Y. Xiong, B. Xu, B. Xu, J. Xu, J. Xu, J. Xu, J. Xu, L. H. Xu, Q. Xu, S. Xu, S. Xu, T. Xu, T. Xu, W. Xu, X. Xu, Y. Xu, Y. Xu, Y. Xu, Z. Xu, H. Xue, J. Yan, Y. Yan, F. Yang, G. Yang, H. Yang, J. Yang, R. Yang, W. Yang, X. Yang, X. Yang, Y. Yang, Y. Yang, Y. Yang, Y. Yang, Z. Yang, Z. Yang, Z. Yang, Z. Yang, H. Yao, D. Ye, H. Ye, W. Ye, Z. Ye, B. Yin, H. Yin, X. Yin, C. Yu, H. Yu, L. Yu, S. Yu, S. Yu, T. Yu, E. Yuan, M. Yuan, T. Yue, W. Yue, Y. Yue, D. Zha, H. Zhan, B. H. Zhang, D. Zhang, F. Zhang, H. Zhang, H. Zhang, H. Zhang, J. Zhang, J. Zhang, J. Zhang, K. Zhang, M. Zhang, P. Zhang, Q. Zhang, R. Zhang, R. Zhang, S. Zhang, S. Zhang, X. Zhang, X. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Z. Zhang, Z. Zhang, B. Zhao, C. Zhao, F. Zhao, J. Zhao, J. Zhao, S. Zhao, W. Zhao, X. Zhao, X. Zhao, Y. Zhao, Z. Zhao, H. Zheng, H. Zheng, R. Zheng, S. Zheng, T. Zheng, H. Zhong, L. Zhong, L. Zhong, M. Zhou, Q. Zhou, R. Zhou, R. Zhou, X. Zhou, Y. Zhou, Z. Zhou, J. Zhu, L. Zhu, X. Zhu, Y. Zhu, Y. Zhu, Z. Zhu, C. Zhuang, W. Zhuang, and X. Zu (2026) Kimi k3: open frontier intelligence. External Links: 2607.24653, Link Cited by: §3.2.
• [49] G. Wang, Z. Sun, S. Ye, Z. Gong, Y. Chen, Y. Zhao, Q. Liang, and D. Hao (2026) Do advanced language models eliminate the need for prompt engineering in software engineering?. ACM Trans. Softw. Eng. Methodol. 35 (8). External Links: ISSN 1049-331X, Link, Document Cited by: §1.
• [50] Q. Wang, D. Downey, H. Ji, and T. Hope (2024) SciMON: scientific inspiration machines optimized for novelty. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, pp. 279–299. External Links: Link, Document Cited by: §1.
• [51] T.B. Ward (1994) Structured imagination: the role of category structure in exemplar generation. Cognitive Psychology 27 (1), pp. 1–40. External Links: ISSN 0010-0285, Document, Link Cited by: §6.
• [52] G. Xu, Z. Qi, H. Su, W. Ye, H. Lakkaraju, S. M. Kakade, and Y. Du (2026) Self-improving language models with bidirectional evolutionary search. In 3rd AI for Math Workshop: Toward Self-Evolving Scientific Agents, External Links: Link Cited by: §1.
• [53] Y. Xu, S. Zhao, J. Song, R. Stewart, and S. Ermon (2020) A theory of usable information under computational constraints. In International Conference on Learning Representations, External Links: Link Cited by: Appendix A, §2.
• [54] Y. Yamada, R. T. Lange, C. Lu, S. Hu, C. Lu, J. Foerster, J. Clune, and D. Ha (2025) The ai scientist-v2: workshop-level automated scientific discovery via agentic tree search. External Links: 2504.08066, Link Cited by: §1.
• [55] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025) Qwen3 technical report. External Links: 2505.09388, Link Cited by: Appendix A, Appendix A, §B.4.
• [56] H. Ye, H. Lin, J. Tang, Y. Luo, R. Thapa, C. Yang, C. Su, R. Yang, R. Liu, R. Li, Z. Li, P. Sun, C. Gao, D. Ding, G. He, M. Zhang, L. Sun, W. Wang, Y. Zhong, Z. Shen, P. Li, P. Lu, B. Cui, D. He, J. Ma, J. Li, H. Baoyin, Y. Choi, S. Ermon, X. Chu, T. Li, Y. Xu, and J. Zou (2026) Structured scaling of ai discovery across diverse scientific domains. External Links: 2604.19341, Link Cited by: §1, §5.1, §6.
• [57] M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, and Y. Sun (2026) Learning to discover at test time. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1.
• [58] J. Zhang, S. Yu, D. Chong, A. Sicilia, M. Tomz, C. D. Manning, and W. Shi (2026) Verbalized sampling: how to mitigate mode collapse and unlock LLM diversity. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §6.
• [59] Y. Zhang, H. Diddee, S. Holm, H. Liu, X. Liu, V. Samuel, B. Wang, and D. Ippolito (2025) NoveltyBench: evaluating creativity and diversity in language models. In Second Conference on Language Modeling, External Links: Link Cited by: §6.
• [60] J. Zhu, S. Sun, and X. Zhou (2021) SPARK-x: non-parametric modeling enables scalable and robust detection of spatial expression patterns for large spatial transcriptomic studies. Genome Biology 22 (1), pp. 184. External Links: ISSN 1474-760X, Document, Link Cited by: Table 2.

Appendix - Table of Contents

Appendix A Details about Information Level

By definition, the two 𝒱-entropies in equation 4 are the shortest code lengths that any predictor in 𝒱 achieves for the problem description S, with and without the context I [53]:

Here 𝒱 should be the family of models that actually read the context, i.e. the large language models used as proposers, so the direct estimate would score S with each proposer and take the minimum code length. In practice this is not feasible since the proposers are large mixture-of-experts models, and the log-likelihood such a model assigns to a fixed text is not stable, because it varies with the compute environment.

We therefore use three off-the-shelf small dense models, Qwen3-8B [55], Gemma-3-4B-it [47] and Llama-3.1-8B-Instruct [12], and average their estimates. S is the benchmark’s own description of the task. Each model M reads I followed by S, and we sum the base-2 log-probabilities of the tokens of S, which gives log2⁡PM​(S∣I); log2⁡PM​(S) is obtained from the same text with I left empty. The estimate is

Under a fixed evaluation setup, these small dense models yield reproducible token log-probabilities, enabling stable estimation of information levels. Beyond stability, we also examine whether the estimates depend on the choice of dense models.

To this end, we scored 32 example contexts using 10 open-weight models: our three selected models, along with Llama-3.2-3B-Instruct [12], Mistral-7B-v0.3, Qwen2.5-7B, Qwen3-4B [55], Qwen3.5-4B, Qwen3.5-9B, and Gemma-3-12B-it [47]. We computed the Pearson correlation between each pair of models’ estimates across the contexts.

Figure 7 shows strong agreement across models, with a median Pearson correlation of r=0.91 across the 45 model pairs. Furthermore, the mean estimate from our three selected models correlates with the mean across all 10 models at r=0.99. These results indicate that our estimates are robust to the choice of dense models.

Appendix B Experimental Setup

This section describes the models, the twelve discovery tasks, and the settings of blind variation and selective retention together with every baseline they are compared with. Prompts are collected in Appendix D.

B.1 Models

Proposals are generated by five LLMs: Kimi-K3, GLM-5.2, GLM-5.3-Flash, DeepSeek-V4-Flash and DeepSeek-V4-Pro. Originality and non-obviousness (Section 2) are estimated by two OpenAI models. For originality, GPT-5.6-Sol reads all mechanism sentences and groups those that express similar ideas. For non-obviousness, GPT-5.6-Luna reads the task description, the chosen paper, and the proposal, and returns v^​(x)∈[0,1]. The SR selector is a fine-tuned Qwen3.5-9B-Base model.

B.2 Datasets

We use twelve discovery tasks from five benchmark families (Table 3): four single-cell tasks [25], two tasks from MLS-Bench [27], two scaling-law tasks from SLD-Bench [22], two gravity-assist trajectory tasks, and whole-brain activity forecasting from ZAPBench [26] at horizons 16 and 32. Every task asks for one executable program with a fixed entry point, which the benchmark’s own evaluator runs end to end to produce the score.

B.3 Blind Variation Setup

In blind variation, the model receives limited task information to generate proposals, which are subsequently implemented as code. To vary the information available during proposal generation, we use Claude Opus 5 to generate four contexts with different amounts of task information for each task. Appendix D.1 provides the prompts used in BV, along with the prompt used to generate these contexts with Claude Opus 5.

We compare blind variation with three baselines: analogical reasoning, cross-domain search, and no-domain search, using the official prompts provided by Shen et al. [37]. In no-domain search, the model independently searches for papers it considers most helpful. In cross-domain search, the model is instructed to search for papers across domains, while in analogical reasoning, it is instructed to identify relevant papers through analogy. BV (Full Information) always uses the full task description S as its context.

B.4 Selective Retention Setup

Selective retention ranks the candidates of a pool and discards the lowest-ranked fraction. The ranking model is Qwen3.5-9B-Base [55], fully fine-tuned with a zero-initialized linear head that maps the final hidden state of the last input token to a score fθ, of which only the order is meaningful. Its input is the task description followed by the candidate proposal.

We compare the selector with six ranking baselines. The SFT baseline fine-tunes the same model on the same input but regresses, by mean squared error, the standardized within-run percentile rank of the verified score.

We also include four LLM judge baselines. Here, each cell is ranked by its own proposer. Pointwise scoring asks for an integer from 0 to 100 per candidate, with ties broken at random. The tournament plays six Swiss-system rounds of pairwise comparisons and ranks candidates by points, then by Buchholz score (the sum of the opponents’ points), then at random. Setwise ranking asks which three of ten implementations to discard, over ten random partitions of the cell, and ranks candidates by how rarely they are named. Listwise ranking uses the sliding window of Sun et al. [45] and averages each candidate’s position over three random initial orders.

The in-context learning (ICL) baseline gives the judge access to the same training dataset as SR. For each test candidate, we give four similar proposals from the SR training set, which covers the same tasks and is disjoint from the test set. Each demonstration shows the implementation and its within-run percentile rank of the verified score. The judge then scores the test candidate from 0 to 100 as in Pointwise.

You are deciding how worth it is to spend an expensive evaluation on ONE candidate IMPLEMENTATION for a task. It is real code that will be executed on the task’s data and scored by the task’s metric. Judge how likely the implementation is to (1) run without error and produce a valid output of the required form, and (2) score well on the task’s metric.

TASK SPECIFICATION (data/inputs provided, required output, metric):

<task specification>

IMPLEMENTATION:

“‘python

<candidate code>

“‘

Read the implementation carefully. Consider: does it correctly consume the provided data/inputs; does it actually compute something aligned with the task’s objective (not a trivial constant, a degenerate output, or leakage of the target values); is it likely to run without crashing; and does its approach plausibly beat the baseline. Judge the ACTUAL code, not the comments’ claims. Ignore length/verbosity.

First THINK briefly about the points above (a few sentences). THEN give your FINAL ANSWER as one JSON object in a “‘json code fence:

“‘json

{"score": 73}

“‘

where "score" is an integer from 0 (certain to fail or score poorly) to 100 (certain to run and score among the best), using the full range so that different implementations get different scores.

You are deciding which of TWO candidate IMPLEMENTATIONS is more worth spending an expensive evaluation on, for the SAME task. Each is real code that will be executed on the task’s data and scored by the task’s metric. Judge which implementation is more likely to (1) run without error and produce a valid output of the required form, and (2) score well on the task’s metric.

TASK SPECIFICATION (data/inputs provided, required output, metric):

<task specification>

IMPLEMENTATION A:

“‘python

<code of candidate A>

“‘

IMPLEMENTATION B:

“‘python

<code of candidate B>

“‘

Read both implementations carefully. Consider: does it correctly consume the provided data/inputs; does it actually compute something aligned with the task’s objective (not a trivial constant, a degenerate output, or leakage of the target values); is it likely to run without crashing; and does its approach plausibly beat the baseline. Judge the ACTUAL code, not the comments’ claims. Ignore length/verbosity and which one is labelled A or B.

First THINK briefly, comparing the two implementations directly on the points above (a few sentences).

THEN give your FINAL ANSWER as one JSON object in a “‘json code fence:

“‘json

{"winner": "A"}

“‘

where "winner" is exactly "A" or "B".

You are screening candidate IMPLEMENTATIONS of the SAME task before an expensive evaluation step. Each is real code that will be executed on the task’s data and scored by the task’s metric. Some of them will be DISCARDED without evaluation. Pick the ones that are LEAST likely to (1) run without error and produce a valid output of the required form, and (2) score well on the task’s metric.

TASK SPECIFICATION (data/inputs provided, required output, metric):

<task specification>

IMPLEMENTATION [1]:

“‘python

<code of candidate 1>

“‘

…

IMPLEMENTATION [N]:

“‘python

<code of candidate N>

“‘

Read every implementation carefully. Consider: does it correctly consume the provided data/inputs; does it actually compute something aligned with the task’s objective (not a trivial constant, a degenerate output, or leakage of the target values); is it likely to run without crashing; and does its approach plausibly beat the baseline. Judge the ACTUAL code, not the comments’ claims. Ignore length/verbosity and the order in which they are listed.

First THINK briefly (a few sentences). THEN give your FINAL ANSWER as one JSON object in a “‘json code fence with labels:

“‘json

{"discard": [4, 9, 2]}

“‘

You are ranking candidate IMPLEMENTATIONS of the SAME task by how worth it is to spend an expensive evaluation on each. Each is real code that will be executed on the task’s data and scored by the task’s metric. Rank them by how likely each is to (1) run without error and produce a valid output of the required form, and (2) score well on the task’s metric.

TASK SPECIFICATION (data/inputs provided, required output, metric):

<task specification>

IMPLEMENTATION [1]:

“‘python

<code of candidate 1>

“‘

…

IMPLEMENTATION [N]:

“‘python

<code of candidate N>

“‘

Read every implementation carefully. Consider: does it correctly consume the provided data/inputs; does it actually compute something aligned with the task’s objective (not a trivial constant, a degenerate output, or leakage of the target values); is it likely to run without crashing; and does its approach plausibly beat the baseline. Judge the ACTUAL code, not the comments’ claims. Ignore length/verbosity and the order in which they are listed.

First THINK briefly (a few sentences). THEN give your FINAL ANSWER as one JSON object in a “‘json code fence, listing ALL labels from BEST to WORST:

“‘json

{"ranking": [3, 1, 7, …]}

“‘

Appendix C Intuition on BVSR

C.1 Information-Induced Anchoring and Proposal Originality

Assumption C.1 (Shared-Signal Generation).

Let b=info⁡(I) and 𝒵={z1,…,zK} denote a fixed set of semantic idea classes. Each idea zi has a task-association score ai. For generation t, the model selects:

where εt,i are i.i.d. Gumbel random variables and λ⁡(b)≥0 is differentiable and non-decreasing, i.e., λ′​(b)≥0. Thus, additional task information increases the influence of the shared task signal relative to generation-specific exploratory variation.

Under Assumption C.1, the induced probability of generating idea zi is

Thus, increasing task information strengthens the common task signal shared across generations relative to independent exploratory variation.

Proposition C.1.

Under Assumption C.1, the expected originality of generated proposals is non-increasing with task information:

Moreover, for a fixed generation budget n, when higher-utility ideas are less strongly associated with the shared task signal, increasing task information also decreases the expected utility of the best generated proposal.

Proof.

The proof proceeds in three steps. We first derive the proposal distribution induced by shared-signal generation. We then show that stronger task information reduces expected originality. Finally, we characterize when the same concentration effect reduces the expected utility of the best proposal under a fixed generation budget.

Induced proposal distribution.

Under Assumption C.1, generation t selects Zt​(b)=arg⁡maxi​{λ⁡(b)​ai+εt,i}. By the Gumbel-max identity, the probability of selecting idea zi is

which means increasing b strengthens the shared task signal relative to generation-specific variation.

Effect on originality.

Consider n>1 independently generated proposals. Let Ni denote the number of proposals belonging to semantic idea class zi, with empirical frequency π^i=Ni/n. For any proposal c in class zi, we have |g⁡(c)|=Ni, so its log-scaled originality is o(c)=1−logNi/logn=−logπ^i/logn. Averaging over all proposals gives

where H(π^)=−∑i=1Kπ^ilogπ^i is the empirical entropy of the semantic idea distribution. It therefore suffices to show that the expected empirical entropy is non-increasing in b.

Without loss of generality, order the task-association scores as a1≥a2≥⋯≥aK. For any r<K, let Sr​(λ)=∑i=1rπb​(zi) denote the probability assigned to the r most task-associated ideas. Differentiating Eq. 10 gives

The inequality holds since the first r ideas have no smaller association scores than the rest. Hence, increasing λ shifts probability mass toward the most task-associated ideas, so πb becomes increasingly concentrated in the sense of majorization.

We next show that this concentration cannot increase the expected empirical entropy. Since (N1,…,NK) follows a multinomial distribution with probabilities πb, each marginal satisfies Ni∼Binomial⁡(n,πb​(zi)). Define hn​(p)=𝔼N∼Binomial⁡(n,p)​[−Nn​log⁡Nn]. Then

The function hn is the Bernstein polynomial of the concave function h⁡(x)=−x​log⁡x, and is therefore concave. Consequently, 𝔼⁡[H⁡(π^)] is symmetric and concave in πb, and hence Schur-concave. Since increasing λ makes πb more concentrated, dd​λ​𝔼​[H⁡(π^)]≤0. Together with λ′​(b)≥0, the chain rule gives

Combining this with Eq. 13 and log⁡n>0 yields

Effect on the best proposal.

We finally characterize how the same concentration affects discovery quality. Let each semantic idea class zi have utility ui, and reorder the classes such that u1≤u2≤⋯≤uK. Suppose higher-utility ideas are weakly less associated with the shared task signal, so that a1≥a2≥⋯≥aK. Let Fj​(λ)=∑i=1jπb​(zi) denote the probability of generating an idea with utility at most uj. By the same argument as Eq. 14,

Let Mn​(b)=maxt=1,…,n⁡u⁡(Zt​(b)) denote the utility of the best idea among n independent generations. Since Pr⁡(Mn​(b)≤uj)=Fj​(λ)n, its expectation can be written as

Differentiating with respect to λ gives

Since λ′​(b)≥0, we therefore obtain

Thus, stronger task information reduces expected proposal originality, and when higher-utility ideas are less aligned with the shared task signal, it also reduces the expected utility of the best proposal within a fixed generation budget. ∎

C.2 Utility-Gap Advantage in Selective Retention

Assumption C.2 (Utility-Gap Comparison).

For any pair of candidates with utilities u⁡(ci) and u⁡(cj), let Δ=|u⁡(ci)−u⁡(cj)| denote their utility gap. We assume that the probability of correctly ordering the pair, Pcorrect​(Δ), is strictly increasing in Δ.

Thus, candidates with larger utility gaps are easier to distinguish.

Proposition C.2.

Let the n candidates be ordered by utility as u⁡(c(1))≤⋯≤u⁡(c(n)). For a discard ratio d, let m=⌊d​n⌋ and define the utility gap across the selective-retention boundary as ΔSR​(d)=u⁡(c(m+1))−u⁡(c(m)). We further define the gap between the best candidate and its closest competitor as Δbest=u⁡(c(n))−u⁡(c(n−1)). Under Assumption C.2, if ΔSR​(d)>Δbest, then

Proof.

The proof follows directly from the utility-gap comparison assumption. Selective retention only needs to separate candidates across the discard boundary. At discard ratio d, the closest such pair is c(m) and c(m+1), whose utility gap is ΔSR​(d)=u⁡(c(m+1))−u⁡(c(m)). In contrast, identifying the best candidate requires distinguishing c(n) from its closest utility competitor c(n−1), with gap Δbest=u⁡(c(n))−u⁡(c(n−1)).

By Assumption C.2, Pcorrect​(Δ) is strictly increasing in Δ. Therefore, whenever ΔSR​(d)>Δbest, we have

Thus, when the utility separation at the selective-retention boundary is larger than the separation among the top candidates, screening across the retention boundary is more accurate than identifying the single best candidate. ∎

C.3 Safe Discarding under Selective Retention

Proposition C.3.

Let d denote the overall discard ratio and βq​(d) the expected fraction of top-q proposals discarded by selective retention. For a tolerated loss ϵ, define

Then dq⋆​(ϵ) is the largest discard ratio that limits the expected loss of top-q proposals to at most ϵ, and therefore minimizes the number of proposals requiring verification under this constraint.

Proof.

For a fixed ranking of n proposals, let 𝒟⁡(d) denote the set of proposals discarded at ratio d. Since selective retention discards proposals from the bottom of the same ranking, for any d1≤d2, 𝒟⁡(d1)⊆𝒟⁡(d2). Hence, the expected fraction of top-q proposals discarded, βq​(d), is non-decreasing in d.

For a tolerated loss ϵ, define the feasible set ℱq​(ϵ)={d:βq​(d)≤ϵ}. By definition, dq⋆​(ϵ) is the largest feasible discard ratio.¹¹ 1 For a finite set of candidates, discard ratios lie on a finite grid, so the supremum is attained. Therefore,

At discard ratio d, n⁡(1−d) proposals remain for verification. Thus, choosing dq⋆​(ϵ) minimizes the number of proposals requiring verification while keeping the expected loss of top-q proposals within ϵ. ∎

Appendix D Prompt Templates

D.1 Blind Variation

Prompt  is the proposal prompt. {task_context} is the task context. Every ideation turn also carries the literature-search tool.

=== TOOLS ===

[

{

"type": "function",

"function": {

"name": "search_literature",

"description": "Search the scientific literature (all disciplines) for real, published papers. Returns title, authors, year, citation count and abstract for each hit. Call it as many times as useful, refining your query between calls.",

"parameters": {

"type": "object",

"properties": {

"query": {

"type": "string",

"description": "search query (keywords or a phrase)"

},

"n": {

"type": "integer",

"description": "how many papers to return (default 6, max 15)"

}

},

"required": [

"query"

]

}

}

}

]

=== USER MESSAGE ===

{task_context}Find ONE real paper from the field {field} whose single core mechanism is most promising to transplant into a different algorithm.

Use the search_literature tool to find real papers in this field. Search as many times as you find useful, refining your queries as you learn what exists.

Then reply EXACTLY in this format:

PAPER: <authors, title, year>

MECHANISM: <its ONE core mechanism, in one sentence>

ONE paper, ONE mechanism - do NOT combine several. Do NOT write any code.

Prompt  is to implement the proposal. {task_interface}, {baseline_pool} and {task_guidance} form the full task specification, and {output_format} the code the task expects.

{task_interface}

{baseline_pool}

# YOUR JOB

PAPER: {paper}

MECHANISM: {mechanism}

Implement a REAL transplant of THAT ONE mechanism. ONE mechanism only; do NOT add other tricks.{task_guidance}

OUTPUT FORMAT (exactly):

“‘python

{output_format}

“‘

D.2 Selective Retention

The SR selector and the SFT regressor read the same input, Prompt , including {task_specification} and the candidate’s complete code. Both map this input to a single score.

### TASK

{task_specification}

### IMPLEMENTATION

{code}

### PREDICTED PERCENTILE OF VERIFIED SCORE:

Appendix E Examples of Contexts under Different Information Levels

The information levels differ only in the task context at the start of the proposal generation message (Prompt ). Prompts – show example contexts for two tasks.

Context: the samples sit on a 2-D lattice, and the goal is to rank features by how much of their variation is organized across the lattice rather than noise.

Context: this is spatially-variable-gene detection in spatial transcriptomics - ranking genes by whether their expression varies in a spatially structured way across a tissue section.

CONTEXT: You are helping improve a procedure that reads a short stretch of many simultaneously recorded numeric signals and continues each of them forward for 16 further steps.

CONTEXT: The procedure sees the last four time steps of tens of thousands of simultaneously recorded signals and must predict the next 16 steps for every one of them jointly. It is judged by mean absolute error against held-out continuations, averaged over windows, over the forecast steps and over all signals.

Appendix F Human Validation Protocols

This section details the human evaluation protocols used in Section 5.4.

F.1 Originality

Our originality measure depends on grouping proposals that express the same underlying scientific or algorithmic idea. To validate this semantic-equivalence step, we construct a set of 150 proposal pairs, balanced between pairs grouped together and pairs assigned to different groups by the LLM evaluator. The pairs are sampled across tasks, proposer models, and generation methods.

Five PhD-level researchers each evaluate 30 pairs. For each pair, the evaluator judges whether the two proposals express essentially the same underlying mechanism, ignoring differences in wording and implementation details. Pairs marked Unsure (4 in total) are excluded from the agreement analysis. An example evaluation sheet is provided in Appendix G.1.

F.2 Non-obviousness

We sample 150 high-quality proposals from BV and AR, defined as proposals whose verified score exceeds the best baseline of their task. We remove all method identifiers and ask five PhD-level researchers to evaluate their non-obviousness.

For each proposal, evaluators are shown the task name, a summary of the source paper, and a summary of the proposed method. Both summaries are rewritten by Claude Opus 5 into a uniform style to reduce cues that could reveal the generating method. Evaluators rate non-obviousness on a 1–5 scale, considering how readily the method’s effectiveness on the task could be inferred from the source paper and their own knowledge of the literature. Higher scores indicate greater non-obviousness.

Proposals are split into forms of 30, each balanced across tasks and methods and presented in random order. Each evaluator completes one form. An example evaluation sheet is provided in Appendix G.2.

Appendix G Human Evaluation Example Form

G.1 Human Evaluation Example Form of Originality

Each page gives the task name and the one-sentence mechanism descriptions of two proposals written for that task, followed by one multiple-choice question: do the two proposals express essentially the same underlying scientific or algorithmic mechanism (Yes / No / Unsure)? Both proposals of a pair come from the same task, the method that produced them and whether the grouping placed them together are not shown, and pairs appear in random order. Figure 8 shows the first pair page of one form.

G.2 Human Evaluation Example Form of Non-Obviousness

Evaluators rate proposals in a Google Form. The form opens with a short instruction page and one field for the evaluator’s identifier; it then shows one proposal per page, 30 pages in total. Each page gives the task name, a summary of the paper the proposal draws on, and a summary of the proposed method, followed by a 1–5 question on non-obviousness. The method that produced each proposal is not shown, proposals appear in random order, and summaries are written by Claude Opus 5 in one uniform style so that the wording does not reveal the method. Figure 9 shows the first page of one form.

注意转载授权 · 阅读原文