Skip to main content

Assured SI for Autonomous Biology

Full Campaign Name:

Trust, Assuredness, and Validation of Super Intelligence (SI)-Driven Autonomous Biological Laboratories

Swirls of blues and purples depict a half-circle depicting experimental loops

Self-driving laboratories (autonomous systems that design, run, and interpret their own experiments) can now propose candidate discoveries faster than scientists can verify them. The machine learning models steering these experimental loops are often most confident exactly where they are wrong, and that failure stays invisible until budget and samples have already been spent. In the biological sciences, where every experiment is costly and consequential, acting on an automated decision that no one has independently checked is a growing risk.

The Environmental Molecular Sciences Laboratory (EMSL) has developed its own self-driving laboratory, the Anaerobic Microbial Phenotyping Platform (AMP2), and will begin construction this year on a larger platform: the Microbial Molecular Phenotyping Capability (M2PC). These capabilities are composed of connected instruments and a growing set of machine learning and large language model (LLM)-driven tools that read the literature and propose and execute experiments. Building for platforms like these, this campaign leverages EMSL's expertise in verification, validation, and uncertainty quantification to apply and assess an independent assurance layer: diagnostics that measure whether the decisions made in an experimental loop can be trusted and computed on real lab records, together with an EMSL-drafted, community-ratified standard for what an autonomous experimental loop must record to be auditable.

Through this Computing, Analytics, and Modeling community science campaign, EMSL will partner with research community members to independently assess the trustworthiness of autonomous and LLM-driven workflows, contextualize the results with others' experimental data, and together define the emerging standards that make autonomous discovery assured rather than merely fast.

The diagnostics, trace standards, and reference base developed for this campaign will be released for community adoption, giving EMSL users of biological autonomous laboratories a way to certify, and later continuously monitor, the autonomous workflows they depend upon. This will provide the beginnings of a rigorous trust suite that can be used to evaluate, escalate, flag, or stop experimental loops as they run and provide assurance for cross-facility data sharing. These evaluative techniques will be built to interoperate with EMSL's existing shared evaluation, tracing, and reporting abilities, and they may be adapted by external users for their own autonomous applications.

Campaign participants will:

  • Help solve a big science problem
  • Drive important outcomes
  • Advance their own research
     

Participation

Who should participate?

Researchers with relevant domain expertise, experience, and overlapping interests with autonomous laboratories for biological science and the instrumentation of assuredness for Super Intelligence (SI)-driven systems are encouraged to submit a proposal for participation in this campaign.

Required participant background

Expertise

An active or recently completed experimental loop in the life sciences that is autonomous, active-learning, or driven by machine learning or an LLM. Examples include microbial media or strain optimization, enzyme or pathway engineering, or cell-culture and bioprocess optimization in which a model selected, ranked, or proposed the experiments to run. Participants will bring deep knowledge of their own organism, assay, and platform. EMSL will bring past data as well as the trust-evaluation methodology and the shared framing.

Data, approved/authorized for sharing

Records from experimental loops that participants are authorized to share that include, for each experimental cycle, the model's numerical predictions and associated predictive uncertainties recorded before measurement (not point outputs alone), the experimental designs the model selected, and the realized measured outcomes resulting from running that experiment. EMSL is also open to LLM-driven workflows that design experiments without predictive uncertainties and intend to perform a separate evaluation of such tools. A common minimal reporting format will be agreed upon so that records are comparable across groups, organisms, and platforms.

Valued but not required: Participants' ability to also provide (or run) a simple baseline (e.g., space filling or random selection) on their respective system, which would enable the examination of decision-quality and regret-based diagnostics.

Collaborative research interests

Participants' willingness to apply EMSL's assurance diagnostics to their own workflow and contribute records to a shared reference, helping define community reporting standards and best-practice guidance for trustworthy autonomous experimentation that their own work could eventually be measured against.

How will participants contribute?

Identify and Prioritize Knowledge Gaps and Needs

Participants will help determine which validation and verification failures matter in practice for autonomous biological laboratories. This will involve determining which diagnostics are decision-relevant for their systems and differentiating between model behaviors that genuinely incur costs for experiments and those that are harmless. Participants will also help identify the misspecifications and artifacts that are realistic for their own organisms/assays, helping ground the community-refined diagnostic suite in real failure modes.

Supply Data

Participants will contribute experimental loop records they are authorized to share (model predictions and predictive uncertainties recorded before measurement, the designs selected, and realized outcomes) as real-world cases that feed the shared reference. EMSL is also open to LLM-driven experimental loops for which data were produced but no predictive uncertainty intervals are available. Each contributed dataset will receive its own "trust" assessment as well as strengthen the reference against which every other participant's results are interpreted.

Expert Input

Participants will bring domain expertise to interpret the assurance readouts for their own workflows, collaborate with EMSL to test and refine the diagnostics, and coauthor the resulting best-practice and reporting guidance for the field.
 

About the campaign

EMSL is a U.S. Department of Energy Office of Science user facility whose mission is to steward scientific capabilities that no single research group can build or maintain alone. EMSL has invested in autonomous experimentation via AMP2 and begins construction this year on M2PC. These capabilities interoperate with a growing set of machine learning and LLM-driven tools that mine the literature and propose, build, and refine experiments. EMSL's Computing, Analytics, and Modeling Science Area includes deep expertise in uncertainty quantification and the verification and validation of scientific models.

Autonomy in the biological sciences is advancing faster than the means of establishing trust in it. Automated decision procedures now select and run experiments at a pace that outstrips scientists' ability to check them, and the models steering these experimental loops are frequently most confident precisely where they are least correct. Yet there is no standardized, independent way to establish whether an autonomous laboratory's decisions can be trusted. As groups move to raise the autonomy of their workflows, that assurance gap is becoming the limiting factor, and the field is positioned to accelerate discovery it cannot yet verify.

This campaign closes that gap by acting as an independent assurance layer. Experimentalists who run autonomous or LLM-driven workflows but lack a dedicated validation, evaluation, or assuredness team will gain a rigorous, independent assessment of decision validity that they can cite in their own publications and proposals. Modelers will gain a shared reference that makes any single result interpretable and a voice in the record-keeping standards that will define auditable autonomy.

This campaign supports the recent presidential memorandum on fiscal year 2027 national research and development priorities (PDF) and demonstrates how SI accelerates scientific discovery within the life sciences and autonomous experimentation.

Campaign timeline

  • November 5, 2026: Call for proposals
    Open call for proposals launches.
     
  • December 1, 2026: Campaign proposal deadline
    Deadline for campaign participants to submit proposals for projects contributing to overall campaign goals.
     
  • December 15, 2026: Initiate work on accepted proposals
    Work begins on accepted proposals for projects contributing to overall campaign goals.
     
  • December 2026 – September 2027: Apply trust and assurance diagnostic workflows
    Community-refined assurance diagnostics are applied to participants' shared campaign records. A trust assessment is returned to each participant, and the cross-campaign reference is built. The run-to-run reliability of LLM-driven experiment-proposing workflows is evaluated.
     
  • September 2027: Complete campaign
    The campaign is finished, with publication of the shared reporting standard, the diagnostic suite, and the cross-campaign reference, together with the community best-practice guidance manuscript coauthored with participants. These resources are made available to the EMSL user community.

Campaign methods

Establish a common record standard

Working with participants, AMP2, and EMSL's Autonomous Analysis and Discovery Integrated Research Platform, the campaign will define a reporting schema for autonomous experimental loops, including defining the additional predictions, uncertainty estimates, decision records, and outcome links these methods require. This common format is what makes records from different organisms, platforms, and acquisition strategies comparable. These recorded fields and diagnostic methods can be adapted by users and incorporated into EMSL's existing evaluation, tracing, and reporting abilities and be used to escalate, flag, or stop autonomous experimental loops as they progress.

Compute trust diagnostics directly from autonomous experimental records

A curated set of lightweight diagnostics will score each experimental loop from its trace alone, without access to the underlying model and with no need for refitting or simulation. These diagnostics will assess whether a model's stated uncertainty holds up where the autonomous laboratory spent its budget, whether its recommendations carry genuine decision value, and whether its behavior stays stable as evidence accumulates.

Establish and test comparability across experiments

Many diagnostics are dimensionless, but what a given value implies across different assays, objectives, experiment sizes, and selection strategies is not yet understood. EMSL will establish and test comparability using randomized resampling methods appropriate for adaptive, self-selected data, and the required assumptions for those methods will be explicitly implemented for each study. EMSL will then compare diagnostic results across participants' experimental loops to learn which trust behaviors are robust to differing designs, distinguishing statistically typical from scientifically acceptable results.

Measure LLM-orchestrator reliability

For the growing class of LLM-based tools that read the literature and propose experiments or translate between protocols and code, the campaign will quantify run-to-run consistency. Delivered as a lightweight, reusable harness, this will be a focused reliability measurement tool rather than a general model-evaluation suite. Along with the diagnostic suite, this may be incorporated into EMSL's existing SI architecture as a live monitoring tool, or users may adapt it to their own applications.

Demonstrate on an autonomous experiment run at EMSL

The diagnostics and trace standard will be exercised end-to-end on EMSL's own autonomous platform with preexisting data and on participants' systems so that every method is validated on real experimental loops rather than in simulation alone.

Solve a big challenge

This campaign helps address a challenge the field has barely begun to confront: Autonomous laboratories now act on model predictions and their stated uncertainties, yet the community has no shared practice for checking whether that uncertainty can be trusted or what a "trustworthy" value looks like. The problem is compounded by every group using a different experimental-design algorithm, so even when uncertainty is examined, the results are not comparable across labs, and the familiar checks (e.g., a p-value threshold or a nominal coverage level) silently assume a random sample, which is not the case for a self-selecting experimental loop.

EMSL and the campaign community will overcome this by getting diverse groups to measure the same trust properties on their own experimental loops using methods appropriate for adaptive data and then making comparisons across loops to determine which failures are universal and which are idiosyncratic, establishing the shared vocabulary and first evidence base for interpreting prediction uncertainty in autonomous science. These findings will inform a best-practices manuscript to be coauthored with participants and tools for the community to use in their own autonomous laboratories.

Expected campaign outcomes

AN OPENLY AVAILABLE TRUST-DIAGNOSTIC SUITE AND REPORTING STANDARD

EMSL will deliver a suite of trust diagnostics that run directly on an autonomous experiment's records, together with a trace/reporting standard defining the predictive statistics and outcomes an experiment must log to be auditable. Assessed with the campaign community and demonstrated on EMSL's AMP2 platform and participants' systems, it will be built so a group can run it on its own experimental loop.

PUBLICATIONS AND INSIGHTS FOR THE RESEARCH COMMUNITY

By applying the diagnostics across participants' data, the campaign will build a shared, cross-lab reference that makes any single result interpretable and reveals which trust failures recur across species, platforms, and experimental designs. This will be captured in a coauthored "common protocol across real experimental loops" publication and best-practice guidance, identifying what is easy to assess, where systems surprisingly fail, and what good practice looks like.

LLM WORKFLOW RELIABILITY METHODS AND REUSABLE EVALUATION MODULES

EMSL will deliver a lightweight harness that measures the run-to-run consistency of the LLM orchestrators that read the literature and propose experiments, quantifying how reproducibly they translate scientific goals, protocols, and code. This will include the scientific assessment methods and reusable code to implement them, designed to work with EMSL's shared evaluation, tracing, and reporting abilities; the modules could also be used by external participants.

Advance your research

  • Accelerate your science
  • Access advanced technology
  • Gain experience and key knowledge
  • Coauthor scientific publications

Why "community" science campaigns?

Each community science campaign is intended to bring together researchers with a wide variety of expertise to collaboratively tackle strategically identified challenges that are bigger than what an individual principal investigator or small team research effort can address alone.

Together with the scientific community, EMSL can build a shared reference spanning diverse organisms, platforms, and experimental designs, making each participant's trust diagnostics interpretable and distinguishing the failures that recur across autonomous science from those unique to any single lab.

Contacts

Campaign leaders (science domain experts)

EMSL User Program Services (logistics)

 

View proposal calls overview.