Adaptive AI & Lab Benchmarks
Google Debuts Gemini Decisions API as OpenAI and METR Test Frontier AI
Google AI introduces an experimentation and contextual bandit service for real-time optimization while OpenAI open-sources a 242-question wet-lab biology benchmark and METR evaluates AI R&D speed at Anthropic.

Your listening Queue & catch up
Add a story or build a catch-up queue to start listening.
Frontier artificial intelligence laboratories are pairing adaptive production routing with empirical benchmarks designed to measure both physical wet-lab fluency and internal research acceleration. Across announcements published on November 11, 2026, Google AI introduced the Gemini Decisions API for real-time experimentation and contextual personalization, OpenAI open-sourced its 242-question BioBasicsBench evaluation alongside RedQueenBio, and METR outlined 4 empirical approaches for assessing how AI tools affect research and development velocity at Anthropic.
Authored by Google Research Director Tulsee Doshi and Principal Scientist Minmin Chen, the Gemini Decisions API announcement details a developer service built on reinforcement learning and contextual bandits that turns static software rules into adaptive workflows. Rather than relying solely on traditional A/B tests that assign fixed traffic splits over weeks before declaring a single average winner, the API evaluates user context and historical outcomes in real time to select from an "arm space" of candidate configurations.
Developers define candidate options—ranging from onboarding layouts and notification schedules to distinct Gemini system instructions or model tiers—pass contextual signals to the endpoint, and log delayed reward metrics when users complete target actions. Behind the scenes, the service balances exploration of newly added options against exploitation of proven variants, shifting traffic dynamically as seasonal demand or user behavior evolves. According to Google AI, the underlying decision engine has been tested across 12 internal and external deployments over the past year. In a cited user-onboarding case study, replacing a fixed onboarding sequence with contextual decision routing produced a 56% increase in user engagement and a 32% reduction in onboarding abandonment. Developers can apply to join the early access program through Google's developer portal to test adaptive UI flows, model routing, and prompt optimization.
While software teams automate interface and prompt selection, OpenAI's wet-lab biology report examines whether frontier language models can reliably assist bench scientists with everyday experimental nuances. OpenAI pointed to 3 earlier dual-use and laboratory studies: a 2023 biosecurity evaluation with biological safety experts, a summer 2024 dry-lab and wet-lab troubleshooting benchmark created with Los Alamos National Laboratory, and a spring 2026 molecular cloning study conducted with RedQueenBio. In that spring 2026 study, an iterative GPT-5 setup improved transformation efficiency more than 79-fold while deploying novel enzyme combinations.
To capture the tacit procedural knowledge that rarely appears in published journal methods, OpenAI partnered again with RedQueenBio to build and open-source BioBasicsBench. Bench scientists recorded practical failure modes and day-to-day techniques as they worked at the lab bench, yielding 242 multiple-choice questions across 5 core workflows: molecular cloning, cell culture, protein purification, CRISPR editing, and RNA extraction. Sample items test whether a model recognizes when a water filter is incompatible with DMSO, how Trypan Blue dye behaves when allowed to sit too long during viability counts, and how to handle fragile large plasmid DNA during pipetting. Every question in BioBasicsBench was reviewed by at least 2 domain scientists for factual clarity and calibrated against 5 human PhD-level biologists. Across all 242 questions, GPT-5.4 achieved an accuracy of 83.2%, topping the evaluated frontier models and outperforming both the average human PhD score of 75.5% and GPT-5 at 74.5%. OpenAI added that its top-performing biological reasoning configuration, GPT-5.4 Thinking, remains protected under high-capability biosecurity safeguards including real-time output monitoring and biodefense researcher access controls.
Complementing model-level benchmarks, METR's organizational assessment investigates how frontier AI deployment alters the pace of AI research and development inside Anthropic. Seeking to avoid reliance on any single measurement, METR combined 4 distinct lines of evidence: a developer survey of Anthropic technical staff, longitudinal pull-request telemetry, mathematical modeling of time-allocation bottlenecks across the software lifecycle, and a retrospective productivity analysis led by Ryan Greenblatt. Together, the reports provide engineering and research teams with quantitative methods for tracking both applied production gains and scientific research throughput.
Sources
Comments (0)
No comments yet. Be the first to share a thought on this story.
Nominate a neighbor or local businessRequest a correctionContact the newsroom