Most AI biorisk studies look at how AI models might help novices complete biological tasks. But experts with AI assistance could be far more dangerous.
On "... AI models cannot adequately guide novices in experimental work..." I think this *might* have changed. Hong et al. tested Opus 4.1 and GPT-5 (which have 121 and 134 BCI, respectively). Opus 5 is now at 178. Do you think if they ra-ran the trial with late-2026 models, would uplift be more than just modest?
But yeah we need a well-designed RCT trial like Hong et al. but for experts! Great post
Your western blot story makes the point better than any benchmark could. Protocols assume reagents work, yet the fix came from years of hands-on judgment that no paper records. That is why testing novices tells us so little about the risk. If experts are 20 to 200 times more likely to succeed, should safety frameworks not require expert wet-lab trials before a model ships?
Good call making the expertise question explicit. If the concern is practical lab work, measuring novices on written tasks doesn’t settle it. Benign trials with experienced researchers sound like the right next test.
On "... AI models cannot adequately guide novices in experimental work..." I think this *might* have changed. Hong et al. tested Opus 4.1 and GPT-5 (which have 121 and 134 BCI, respectively). Opus 5 is now at 178. Do you think if they ra-ran the trial with late-2026 models, would uplift be more than just modest?
But yeah we need a well-designed RCT trial like Hong et al. but for experts! Great post
Your western blot story makes the point better than any benchmark could. Protocols assume reagents work, yet the fix came from years of hands-on judgment that no paper records. That is why testing novices tells us so little about the risk. If experts are 20 to 200 times more likely to succeed, should safety frameworks not require expert wet-lab trials before a model ships?
Very much agree with this post! I wrote similarly re: the need for expert uplift studies back in March: https://counterfactual.blog/p/llms-didnt-help-novices-build-viruses
Disappointing to see that there's still no published expert wet-lab trial! What would you say is the main bottleneck here?
Read up on mirror proteins if you really want to be terrified. Work has been halted by the responsible science community.
Note the word responsible.
Good call making the expertise question explicit. If the concern is practical lab work, measuring novices on written tasks doesn’t settle it. Benign trials with experienced researchers sound like the right next test.