Human researchers throughout the United States are increasingly involving Artificial intelligence (AI) agents in the scientific pipeline. Scientists prompt agents to read papers, propose hypotheses, conduct simulations on supercomputers, and operate laboratory machinery.
Unfortunately, that also opens the pipeline to new security risks. Attackers can hide malicious instructions in apparently legitimate databases, papers, and messages from other agents that redirect what the agent does.
Suya
“Because the manipulation lives inside the model rather than in code, standard cybersecurity tools can’t see it,” said Assistant Professor Suya, an AI cybersecurity expert in the Min H. Kao Department of Electrical Engineering and Computer Science (EECS).
Suya has been studying AI model attack and defense for nearly a decade, developing new ways to poison data and hijack models and then protecting AIs from those novel attacks.
When the United States Department of Energy announced the new Genesis Mission—a $5 billion program to create a nationwide ecosystem of AI tools and services that will help researchers address pressing energy, scientific, and engineering challenges—Suya decided to apply his expertise on a national level.
Suya and North Carolina State University Assistant Professor Xiaorui Liu co-developed a Genesis proposal to defend scientific agentic AIs from adversarial attacks. The project, which will be led by Liu, was one of just 277 Phase I projects awarded funding by the DOE this summer.
Mahshid Ahmadi, an associate professor in the Department of Materials Science and Engineering (MSE), is their co-principal investigator; her autonomous material lab will be the project’s testbed.
“The Genesis Mission’s call for ‘AI for securing AI’ matched what Dr. Liu and I both believed: the only defense that can keep pace with AI-driven attacks is one that improves itself,” Suya said. “Pairing a mathematically hardened architecture with an autonomous attack-and-repair loop, and proving it on a lab that actually makes materials, was the natural way to test that idea.”
Defending AI Agents from Attack
While compromised AI agents can cause frustration and security issues in any field, the stakes in scientific work are often much higher.
“An adversarial attack or corrupted information could quietly influence an AI agent’s decisions, potentially leading to incorrect scientific conclusions, wasted experiments, or unsafe experimental decisions,” Ahmadi explained.
Common defenses for agentic AIs fall short in multiple ways, Suya says. Protections like filters and prompt guards sit on the surface of a model, so adaptive attackers can get around them. Retraining a model to resist attacks takes resources away from its main task, making it perform poorly.
The key issue, however, is that attackers can now use AIs of their own to invent attacks faster than people can combat them.
“When Dr. Liu and I compared notes, the gap was obvious,” said Suya. “Scientific AI agents were being deployed on national lab infrastructure faster than anyone was securing them, and the existing defenses were exactly the kind I knew how to break.”
Liu’s group has demonstrated that it’s possible to rewrite the basic operations inside a neural network so that robustness is a built-in property, making the model much more secure than filters and guards patched onto it afterward. With that foundation, Liu, Suya, and Ahmadi will develop AI systems that can recognize and defend against evolving threats while maintaining the accuracy and efficiency needed for autonomous science.
Suya’s lab will design new attacks against scientific workflow agents, develop tailored defenses, and determine what types of tests will rigorously evaluate the protections.
“We deploy AI ‘red’ teams and ‘blue’ teams that continuously attack and repair the system, so the defense evolves as fast as the threats,” Suya explained.
Ahmadi will integrate the team’s full security framework into her self-driving materials laboratory, where AI agents connect hypothesis generation and computational screening with automated synthesis and real-time characterization.
“This strategy allows us to stress-test the cybersecurity approaches in an actual autonomous scientific workflow,” Ahmadi said, “and ultimately demonstrate that agentic AI can accelerate materials discovery while remaining trustworthy and resilient.”
Contact
Izzie Gall (egall4@utk.edu)