As the Director of Applied Performance Science for the University of Tennessee football program, Jermone Learman is always trying to give the Vols an edge. Maximizing player performance and reducing injuries are at the top of the list.

Learman and his team have been conducting video analysis manually, trying to spot any potential biomechanical flaws in players during practices and games. The process can be time consuming, so Learman began exploring if any machine learning or AI models could help speed up the process.

Once he didn’t find anything that he could trust, Learman contacted the Tickle College of Engineering and was put in touch with Hector Santos-Villalobos, an assistant professor in the Min H. Kao Department of Electrical Engineering and Computer Science.

Santos-Villalobos performs cross-cutting research focused on the design, development, and explainability of multi-modal artificial intelligence systems. He has a background in computer vision, which enables computers to understand the world through images and videos.

Santos-Villalobos began working with Learman earlier this year to develop AI-driven performance analytics for the UT football team to try and reduce player injuries.

The project recently received $60,000 in funding from AI Tech X, a new initiative created by the UT to empower Tennessee communities and industry partners to adopt artificial intelligence technologies that enable high-quality job creation.

Santos-Villalobos is leading the Multi-Modal Analytics, Reasoning, and Computational Imaging (MARCI) team, comprising EECS researchers and students in partnership with UT Athletics and the Joe Gibbs Human Performance Institute, a biomechanical engineering research facility in Charlotte, North Carolina.

“It’s been amazing. He’s brought some great students on board, and they’ve made some major strides with everything,” Learman said. “Obviously, it’s very time consuming. We’re not there yet, but we’ve seen great progress.”

AI-driven Holistic View

Santos-Villalobos is passionate about the health applications of technology and was excited to collaborate with the Vols because of the football team’s importance to the university and the pride everyone takes in watching them succeed.

The research team, which includes EECS Professor Fei Liu and graduate student Mridula Venkatasamy, started by automating the tabulation and visualization of some of the football team’s drill data to create weekly summaries for the coaching staff.

“I don’t like doing science for the sake of science. I like using it to solve real problems. That’s very important to me,” Santos-Villalobos said. “When I have a new stakeholder, I want to learn what their needs are and then see if I can solve something that is important to them very quickly so that I can gain their trust. Then, after we establish the relationship, we can try to do new science.”

Santos-Villalobos and his team meet regularly with Learman and his team to find out how they are currently assessing the players. One aspect is looking at how they sprint. Is the heel of the back foot above the calf? Are the knees fully extended?

The AI-driven data analysis will provide football trainers and coaches with actionable information they can use to help improve a player’s biomechanics or flexibility.

“Our goal is to try to mimic the human expert assessment with the computer,” Santos-Villalobos said. “Right now, they watch a video of an athlete, assess over 17 different movement patterns, and give the athlete a score. They have a bunch of athletes, so it’s very time-consuming, repetitive, and sometimes subjective. This is the perfect task for the computer. It can do this 24/7 and doesn’t get tired or bored.”

The next step in the process is to eventually integrate the images with other information that is collected, such as diet, biomarkers, and vision.

“In the 30,000-foot view, if I combine every data available on an athlete, we may be able to connect daily habits, routines, and diets to better physical and mental health and superior performance,” Santos-Villalobos said. “What kind of environment or exercises or food will make them perform optimally? How can we keep them safe? We are living in a time where we can use all this data to personalize sports science and methods to the individual.”

On-the-Job Training

An impending milestone of the project is having the computer recognize players while they are in action since most are moving at a fast rate. Then, the research team wants to estimate certain key points of the body like the head, wrist, ankles, knees, and hips. The more feedback will lead to more refinement.

“We want to be able to improve that detection accuracy, so that we can detect the key points with high precision and high sensitivity,” Santos-Villalobos said. “After that, we want to move into estimating meshes from the athletes. The mesh is a little bit different than the key points, because that involves determining the texture, shape, and orientation of the body parts.”

The goal is for the project to be completed by the end of summer 2026. The results will provide direction about how much the research team can expand the results beyond the original scope and perhaps work with other sports teams on campus.

“The key is to show value to the football team, so that we can continue the work and, especially, continue developing the expertise of the engineering students working on this project,” Santos-Villalobos said. “Because this is a career. Our students can make a career working on these types of problems.”

Contact

Rhiannon Potkey (rpotkey@utk.edu)