Will this smartphone app become your exercise coach?

0 190

It analyzes silhouettes to tell users if they are performing their moves correctly

When the COVID-19 pandemic closed gyms and put school sports on hold, many teens looked for other ways to stay active. Some took up at-home yoga or running. For high-school sophomore Michelle Hua, that wasn’t enough. This 16-year-old student at Cranbrook Kingswood School in Bloomfield Hills, Mich., invented an app to track her movements. It identifies her exercises and even gives her coaching advice.

It’s helped her stay active during the pandemic. Even more rewarding, that app helped her win the $75,000 George D. Yancopoulos Innovator Award, this week. It’s the top prize at this year’s Regeneron International Science and Engineering Fair. (For more award winners, see box at bottom.) Created by Society for Science (which publishes this magazine), ISEF brought together almost 2,000 high school finalists. This year, the annual science competition was held virtually.

Michelle is a rhythmic gymnast — someone who does floor exercises with props such as a hoop, ribbon or ball. The sport blends gymnastics and dance. But during COVID-19, her gym shut down. She continued to practice online at home, but Michelle wasn’t satisfied. She wanted to up her training.

So the teen developed an app that tracks her movements. It even tells her whether she is performing them correctly.

Some movement-identifying apps use models of skeletons to determine movement. They analyze a video to identify body parts and identify movement based on that. But that approach is not very accurate, Michelle says. “It has to know where the head, shoulders, arms, legs, feet, etc. are in each frame of the video,” she notes.

Michelle decided instead to use silhouettes, outlines of whole people. With silhouettes, the program wouldn’t “need information about the location of body parts,” she explains. “It only needs to separate the shape of the human — regardless of where the head, arms, [and] legs are — from the background it is in.”

Machine learning includes deep learning and neural nets

The gymnast designed her program using a neural-net system. This is a type of artificial intelligence program that can learn from the data on which it trains. Michelle’s trained hers using data from different sets of movement files. Those thousands of videos show people in all sorts of motions, from sitting to jumping and running. Her program analyzed each video, drew a silhouette and then learned what that silhouette was doing.

The program now can recognize everything from brushing your hair to chewing gum. It also can recognize exercises such as jumping jacks. But rhythmic gymnastics wasn’t in any of those movement data sets on which her program trained. So the teen took videos of herself performing. “I labeled my own data and trained my model with it,” she says.

Her new app knows what each exercising silhouette should be doing. When someone performs a jumping jack, for example, the app takes a silhouette — and then might tell the user to lift her arms higher. “Feedback from the app helps users correct their position to prevent any exercise-related injury,” Michelle says. She hopes people will use her app to exercise more effectively. People also could use it to analyze how well they perform the physical therapy prescribed to people recovering from injuries.

In developing her project, the teen worked with Zichun Zhong. He’s a computer scientist at Wayne State University in Detroit, Mich. Together, they published results of Michelle’s research in the journal Computer Aided Geometric Design.  

The next step, Michelle says, is to put her app on the Apple app store. In the meantime, she notes, “my younger brother and I have been using my app. It has helped us by keeping us active and exercising throughout the year.”

Teens take home millions for apps, medical diagnostics and more

For her development of an exercise app, Michelle Hua today takes home the $75,000  top prize at this year’s Regeneron International Science and Engineering Fair. Created by Society for Science, the annual event brought together almost 2,000 high school students. They came from 64 countries, regions and territories to share their science fair projects. Due to the COVID-19 pandemic, the 2021 fair was held entirely online. The teens faced panels of scientific experts who examined their projects, and still got a chance to show their work to the public. The finalists competed over the past week for $5 million in awards.

Other major award winners, named this week, include:

Daniel Shen, 17, a junior at William G. Enloe High School in Raleigh, N.C. He won the Regeneron Young Scientist Award of $50,000. He developed an app that can respond quickly to someone’s facial cues.Catherine Kim, 18, a senior at Jericho High School in Jericho, N.Y. She took home the $50,000 Regeneron Young Scientist Award for developing a computer program that can help predict when medicines will interact badly with each other in someone’s body.John Benedict Estrada, 16, a sophomore at Clovis North High School in Fresno, Calif. For creating a robotic arm that can detect whether a plant is getting stressed by drought , he got the $50,000 Gordon E. Moore Award for Positive Outcomes for Future Generations of $50,000.Arya Tschand, 17, a senior at High Technology High School in Lincroft, N.J.  He also tackled drought, using a drone to detect dry plants and sending a signal to adjust how much water the plants received. For that, he was took the $10,000 Craig R. Barrett Award for Innovation.Neha Mani, 17, a senior at Hunter College High School in New York City. Her H. Robert Horvitz Prize for Fundamental Research, worth $10,000, rewards her development of a computer program that can distinguish between swimming and swarming bacteria to help doctors diagnose gut diseases.Franklin Wang, 17, a junior at Palo Alto Senior High School in Palo Alto, Calif. For creating a computer program that can detect speedy near-Earth asteroids, he won the $10,000 Peggy Scripps Award for Science Communication. His program has already found six new asteroids.

Along with 23 other projects from individual and teams of students, the top seven students also won first place awards of $5,000 for their research categories. Another 78 projects got second place awards of $2,000, and another 121 got third place awards of $1,000. A final 157 took home fourth place awards of $500. Even more received sponsored special awards.


Leave A Reply

Your email address will not be published.