Description

The AI Yoga Learning Assistant is a web platform to learn yoga in a fun way using advanced ML pose detection technology. Made for anyone with a laptop and webcam, your yoga poses can be estimated with feedback provided in real-time. Your webcam video stream is displayed side-by-side with a lesson video. The platform analyzes your webcam frames and detects 17 body joints in 2D space and compares it to the reference pose in time lesson video. Real-time feedback enables you to improve your posture and take your yoga practice to the next level. The technologies we have chosen are: -PoseNet on TensorFlow.js. Tensorflow is a JavaScript library developed by Google that allows you to run machine learning models directly in the web browser. Posenet is a pre-trained model that detects the position of key body joints on a 2D graph. -React.js for our front-end interface -Node.js and Express.js for our backend API to process uploaded videos by pre-detect poses and store them -MongoDB for our NoSQL database to store video details At present, the features we have developed include the instructor's ability to upload new lesson videos. Our platform pre-detects poses on any custom lesson videos. While students follow the lesson video, their real-time body limb positions are scaled and compared on time with the lesson video. The lesson experience is also enhanced by face filter stickers, ability to add background music, receive performance scores, and review playback of your practice. Future directions include gesture recognition to replace the in lesson menu controls, voice feedback to cure body adjustments, and multiplayer mode to detect multiple players.

History

We found this project interesting among all projects that were listed from industry partners. During the project planning phase, our client suggested to use Kinect (a motion sensing input device to detect poses), but came up with the idea to use a Machine Learning library that runs on web browsers (Tensorflow.js). We divided tasks to each member (managed by the Project Manager) and have meetings every week to discuss all possible solutions for the project. We always try to find new ideas and discuss on what would be the best solution to meet our client's requirements and ensure low cost (our client is a non-profit organization). Every week, we have meetings with our Capstone professor to show what we have done, then get her feedback and suggestions to improve the project.

Features

  • Add new lesson videos to database (pre-detect poses from the whole video)
  • Delete or edit lesson videos (description, title, level)
  • Search lesson videos by name
  • Detect poses in real time
  • Provide pose feedback in real time
  • Review performance after finish each attempt
  • Display stickers for engagement
  • Configure game play by modifying game settings

Videos

AI Yoga Learning Assistant Pitch

Images

There are currently no logos or icons available for AI Yoga Learning Assistant. Check back later for more or contact us for specific requests!

About Tech Cat Dream

Our team of programmers love to explore computer vision technology. With new javascript libraries that are making ML more approachable, our team has taken on a unique project. We have collaborated together to help people learn yoga in an interactive way. Our project manager has a background in research for a motion capture studio. We also have team members with co-op experience in full-stack and front-end development in banking and in government. We aim to create solutions that are maintainable and have a user-centered focus.

AI Yoga Learning Assistant Credits

Steven Carino
Project Manager
Ngoc Phuong Thao Doan
Full-Stack Developer
Shannon Lim
Database Admin
Seunghyun Ban
Front-end developer