Representation Engineering: A Top-Down Approach to AI Transparency

A system leveraging Vision-Language Models (VLMs) to assist users in performing exercises correctly by comparing their execution against reference videos of expert demonstrations. The system uses frame-level visual and motion comparison, integrated with language-based feedback, to generate natural language guidance that helps users improve their form and reduce the risk of injury.

Contributors

Abrar Ahmed¹, Shahriar Kabir²

¹Trainee-AI/ML, Celloscope ²Senior AI Research Engineer, Celloscope

What is Celloscope Exercise Monitoring System?

Celloscope Exercise Monitoring System is an intelligent system that helps users improve their exercise form by analyzing video input and providing personalized feedback on workout mistakes.

How It Works?

The system first extracts 2D keypoints from both the user and reference workout videos. These pose representations are then compared to detect alignment errors, independent of body type or appearance.

AI-Powered Feedback

We feed the pose comparison data into a vision-language model (VLM), which generates concise, human-readable feedback on form issues and how to fix them — like a smart personal trainer.

Video Analysis & Comparison

Reference Video - Text Removed

User Video - Text Removed

Reference Keypoints

User Keypoints

AI Feedback Summary

Mistakes Feedback

The user is performing a series of crunches, a core-strengthening exercise. Comparing the user's performance to the reference frames, several key observations can be made: Posture and Alignment: The user's overall alignment is somewhat off. The spine appears to be slightly curved, and the lower back is not fully engaged, which can reduce the effectiveness of the exercise. In the reference frames, the spine is more neutral, and the lower back is flat against the ground. Control and Timing: The user exhibits a lack of control, particularly in the descent phase. The movement appears rushed, and there is a noticeable lack of tension in the core muscles. The reference frames demonstrate a controlled, deliberate movement, emphasizing the engagement of the abdominal muscles throughout the range of motion. Common Mistakes: The user is pulling the neck forward, a common mistake that can lead to strain. Additionally, the user's hands are not positioned correctly, which can affect the engagement of the core muscles. The reference frames show the hands placed behind the head, with the elbows wide, which helps maintain proper neck alignment and core engagement.

Improvement Feedback

To improve the user's performance, the following instructions are recommended: Proper Setup: Lie on your back with your knees bent and feet flat on the floor. Place your hands behind your head, with your elbows wide, and keep your neck relaxed. Engage the Core: As you lift your upper body, focus on contracting your abdominal muscles. Imagine pulling your belly button toward your spine to engage the core fully. Controlled Movement: Perform the crunches slowly and deliberately. Avoid using momentum; instead, focus on a controlled ascent and descent to maximize core activation. Maintain Alignment: Keep your lower back flat against the ground throughout the exercise. This ensures that the core muscles are engaged properly, and the movement is effective. Avoid Neck Strain: Do not pull your head forward or use your neck muscles to lift. Keep your neck relaxed and focus on the core muscles to perform the movement. By following these instructions, the user can improve their form and maximize the benefits of the crunch exercise.