show: all  february  march  april  may
week 02feb 05, 2025In order to build a prototype, I first needed to find a machine learning model that would be able to separate a person’s body from the background of the video stream. For one of my projects from last semester, I used a MediaPipe model to get the pose (i.e. body position) of a subject from a video stream, so I looked there first for any solutions. I found that they had a prebuilt SelfieSegmentation model, and a code example to go along with it. The model is built specifically for separating a person from their background in a webcam feed, which felt promising considering that most models were built for general image segmentation. 

I created my own version of the code, and began testing how accurate and responsive the model was for different forms of movement. I found that it had trouble handling video feeds with cluttered backgrounds, and furthermore, the segmentation just wasn’t precise enough. 

For my arms and hands, it would segment the general region, instead of a tight boundary. Even though Blobby will eventually just render a ‘blob’ version of the body contour, I want the inner model to be accurate so that the blobs adjust as expected according to the user’s movements. 

As such, I took away that this model was not what I was looking for, but I was happy that I was able to experiment with body segmentation on a live video.


©Aditi Gupta
New York University
Integrated Design & Media (IDM) Graduate Thesis