This week, I focused on improving the Python webcam demo to bring it to the same level as the JavaScript version I had created. After looking at the documentation more closely, I realized I was using an outdated version of the MediaPipe segmentation model. Updating the model to the Python equivalent of BlazePose immediately fixed the tracking capabilities.
In the JavaScript version, the library automatically outlines the outputted segmentation mask, but I couldn’t find a similar parameter in the Python version. To address this, I used OpenCV’s
findContours method to extract the outline of the segmentation mask and drawContours to display it on the screen. Importing OpenCV in Python was straightforward, unlike in my previous JavaScript project, and it significantly simplified my code by eliminating the need to write custom image processing functions.- Use OpenCV’s
findContours()to extract the outline of the body. - Use OpenCV’s
approxPolyDP(e)to approximate the outline as a polygon, where e controls how closely the approximation follows the original contour. - Use SciPy’s
CubicSplineclass to transform each line segment of the polygon into a smooth curve. - Use OpenCV’s
polylines()to draw the curves onto the image.
Throughout this process, I used NumPy to ensure data compatibility with these functions. I tested the function with the 'lite' version of the BlazePose model, and my computer handled the webcam feed so smoothly! It responds suprisingly well to my movements.
It still retains some of the jitter present in the polygon prototype from last week (which I assume is happening because it draws the shape before fully determining if there is more of the body to trace). However, this is something I can refine later. Additionally, this is just an initial blobbification prototype, so I plan to adjust the actual shape of the blobs over the coming weeks with further experimentation.
For my next steps, I plan to create a link between the Python script and the JavaScript webpage to enable the webpage to display the generated blobs. This webpage will be where the tile algorithm will be displayed in its entirety.
Now that I have a somewhat ‘blobbified’ version of the webcam feed ready for demo day, I want to take advantage of having an audience by gathering feedback on the shape of the blob—specifically, whether it aligns with what they envisioned when the project was first introduced. I have some flexibility in adjusting the blob’s shape, but my goal is to preserve subtle movements without making the silhouette too human-like. At the same time, I want to avoid creating blobs that look too similar to each other due to limited responsiveness to subtle movement. Striking this balance is something I must refine in the coming weeks, but only after I make significant progress on the networking side. I don’t want to get too caught up in this one aspect before I have the basics in place.