Recent advancements in machine learning and AI hardware acceleration have enabled the use of complex models for solving computer vision problems in real-time applications. Pose estimation is one such problem, involving the detection of keypoints of the human body within an image.

In this talk, we show how PosePIE uses pose estimation to control any game or program using hand or body movements. By using state-of-the-art models, PosePIE does not require expensive specialized sensors but works entirely on the monocular image from an off-the-shelf webcam. By leveraging readily available Graphics Processing Unit (GPU) hardware, it is able to do all processing at a high frame rate to support interactive applications.

As PosePIE is fully configurable by the user through Python scripts, it can be easily adapted to new applications. This lowers the barrier to use pose estimation and gesture recognition in creative ways and for novel applications.

The source code of PosePIE is available on GitHub under the GNU GPLv3+ license.