Interactive climbing wall using computer vision and real-time projection mapping

Treball de fi de grau en Enyingeria de Sistemes Audiovisuals ; Tutora: Gloria Haro Ortega ; Interactive climbing walls are a technology that has gained popularity in recent years along with the rise of indoor climbing. The current limitations in the existing systems have motivated this project to investigate if a general, simple, and robust solution can be achieved using widely available tools. The developed system uses a regular projector, an uncalibrated camera (such as the one in a mobile phone), and a custom-built application to create an interactive climbing wall experience with real-time projection mapping. The implemented solution incorporates computer vision techniques such as image pre-processing, object detection and segmentation, feature matching, au- tomatic homography computation and ranking, as well as deep learning models for human pose estimation. For the application, a custom GUI has been created to facilitate easy operation by the user. The result is an interactive climbing wall that can be setup with minimal equipment, configured in less than 10 minutes, and compatible with existing walls of any shape. Finally, a simple demo of an interactive experience has been created to illustrate what can be achieved with the system and the available data structures. ; Les parets interactives d’escalada son una tecnologia que ha guanyat popularitat els ultims anys juntament amb l’auge de l’escalada indoor. Les limitacions dels sistemes actuals han motivat aquest projecte per investigar sí es possible aconseguir una solucio general, simple i robusta utilitzant eines ́ ampliament disponibles. El sistema desenvolupat utilitza un projector estandard, una càmera no calibrada ` (com la d’un telefon mòbil), i una aplicaciò o de disseny propi per crear l’expèriencia d’una paret d’escalada interactiva amb ”projection mapping”a temps real. La solucio implementada incorpora técniques de visiò per computador com ara preprocessament d’imatges, deteccio i segmentació d’objectes, ”feature matching”, calcul i rànquing ...