De Imagem a Artefato: Geração de Objetos 3D Guiada por Texto para Experiências Interativas em Museus

Abstract

Currently, many museums are integrating virtual reality into their tours, offering both virtual and mixed reality experiences. This process is still in its early stages and often requires skilled professionals and high-cost computational techniques such as photogrammetry to generate good results. In this context, this work presents a streamlined approach that starts from a single 360° panorama and a natural language prompt, producing textured 3D models suitable for VR applications. The proposed pipeline combines prompt-guided segmentation with recent single-image 3D reconstruction models, eliminating photogrammetry and manual mesh cleanup. The reconstructed artifacts were integrated into an interactive virtual museum developed in Unity, enriched by a Retrieval-Augmented Generation (RAG) assistant capable of answering visitor questions in English based on verifiable curatorial sources. A preliminary user evaluation demonstrated that participants were able to interact with the artifacts and perceived the information provided as sufficient for understanding the objects. By reducing human effort and computational overhead, the proposed pipeline enables more interactive, immersive, and easily updatable museum experiences for institutions of any size.