Abstract
We present a Mixed Reality system called Editing Reality, which integrates multiple Generative A.I. (GenAI)models. We have identified three functions: Addition, Erasing, and Modification. These functions are employed to add generated objects to the real environment, hide or subtract actual objects, and alter the real environment. Through the combination of GenAI and Editing Reality, users gain the capability to virtually manipulate reality in real-time with their speech.The potential usage of Editing Reality are interior design, prototyping for products, accessibility use, and also the exploration of security issues in MR.
Publication
S. C. -C. Weng, Y. -M. Chiou and E. Y. -L. Do, "Dream Mesh: A Speech-to-3D Model Generative Pipeline in Mixed Reality," 2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Los Angeles, CA, USA, 2024, pp. 345-349, doi: 10.1109/AIxVR59861.2024.00059.