Decart XR

DecartXR brings Quest 3 passthrough, Unity, WebRTC, and Decart's real-time video models into a single loop. The goal is simple for developers: capture the headset's camera feed, stream it to an AI model, receive transformed frames, and display them in VR with predictable latency.

• Quest passthrough access Android Camera2 discovery with Meta-specific metadata, permission handling, and stable WebCamTexture setup.
• Unity XR layer World Space canvases for display, a dedicated capture camera for WebRTC encoding, and scene objects wired for controlled timing and rendering.
• WebRTC transport WebSocket signaling, STUN-based peer setup, VP8 encoding at 30 fps, manual bitrate configuration, and automatic receiver creation for incoming textures.
• Decart AI video models Mirage for environment transformations and Lucy for character-focused transformations, selected directly through the model parameter in the WebSocket URL.
• Processed video routing Dynamic receiver discovery and unified display logic for streaming the returned AI-processed frames inside VR.
• Voice-driven prompts Wit.ai speech-to-text integrated into the same WebRTC prompt flow for hands-free model control.

Get started

Clone the project and run it on a Quest 3 to explore the full camera -> WebRTC -> Decart model -> VR loop.