Hi XR Developers! Today, we are looking at something you all have been waiting for: Access to the Meta Quest Passthrough Cameras, which just got enabled with Horizon OS v74. This API lets us tap into the raw camera feeds on Quest headsets, opening the door for features like real-time environment sampling, AR overlays, or even object detection.
Additionally to Meta’s Passthrough Camera Samples GitHub repo I also open sourced my experiments with the Passthrough Camera Access in a project called QuestCameraKit. If you are interested in exploring more use cases such as Color Picker, Object Detection, QR Code tracking, and visual effects such as frosted glass shader, feel free to check it out and leave a star!
✨ https://github.com/xrdevrob/QuestCameraKit ✨
Resources:
🔗 Meta Passthrough Camera Documentation: https://developers.meta.com/horizon/documentation/unity/unity-pca-documentation
🔗 Meta Passthrough Camera Samples: https://github.com/oculus-samples/Unity-PassthroughCameraApiSamples
🔗 QuestCameraKit Samples: https://github.com/xrdevrob/QuestCameraKit
Other Videos:
📹 Colocation with Meta’s Shared Spatial Anchors & the new Colocation Discovery API: https://www.youtube.com/watch?v=elf6WuGzwdI&t
📹 Destructible Global Mesh, Instant Content Placement and MRUK Trackables: https://www.youtube.com/watch?v=Ne2AFjXeanY&t
📹 Create an MR Drawing app using Logitech’s MX Ink Stylus for Meta Quest: https://www.youtube.com/watch?v=KCPEpbY5XQE&t
📹 Use Meta’s Llama 3 Model in Unity with Amazon Bedrock & other partner services: https://www.youtube.com/watch?v=vpjRoGy7kCI&t
📹 Use Meta's Voice SDK for Wake Word Detection & Speech To Text (STT): https://www.youtube.com/watch?v=nKprzndVHQM&t
📹 Apple Vision Pro Input & Object Manipulation with Unity PolySpatial: https://www.youtube.com/watch?v=k9cJfn4JcTA&t
📹 Meta Quest Spatial Anchors: https://www.youtube.com/watch?v=vaAQvdGYeak
Join the Discord!
👾 https://discord.gg/v8fVjweUPB
Support the Channel:
🫂 Patreon: https://www.patreon.com/blackwhalestudio
🐦 Twitter: https://twitter.com/xrdevrob
🤝 Join this channel to get access to perks:
https://www.youtube.com/channel/UCat2tlm29PRViBe8_yz4j5Q/join
Need professional XR development for your project?
👨💻 Get in touch with us: https://bento.me/blackwhale
00:00 Intro
00:56 Limitations & Requirements
01:58 Meta’s Passthrough Camera Samples
02:36 Camera Permissions class
03:38 Camera Utils class
05:08 WebCamTexture Manager class
06:20 Let’s build a Color Picker!
10:07 Color Picker Demo
10:23 Outro