What’s New for Developers in Vision OS 2.0

Apple’s visionOS 2.0 introduces a suite of new developer APIs and frameworks across UI, AR/VR, graphics, and collaboration domains. These enhancements expand what’s possible on Apple Vision Pro, enabling richer spatial experiences that are more immersive, interactive, and multi-user. Below is a breakdown of the key new APIs by category—UI improvements, ARKit and RealityKit enhancements, Metal optimizations, and collaborative features—detailing their purpose, capabilities, and benefits, with examples of how developers can use them.

VisionOS 2.0 extends SwiftUI and UI frameworks to better support 3D interfaces and user interaction in space.

• Volumetric UI Resizing & Scaling: Apps can present 3D content in volumes (SwiftUI scenes with depth) that are resizable and can scale appropriately with distance. A new SwiftUI scene modifier windowResizability lets developers control whether a volumetric window can be resized by the user. Developers can also specify if a volume’s content maintains a constant apparent size or diminishes with distance.

• Volume Ornaments (Attached UI Elements): Developers can now affix ornaments to volumes – small UI components or indicators that remain attached to a volumetric window. Ornaments could be labels, buttons, or decorative elements that float with the window, improving usability in spatial UIs.

• Hand Depth Interaction Control: visionOS 2.0 gives apps control over how users’ hands render relative to virtual content. A new input option lets developers decide if the user’s hands should appear in front of or behind digital elements. This enhances immersion by properly blending real hand presence with 3D content.

• Custom Hover Effects in SwiftUI: SwiftUI in visionOS 2 adds new hover effect APIs. Developers can define custom visual responses when the user looks at or hovers over a UI element. This provides clearer feedback in the 3D interface, improving usability.

VisionOS 2.0 builds on ARKit’s world understanding capabilities, offering more powerful environmental sensing and tracking APIs tailored for spatial computing.

• Plane Detection in All Orientations: ARKit in visionOS can now detect flat surfaces of any orientation, not just horizontal planes. This greatly expands where virtual objects can be anchored, making apps more versatile.

• Room Anchors: A new room anchor API allows ARKit to understand and label distinct rooms in the user’s environment. Apps can now “remember” spatial content per room, ensuring virtual elements stay in place when the user returns.

• Object Tracking API (Known Object Detection): visionOS 2 introduces the ability to track specific real-world objects using pre-scanned 3D reference models. Developers can include reference object files in their app, and ARKit will detect and continuously track those objects in the user’s surroundings.

• Improved Scene Understanding Fidelity: visionOS 2 significantly extends the fidelity of scene understanding, resulting in more stable and precise anchors. This helps with better placement of virtual content and improves AR interactions.

More details on ARKit updates can be found in Apple’s developer documentation.

RealityKit receives upgrades that enhance rendering realism, physical simulation, and cross-device compatibility.

• 3D Hover Effects & Hand Interactions: RealityKit now supports hover effects on 3D entities. This means 3D models can visually respond when the user focuses on them or hovers a hand over them.

• Physics: Force Effects and Joints: visionOS 2 adds new RealityKit APIs for realistic physics simulation, including force effects and joints. Developers can now apply physical forces (like impulses or gravity fields) and connect entities with joints (hinges, springs) for constrained motion.

• Dynamic Lights and Shadows: Apps can now use real-time lights and shadows in RealityKit, making virtual content blend more naturally into its environment.

• Portal Crossing Enhancements: The portal API now supports partial portal crossing, allowing objects to smoothly transition through a portal instead of popping from one side to the other.

• Cross-Platform Support (iPadOS/macOS): Many new RealityKit APIs are cross-platform, meaning the same code works on iOS, iPadOS, and macOS as well as visionOS.

More details are available in the RealityKit API reference.

VisionOS 2.0 introduces optimizations and new APIs for custom 3D rendering.

• Scene-Aware Projection Matrix: A new API provides a projection matrix that factors in the headset’s camera intrinsics and real-time scene understanding data. This improves how virtual objects align with real-world surfaces.

• Trackable Anchor Prediction (Latency Reduction): visionOS 2.0 provides trackable anchor prediction, reducing rendering latency for AR scenarios by predicting the future pose of moving AR anchors.

• Optimized Compositor and Foveated Rendering: Developers now have better tools for working with foveated rendering, which enhances performance by focusing detail where the user is looking.

• Enterprise-Grade Performance Controls: Apps can request more CPU/GPU compute power for intensive workloads, such as professional CAD or data visualization apps.

More details on Metal updates can be found in Apple’s Metal documentation.

VisionOS 2.0 places a big emphasis on shared experiences, introducing new APIs that let multiple people connect, play, or work together in the same virtual space.

• TabletopKit Framework: A new framework designed for multiplayer experiences around a virtual table. It provides high-level constructs for board games, turn management, and object placement.

• SharePlay and Spatial Persona Integration: visionOS 2 deepens integration with Apple’s SharePlay, allowing apps to invite others into a shared Full Space session. FaceTime spatial Personas can now be used directly in apps for a more immersive social experience.

More details on shared experiences can be found in Apple’s SharePlay documentation.

The new APIs in visionOS 2.0 empower developers to create apps that are more immersive, interactive, and connected. UI improvements like volumetric resizing and hover effects enable intuitive 3D interfaces. Enhanced ARKit scene understanding and object tracking let apps better integrate with the user’s environment. Upgraded RealityKit rendering and physics produce more realistic visuals and interactions, while Metal-level optimizations ensure smooth performance. Finally, frameworks like TabletopKit open the door for multi-user spatial computing, turning Vision Pro into a device not just for solo experiences but shared ones.