• Engineering

Detecting Collisions with RealityKit in visionOS

Our how-to guide

August 10, 2023 by Mikaela Caron and Brian Capps

We’ve been experimenting with visionOS lately at Lickability, and it’s been a lot of fun to learn and share our progress online as we go. Detecting whether two entities have collided with each other was one of the first things we wanted to do when developing for Vision Pro. We can accomplish this with RealityKit, but does it look any different on visionOS? Let’s break it down:

First, let’s imagine we have two entities — one cube and one sphere — created with a visionOS project, using volume initial scene, without an immersive space. (I’m using Xcode 15.0 Beta 6 for this demo, all code is subject to change.)

We want to know if our objects collide, and a straightforward way to do that would be to move one object into another — so let’s drag the sphere into the cube.

Adding collision components

To move the sphere, we need to set up the proper components and add a Drag​Gesture. It already has an Input Target component and Collision component set up in the sample, as noted in the session Build spatial experiences with RealityKit, and both of these components are necessary to drag an entity. The cube also needs a Collision component so we can detect when the sphere has collided with it.

If you’re not sure which collision shapes to use, you can view them in Reality Composer Pro by going to Viewport → Check Collision Shapes.

This isn’t super useful when the collision shapes match the entities, but if we change the collision shape of the sphere to be a box, it’s more defined:

Moving the sphere

Next, we want to create a Drag​Gesture in our Content​View with the following steps:

1. Reference the sphere entity outside of the make closure in the Reality​View
2. Find the sphere entity in the scene
3. Target the entity with the Drag​Gesture
4. Change the position of the sphere when the drag changes

struct ContentView: View {
    
    @State private var sphere: Entity? // 1

    var body: some View {
        VStack {
            RealityView { content in
                if let scene = try? await Entity(named: "Scene", in: realityKitContentBundle) {
                    content.add(scene)
                    sphere = content.entities.first?.findEntity(named: "Sphere") // 2
                }
            }
            .gesture(
                DragGesture()
                    .targetedToEntity(sphere ?? Entity()) // 3
                    .onChanged { value in
                        guard let sphere, let parent = sphere.parent else {
                            return
                        }
                        
                        sphere.position = value.convert(value.location3D, from: .local, to: parent) // 4
                    }
            )
        }
    }
}

Now our sphere can move! 🥳

Detecting collision

Once our sphere can be moved around, all we have to do to detect collision is subscribe to a Collision​Event on the sphere:

1. Create the subscription to Collision​Events.Began for the sphere.
2. Store the subscription as a @State property so that it sticks around

struct ContentView: View {
    
    @State private var sphere: Entity?
    @State private var subscription: EventSubscription? // 1

    var body: some View {
        VStack {
            RealityView { content in
                // Add the initial RealityKit content
                if let scene = try? await Entity(named: "Scene", in: realityKitContentBundle) {
                    content.add(scene)
                    sphere = content.entities.first?.findEntity(named: "Sphere")

                    // 2
                    subscription = content.subscribe(to: CollisionEvents.Began.self, on: sphere) { collisionEvent in
                        print("💥 Collision between \(collisionEvent.entityA.name) and \(collisionEvent.entityB.name)")
                    }
                }
            }
            .gesture(
                DragGesture()
                    .targetedToEntity(sphere ?? Entity())
                    .onChanged { value in
                        guard let sphere, let parent = sphere.parent else {
                            return
                        }
                        
                        sphere.position = value.convert(value.location3D, from: .local, to: parent)
                    }
            )
        }
    }
}

That’s it — now, we’re tracking when the sphere collides with the cube! This event tells us when the two begin colliding, but you can also track when collision ends by subscribing to Collision​Events.Ended.

If you want to see where we went from here, we documented our visionOS progress in a thread here and on Mastodon. You can also check out our sample project on GitHub.