Crayon

Spatial Drawing on VisionPro

Unity for Vision Pro

MVC Architecture

Playful AR

User Experience

User Interface

Transform your world into an infinite canvas.

The Vision Pro's release in February 2024 revealed truly immersive potential beyond mobile AR's limitations. This breakthrough gave me the opportunity to rebuild Crayon into the magical experience I had always envisioned. Crayon transforms your environment into an infinite 3D canvas and your hands into the tools for creation.

Hand Palette

Touch your finger tips for color selection— simple, tactile, and only visible when you need it.

Pinch to Draw

It's that simple, with haptic feedback for a natural feel.

3D Object Menu

Pinch both hands to access the menu—smooth and seamless.

Grab Mode

Visual cues highlight tools for grabbing and transforming objects.

Eraser Tool

Delete lines with precision to refine your art.

Process

STEP 1: Prototype simple drawing.

Before jumping into the exciting possibilities of what immersive Crayon could become, it was essential to start by building the foundational experience without the bells and whistles of custom brushes. One color, one size, one user. Expand from there.

STEP 2: Re-imagine experience and interface design for an immersive environment.

When crafting immersive spatial experiences, it’s essential to avoid directly porting mobile design standards into a spatial environment. They are not the same. The affordances of hand tracking opens up an array of possibilities for richer interactions beyond 2D UI design conventions.

Intuitive design is relatively straightforward in traditional 2D applications—where years of web and mobile standards guide user interactions—the world of spatial product design, particularly for controller-free devices like the Apple Vision Pro, remains largely uncharted.

Many existing spatial interfaces cling to outdated 2D design conventions, relying heavily on flat panels within 3D environments. While this design approach makes sense in certain contexts, defaulting to it restricts the full potential of spatial interfaces. Crayon Vision explores a more immersive and playful interaction model, unveiling the unique possibilities of spatial design.

Below are a few examples of how I approached this exploration.

What if a hand could be the paint palette and the toolbox…

... or if each finger pinch could activate a tool menu…

... or if a user could blend colors by touching their palette visualizers?

STEP 3: Publish and Iterate.

Ultimately, I converged on a much more straightforward preliminary experience.

Ultimately, I converged on a much more straightforward preliminary experience. I decided it was better to launch a paired-down version of what is possible and use it to collect user feedback to inform iterations. There are many possible features, but ultimately it’s the feedback from active users that should guide product expansion.

Here is a high-level overview of the unique gesture UX from v1 of Crayon for Vision Pro.