Content Design for Augmented and Virtual Reality
8 principles I learned on Meta's AR Content Design team.
Augmented reality on Snapchat
Augmented reality (AR) and virtual reality (VR) are here to stay. In fact nearly a billion people currently use augmented reality, and this is projected to reach two billion in the next five years. And it’s not just about supercharging gaming—it’s about education, virtual medical training and, ultimately, improving lives— err, well, at least making them more joyful.
Content design is changing too. New devices (smartphones, smartglasses, smartwatches, etc.) with new interfaces require new forms of content and communication. Add AR and VR experiences to these devices and the demand for specialized content for these interactions skyrockets.
A new frontier is opening for content design. That's because AR and VR experiences share in common a need for specialized content design because of the inherent challenges found in both, including:
Our work as content designers in AR and VR is about adapting content and accentuating what works for these environments. Ultimately, we’re still trying to create a rich enough design that enhances the user experience without detracting from it.
To ensure we're on the same page, let's hit a few definitions:
Content design is changing too. New devices (smartphones, smartglasses, smartwatches, etc.) with new interfaces require new forms of content and communication. Add AR and VR experiences to these devices and the demand for specialized content for these interactions skyrockets.
A new frontier is opening for content design. That's because AR and VR experiences share in common a need for specialized content design because of the inherent challenges found in both, including:
- Limited in or no user interface (UI)
- Surface-less interactions
- Constrained content space
- Need for gestures and interactions
Our work as content designers in AR and VR is about adapting content and accentuating what works for these environments. Ultimately, we’re still trying to create a rich enough design that enhances the user experience without detracting from it.
To ensure we're on the same page, let's hit a few definitions:
Augmented reality
Augmented Reality is about enhancing reality.
In AR, the user sees the current reality overlaid with
additions such as user interfaces, effects, information or
other. AR lets you deliver immersive experiences that
seamlessly add virtual objects to the real world.
In AR, the user sees the current reality overlaid with
additions such as user interfaces, effects, information or
other. AR lets you deliver immersive experiences that
seamlessly add virtual objects to the real world.
Virtual reality for medical use
Virtual Reality is about recreating reality.
In VR, the current reality is fully replaced by a virtual environment
which can be setup in many different ways, within the
constraints of the platform that the application is running.
Our job as content designers is to help people interact seamlessly with the AR or VR experience
by using clear signifiers, text labels and easy-to-use instructions.
Here are 8 content design principles for AR/VR experiences.
- Let the user drive the narrative
- Help the user onboard and leave clues
- Design for the environment
- Aim for simple, familiar experiences
- Prioritize real estate
- Help users learn gestures and controls
- Make it useful and delightful
- Always spark joy
Hints used in AR
1. Let the user drive the narrative
In AR and VR experiences, the user is the main character. It's the user’s experience to unfold as they wish. And it’s important for them to drive and feel in control of the narrative. Think of it as the user co-authoring the story. There is no “happy-path” through an AR/VR experience. The user defines the path. Our job is to design, label and create narratives around the myriad of directions the user can go.
We accomplish this by designing for multiple personas and stories. Anticipate the primary users, but resist the temptation to boil them down to one. Instead, aim for at least three persona types. Research users and identify realistic ways the users might engage with the experience. Then look at the variety of offerings on the screen and ensure they are presented equally
so the user can make clear choices.
Our role is to lead and tantalize
Let’s use AR as an example. Currently, most AR comes in the form of social AR—filters for platforms like Snapchat and TikTok. These experiences peak users' interest as soon as they start with an effect. When you look closer, you will see an effect, then a tantalizing hint at more content to discover.
In this effect published in AR/VR Journey: Augmented & Virtual Reality Magazine, the user is viewing AR on a smartphone that provides information about a store. Once they have tried on a few items, the experience offers discounts and other helpful content. The user can use the experience for navigation, but they are also offered supporting stories on the visual screen (e.g. fabric types and origin, return policies, etc.), thus broadening the engagement.
2. Help the user onboard and leave clues
Users want to follow their intuition when interacting with AR/VR. That's because it's supposed to be fun and exciting! This requires alternative or natural ways to onboard users without lengthy product tours or (gosh forbid) offline operating manuals. Ideally, experiences should be intuitive and unfold before the user as they follow their interests or needs.
Of course there are guidelines in place, but always strive to be better than the standards to drive successful onboarding.
What can we do? We can mirror actual reality by leaving clues for the user. Just like in reality when we look for clues and instructions while navigating. Like when you ride the subway and look for maps and signs to guide you to the right train.
Users want to follow their intuition when interacting with AR/VR. That's because it's supposed to be fun and exciting! This requires alternative or natural ways to onboard users without lengthy product tours or (gosh forbid) offline operating manuals. Ideally, experiences should be intuitive and unfold before the user as they follow their interests or needs.
Of course there are guidelines in place, but always strive to be better than the standards to drive successful onboarding.
What can we do? We can mirror actual reality by leaving clues for the user. Just like in reality when we look for clues and instructions while navigating. Like when you ride the subway and look for maps and signs to guide you to the right train.
Offer the user graphics, icons and dynamic animation as guides and have it remain on screen as part of the experience. And it doesn’t always have to be text or instructions. Visuals such as tooltips, hamburger menus and exit buttons help users navigate the experience
Designing a friendly and engaging AR experience can be challenging. User onboarding is the best opportunity to ensure an enjoyable experience. Think outside basic markers and overlays to deliver information. The onboarding experience needs to be more interactive, intuitive, and, most importantly, fun!
Designing a friendly and engaging AR experience can be challenging. User onboarding is the best opportunity to ensure an enjoyable experience. Think outside basic markers and overlays to deliver information. The onboarding experience needs to be more interactive, intuitive, and, most importantly, fun!
AR onboarding
Use of AR in industrial environment
3. Design for the environment
AR experiences are built upon the real world. They augment and are spatially connected with the real world. As a result, the content design of the experience must be tailored to, and capable of adapting to, any environment the user finds themselves in.
As designers, one of the best ways to learn about effective design for different environments is by getting out of your office and experimenting in the real world. View where and how a user interacts with your experience matters.
For example, imagine yourself in a kitchen. The kitchen is the environment and you are the user. You are using a tool that enhances your kitchen with augmented and virtual content. Think about how you will see the virtual content. Most likely through a smartphone’s camera or AR headset. Can you carry your phone or wear a headset in the kitchen while you cook? Will you be able to hear the interaction? How often do you need to touch the screen? The environment has an impact on the experience, that has an impact on the content.
While designing for different environments, here are some things to think about.
- Where is the user?
- Is the user sitting or standing?
- What experiences could enhance this environment?
- Is the room’s lighting sufficient to visualize the user’s environment through a phone camera?
- What hints or additional content would encourage movement?
Designers must think creatively about each type of environmental space and how the UX will change to accommodate the differences.
4. Keep it simple (and familiar)
AR and VR experiences may require new interactions, but expect users to continue to seek familiar interactions and icons. Instead of creating new interactions the user must learn, design simple experiences that leverage existing interactions as much as possible.
Design to the familiar
Many users have little experience with AR or VR. They may be unfamiliar with signifiers that someone who has reviewed several AR apps may consider standard. To guide the user into and through the experience, make 3D objects, environment, and prompts easy to see and use. We can achieve this by leveraging familiar icons and clear, unambiguous, readable instructions.
AR and VR experiences may require new interactions, but expect users to continue to seek familiar interactions and icons. Instead of creating new interactions the user must learn, design simple experiences that leverage existing interactions as much as possible.
Design to the familiar
Many users have little experience with AR or VR. They may be unfamiliar with signifiers that someone who has reviewed several AR apps may consider standard. To guide the user into and through the experience, make 3D objects, environment, and prompts easy to see and use. We can achieve this by leveraging familiar icons and clear, unambiguous, readable instructions.
Familiar interactions used in AR
Beware the feature overload
AR and VR offer so many opportunities. Because it is so new and exciting, it might be tempting to give the user as many features as possible from launch. However, as enticing as it is, keeping it simple is the best approach. Too many features introduces complexity. A user can only do so much at a time, so keeping it simple will help keep the user focused and engaged.
5. Prioritize real estate (ruthlessly)
AR/VR screens generally offer two types of content—the AR/VR reality the user is experiencing and the UI controls and overlays the user applies to customize the scene. In terms of content hierarchy, the lead experience here is AR/VR. Content designers should be cautious not to let the UI crowd the screen and compete with the AR/VR experience.
Clear the screen
In an AR feature, minimize the amount of space covered with instructions, overlays, or UI controls. While these elements play a role, they can interrupt the primary task of the user—especially on a tiny mobile screen. In a study of the Amazon app, the Nielsen Norman Group found that “while the overlay hosted useful controls such as the Add to Cart or Details buttons, it also hindered the main task of viewing and interacting with the AR object in the real-life environment.”
UX writing should strongly embrace a “less is more” ethic, avoiding lengthy paragraphs and scrolling. Similarly, content designers should limit the use of tooltips and microcopy and consider timing the messages or options that appear to the user so they are contextually relevant and don’t overwhelm the screen. Storytelling and notifications should also be reduced to keep the user focused on their task.The user needs to be able to easily see the AR/VR 3D objects, environment, and prompts that guide the experience. This can be achieved through colors, combined with clear, unambiguous, readable instructions.
In this example from Meta, the screen real estate is shared well between the experience and the controls. Additional controls appear as the user explores a direction.
AR and VR offer so many opportunities. Because it is so new and exciting, it might be tempting to give the user as many features as possible from launch. However, as enticing as it is, keeping it simple is the best approach. Too many features introduces complexity. A user can only do so much at a time, so keeping it simple will help keep the user focused and engaged.
5. Prioritize real estate (ruthlessly)
AR/VR screens generally offer two types of content—the AR/VR reality the user is experiencing and the UI controls and overlays the user applies to customize the scene. In terms of content hierarchy, the lead experience here is AR/VR. Content designers should be cautious not to let the UI crowd the screen and compete with the AR/VR experience.
Clear the screen
In an AR feature, minimize the amount of space covered with instructions, overlays, or UI controls. While these elements play a role, they can interrupt the primary task of the user—especially on a tiny mobile screen. In a study of the Amazon app, the Nielsen Norman Group found that “while the overlay hosted useful controls such as the Add to Cart or Details buttons, it also hindered the main task of viewing and interacting with the AR object in the real-life environment.”
UX writing should strongly embrace a “less is more” ethic, avoiding lengthy paragraphs and scrolling. Similarly, content designers should limit the use of tooltips and microcopy and consider timing the messages or options that appear to the user so they are contextually relevant and don’t overwhelm the screen. Storytelling and notifications should also be reduced to keep the user focused on their task.The user needs to be able to easily see the AR/VR 3D objects, environment, and prompts that guide the experience. This can be achieved through colors, combined with clear, unambiguous, readable instructions.
In this example from Meta, the screen real estate is shared well between the experience and the controls. Additional controls appear as the user explores a direction.
AR interaction UI
|
AR interaction selection
|
External view of AR interaction
Consider a “be there when you need it, gone when you don’t” approach to content. In an ideal experience, the user would only see content when it is contextually relevant.
Use of hand gestures to control VR
6. Help users learn gestures and controls
There are various new interaction types within an AR/VR experience, including:
Prioritize existing gestures
Users expect intuitive gestures and will seek the familiar. This means that we should reuse existing gestures and movements from current experiences as much as possible and resist creating new gestures. In the same way we query users with a card sort to determine an intuitive order, we should query users to determine what gestures they use to achieve a given outcome.
UX writing should aim to describe gestures in familiar and accurate ways (spread fingers vs. blip or blop). This takes time to get right, including testing and potentially a card sort. Visual affordances can help, but only if they are easily recognizable. Advanced screen display and lighting technology makes it possible for digitally rendered objects to appear naturally in the user’s environment. UI patterns can complement a 3D mobile AR experience, but it’s still important that they stand out as interactive components so users recognize selection state and functionality.
There are various new interaction types within an AR/VR experience, including:
- Gestures
- Voice
- Gaze
- Arm movements
Prioritize existing gestures
Users expect intuitive gestures and will seek the familiar. This means that we should reuse existing gestures and movements from current experiences as much as possible and resist creating new gestures. In the same way we query users with a card sort to determine an intuitive order, we should query users to determine what gestures they use to achieve a given outcome.
UX writing should aim to describe gestures in familiar and accurate ways (spread fingers vs. blip or blop). This takes time to get right, including testing and potentially a card sort. Visual affordances can help, but only if they are easily recognizable. Advanced screen display and lighting technology makes it possible for digitally rendered objects to appear naturally in the user’s environment. UI patterns can complement a 3D mobile AR experience, but it’s still important that they stand out as interactive components so users recognize selection state and functionality.
7. Make it useful (and delightful)
AR has great potential for creating immersive experiences that educate the public or entertain it by telling a story. Participants in a Nielsen Norman Group study repeatedly shared their excitement about immersive educational AR stories. A user of Kinfolk app thought the app would be helpful to teachers and educators:
I am learning about historical icons in the AR mode. This app is cool. […] I think a lot of educators or people that may work in a school may be interested in something like that because they can create an activity with this app and because it seems very engaging.”
Beyond enjoyable, the AR feature should bring value to the interaction and be useful. Even though users find good augmented reality delightful, creating a usable AR interaction is not easy. Many AR apps suffer from having big features and big usability issues. If it’s clunky or hard to use, users will not bother with the AR feature, especially when it adds little or no value to their experience.
Google Arts & Culture
In the same Nielsen Norman Group study, a participant using Google Arts & Culture expressed how viewing famous paintings in the AR mode was not effective and actually interfered with the experience. She commented:
I don't understand it […] The painting is awesome, but the fact that it's in my living room […] seems ridiculous. And I don't know why they came up with this. […] I mean, unless you have, like, a super-creative mind and you're out in the world and you want to, like, juxtapose Moses and the 10 commandments by a river or a lake or something that would mean something artistically. But I think the average person who's looking at this doesn't want Moses in their living room.”
Just as we content designers do elsewhere, answer the “why” first to establish the need for the AR or VR element. Try to define for yourself and the team how the AR or VR element is adding to the experience for the user. Are we enhancing the user’s experience? Users' familiarity with AR/VR experiences will become more widespread. Until then, avoid gratuitous use of AR/VR for AR/VR sake.
Snapchat AR experience
8. Always spark joy
Above all, AR and VR experiences should be fun to use. They are capable of augmenting our reality or making the impossible become possible. They can transport you to a rock concert or bring a friend into your living room for a glass of wine. If they are useful in adding to or recreating our reality in some way, that reality should be joyful. Generally speaking, the most used AR/VR experiences are also the ones that are the most fun.
In the end, AR and VR isn’t about escaping life, it’s about improving it. As most usability studies will show: when something is fun, people will want to use it, and you will see more results. So be the voice of fun on your team, talk about it, have fun with it and see how you feel. These are human experiences—who better to test than you?
AR and VR have the ability to improve people’s lives simply by enabling them to experience something that wasn’t possible before. Applying UX principles to AR/VR can help content designers create experiences that are clear, human, useful, and perhaps most of all, fun.
Above all, AR and VR experiences should be fun to use. They are capable of augmenting our reality or making the impossible become possible. They can transport you to a rock concert or bring a friend into your living room for a glass of wine. If they are useful in adding to or recreating our reality in some way, that reality should be joyful. Generally speaking, the most used AR/VR experiences are also the ones that are the most fun.
In the end, AR and VR isn’t about escaping life, it’s about improving it. As most usability studies will show: when something is fun, people will want to use it, and you will see more results. So be the voice of fun on your team, talk about it, have fun with it and see how you feel. These are human experiences—who better to test than you?
AR and VR have the ability to improve people’s lives simply by enabling them to experience something that wasn’t possible before. Applying UX principles to AR/VR can help content designers create experiences that are clear, human, useful, and perhaps most of all, fun.
Resources:
Geoffrey O’Brian
Geoffrey is a Content Designer and creative leader who uses content design to champion user needs and help businesses create user-driven content.