SwitchSpace: Understanding Context-Aware Peeking Between VR and Desktop Interfaces

Key Takeaways

• Cross-reality work like 3D modelling or VR prototyping involves quick, temporary switches between VR and desktop, which can be cumbersome.
• Breaking down these switches by separating input and viewing devices enables more design opportunities.
• UI can make use of context like seated/standing, input devices, or viewing devices, to dynamically enable input/output techniques.
• Designing techniques to quickly and temporarily peek between interfaces, without needing to switch hardware, can make cross-reality workflows faster, more accurate, and more comfortable.

Abstract

Cross-reality tasks, like creating or consuming virtual reality (VR) content, often involve inconvenient or distracting switches between desktop and VR. An initial formative study explores cross-reality switching habits, finding most switches are momentary “peeks” between interfaces, with specific habits determined by current context. The results inform a design space for context-aware “peeking” techniques that allow users to view or interact with desktop from VR, and vice versa, without fully switching. We implemented a set of peeking techniques and evaluated them in two levels of a cross-reality task: one requiring only viewing, and another requiring input and viewing. Peeking techniques made task completion faster, with increased input accuracy and reduced perceived workload.

Formative Study and Design Space

We conducted a formative study to understand how people switch between VR and desktop interfaces. We found that most switches are momentary "peeks" between interfaces, with specific habits determined by current context.

Context, in our case, describes the user's current situation across three factors:

1. Input devices: the devices used to interact with the system, such as a keyboard, mouse, or game controller.
2. Viewing devices: the devices used to view the system, such as a VR headset or a desktop monitor.
3. Memory: What were they doing before? What are they likely to do next?

This led us to create a design space for context-aware switches between VR and desktop, which we express as a state machine:

Techniques

We implemented several peeking techniques within this design space, including:

Experiment

We recruited 16 participants to complete an experiment where they had to solve simple math problems. The two halves of the math problem were split between VR and desktop, and participants had to use a peeking technique or fully switch to solve the problem.

Conditions

Results

The ability to use these peeking techniques made solving the problem faster and more accurate than when users were forced to fully switch.

The ability to use peeking techniques resulted in significantly fewer hardware transitions.

The ability to use peeking techniques resulted in significantly reduced perceived workload (as reported by participants completing the NASA-TLX questionnaire).

The ability to use peeking techniques resulted in users removing the headset significantly less.

What Can We Learn?

1. Minimize hardware changes: Shuffling hardware around is the biggest factor in what makes cross-reality workflows cumbersome. Designing techniques to minimize those issues can make these workflows far faster and more comfortable.
2. Design for Phsyical Space: A big factor in our experiment was where the task was placed in VR, which also impacted our results. Peeking techniques that minimize real-world movement can make tasks faster, especially in more constrained real-world environments.

Publication

Johann Wentzel, Fraser Anderson, George Fitzmaurice, Tovi Grossman, and Daniel Vogel. 2024. SwitchSpace: Understanding Context-Aware Peeking Between VR and Desktop Interfaces. In CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 4, 1–16. https://doi.org/10.1145/3613904.3642358

@inproceedings{Wentzel2024,
   author = {Wentzel, Johann and Anderson, Fraser and Fitzmaurice, George and Grossman, Tovi and Vogel, Daniel},
   title = {SwitchSpace: Understanding Context-Aware Peeking Between VR and Desktop Interfaces},
   year = {2024},
   isbn = {9798400703300},
   publisher = {Association for Computing Machinery},
   address = {New York, NY, USA},
   url = {https://doi.org/10.1145/3613904.3642358},
   doi = {10.1145/3613904.3642358},
   booktitle = {CHI Conference on Human Factors in Computing Systems},
   numpages = {16},
   location = {Honolulu, HI, USA},
   series = {CHI '24}
}

Contact Us

• Johann Wentzel (University of Waterloo)
  
• Fraser Anderson (Autodesk Research)
  
• George Fitzmaurice (Autodesk Research)
  
• Tovi Grossman (University of Toronto)
  
• Daniel Vogel (University of Waterloo)