Evaluating Perceptual Deviations in Video See-Through Head-Mounted Displays while Utilizing Physical Touchscreens

Extended reality technology has become a useful tool in many applications, but still suffers from visual deviations that can hamper the utility of the technology. This paper discusses the types of persisting visual deviations experienced when observing the natural world through video see-through head-mounted displays. A generalizable method to measure the effect of these deviations on real-world interaction is designed and used in a human-in-the-loop experiment. The experiment compared video see-through sight through an head-mounted display with normal eyesight in a static set-up, focusing on (camera) lens distortions and display deviations. Participants interacted with a real touchscreen, locating the position of flashed markers shortly after disappearance comparing both conditions to check for deviations in position and time. Results show significant larger mean distance errors between the interaction locations and the original marker positions for video see-through compared to normal eyesight. Moreover, errors increase towards the screen periphery. No significant distance error improvement over time was found, however, response times did significantly decrease for both types of sight.