With the advancements in Augmented Reality (AR) and Virtual Reality (VR), we are now capable of creating realistic environments, where the user can interact with additional layers of information that is not available in the physical space. As a result there is an increasing level of interest in developing virtual 3D scenes for prototyping and evaluating human behavior within different design alternatives. In recent years, many fields such as medicine, education, and military, have adopted mixed reality environments for training and education purposes. Further, these devices have shown to provide realistic simulations of real-world environments where researchers can study human behavior in different contextual settings. In this research we focus on developing automated approaches where AR/VR devices (e.g., HoloLens, Magic Leap, or HTC Vive) can be used to provide real-time assistance to users’ while interacting with virtual objects or different layers of information.