Melissa Steininger, M. Sc.
Personalized Digital Health and Telemedicine
Affiliation:
Department for Epileptology
University Hospital Bonn
Medical Faculty
University of Bonn
Location:
Venusberg-Campus 1,
Building 74, Room 2G-016
53127 Bonn, Germany
Telephone: +49-228/287-52171
Email: melissa.steininger@ukbonn.de
Short CV:
Melissa Steininger earned her Bachelor’s degree in Cognitive Science (B.Sc.) in 2020 from the University of Osnabrück and her Master’s degree in Visual Computing and Games Technology (M.Sc.) in 2023 from the University of Applied Sciences Bonn-Rhein-Sieg. She is now doing her Ph.D. in Computer Science at the University Hospital Bonn/University of Bonn.
Publications
2026
Jansen, Anna; Waldow, Kristoffer; Pötter, Sebastian; Civelek, Turhan; Steininger, Melissa; Perret, Jerome; Wellmann, Markus; Stein, Steffen-Sascha; Lähner, David; Welle, Kristian; Fuhrmann, Arnulph; Krüger, Björn
VIRTOSHA - A VR Training Simulation for Osteosynthesis Procedures with Force Feedback and Tissue Simulation Proceedings Article
In: 2026 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , pp. 921-930, IEEE Computer Society, 2026.
@inproceedings{jansen2026b,
title = {VIRTOSHA - A VR Training Simulation for Osteosynthesis Procedures with Force Feedback and Tissue Simulation},
author = {Anna Jansen and Kristoffer Waldow and Sebastian Pötter and Turhan Civelek and Melissa Steininger and Jerome Perret and Markus Wellmann and Steffen-Sascha Stein and David Lähner and Kristian Welle and Arnulph Fuhrmann and Björn Krüger},
url = {https://doi.ieeecomputersociety.org/10.1109/VRW70859.2026.00171},
doi = {10.1109/VRW70859.2026.00171},
year = {2026},
date = {2026-03-31},
urldate = {2026-03-31},
booktitle = {2026 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) },
pages = {921-930},
publisher = {IEEE Computer Society},
abstract = {Osteosynthesis training requires development of force-sensitive manual skills and an understanding of workflows, which are difficult to acquire through theoretical instruction or cadaver-based training. While Virtual Reality (VR) offers new opportunities for surgical training, existing systems often focus on isolated subtasks, lacking integrated support for realistic interaction, procedural logic, and adaptability. This paper presents a work-in-progress VR training system designed for workflow-oriented osteosynthesis training. The system combines force feedback, physics-based tissue simulation and robust hand tracking in a modular architecture. Additionally, an expert-driven authoring workflow enables medical professionals to define and adapt training scenarios without programming.
Using a reference scenario for fibular fracture osteosynthesis, we describe the system design, core components, and current implementation status. We further discuss technical trade-offs, limitations, and directions for future validation. Our system establishes a foundation for force-sensitive, workflow-oriented VR training and serves as a basis for future studies in surgical education.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Osteosynthesis training requires development of force-sensitive manual skills and an understanding of workflows, which are difficult to acquire through theoretical instruction or cadaver-based training. While Virtual Reality (VR) offers new opportunities for surgical training, existing systems often focus on isolated subtasks, lacking integrated support for realistic interaction, procedural logic, and adaptability. This paper presents a work-in-progress VR training system designed for workflow-oriented osteosynthesis training. The system combines force feedback, physics-based tissue simulation and robust hand tracking in a modular architecture. Additionally, an expert-driven authoring workflow enables medical professionals to define and adapt training scenarios without programming.
Using a reference scenario for fibular fracture osteosynthesis, we describe the system design, core components, and current implementation status. We further discuss technical trade-offs, limitations, and directions for future validation. Our system establishes a foundation for force-sensitive, workflow-oriented VR training and serves as a basis for future studies in surgical education.
Using a reference scenario for fibular fracture osteosynthesis, we describe the system design, core components, and current implementation status. We further discuss technical trade-offs, limitations, and directions for future validation. Our system establishes a foundation for force-sensitive, workflow-oriented VR training and serves as a basis for future studies in surgical education.
Steininger, Melissa; Jansen, Anna; Müllers, Johannes; von Wrede, Randi; Krüger, Björn
Toward Interpretable Cognitive Screening in Epilepsy: Eye Tracking in a VR Trail Making Test Proceedings Article
In: IEEE VR 2026 Workshop: GEMINI, 2026.
@inproceedings{steininger2026b,
title = {Toward Interpretable Cognitive Screening in Epilepsy: Eye Tracking in a VR Trail Making Test},
author = {Melissa Steininger and Anna Jansen and Johannes Müllers and Randi von Wrede and Björn Krüger},
url = {https://www.computer.org/csdl/proceedings-article/vrw/2026/052900a106/2gdqrMPo2s0},
year = {2026},
date = {2026-03-31},
urldate = {2026-03-31},
booktitle = {IEEE VR 2026 Workshop: GEMINI},
abstract = {Cognitive screening is a routine component of epilepsy care. Established pen-and-paper instruments such as the Trail Making Test (TMT) primarily yield summary outcomes (e.g., completion time) that provide limited insight into visual search and executive-control processes affected by epilepsy-related brain network dysfunction. We present an eye-tracked Virtual Reality TMT (VR-TMT) as a controlled research instrument that enables process-level interpretable measurements. The system synchronizes continuous eye-movement streams with timestamped task events (task start/stop and node selections) and logs gaze-to-Area-of-Interest (AOI) intersections. To reduce VR-specific confounds that can compromise cognitive interpretation, we specify concrete design guidelines for 3D stimulus geometry and the VR+eye-tracking setup (e.g., viewing distance, field-of-view placement, target size).
In a feasibility pilot (n=8) usability ratings were favorable and cybersickness was low. Building on this foundation, we outline an analysis framework that derives contextualized gaze features and evaluates their added value in explaining established cognitive screening outcomes in epilepsy cohorts.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Cognitive screening is a routine component of epilepsy care. Established pen-and-paper instruments such as the Trail Making Test (TMT) primarily yield summary outcomes (e.g., completion time) that provide limited insight into visual search and executive-control processes affected by epilepsy-related brain network dysfunction. We present an eye-tracked Virtual Reality TMT (VR-TMT) as a controlled research instrument that enables process-level interpretable measurements. The system synchronizes continuous eye-movement streams with timestamped task events (task start/stop and node selections) and logs gaze-to-Area-of-Interest (AOI) intersections. To reduce VR-specific confounds that can compromise cognitive interpretation, we specify concrete design guidelines for 3D stimulus geometry and the VR+eye-tracking setup (e.g., viewing distance, field-of-view placement, target size).
In a feasibility pilot (n=8) usability ratings were favorable and cybersickness was low. Building on this foundation, we outline an analysis framework that derives contextualized gaze features and evaluates their added value in explaining established cognitive screening outcomes in epilepsy cohorts.
In a feasibility pilot (n=8) usability ratings were favorable and cybersickness was low. Building on this foundation, we outline an analysis framework that derives contextualized gaze features and evaluates their added value in explaining established cognitive screening outcomes in epilepsy cohorts.
2025
Steininger, Melissa; Marquardt, Alexander; Perusquía-Hernández, Monica; Lehnort, Marvin; Otsubo, Hiromu; Dollack, Felix; Kruijff, Ernst; Krüger, Björn; Kiyokawa, Kiyoshi; Riecke, Bernhard E.
The Awe-some Spectrum: Self-Reported Awe Varies by Eliciting Scenery and Presence in Virtual Reality, and the User's Nationality Proceedings Article
In: 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 1267-1277, 2025.
@inproceedings{steininger2025c,
title = {The Awe-some Spectrum: Self-Reported Awe Varies by Eliciting Scenery and Presence in Virtual Reality, and the User's Nationality},
author = {Melissa Steininger and Alexander Marquardt and Monica Perusquía-Hernández and Marvin Lehnort and Hiromu Otsubo and Felix Dollack and Ernst Kruijff and Björn Krüger and Kiyoshi Kiyokawa and Bernhard E. Riecke
},
url = {https://digital-health-bonn.de/wp-content/uploads/2026/05/2025_Steininger_Awe.pdf, Preprint version of the Paper},
doi = {10.1109/ISMAR67309.2025.00132},
year = {2025},
date = {2025-11-11},
urldate = {2025-11-11},
booktitle = {2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
pages = {1267-1277},
abstract = {Awe is a multifaceted emotion often associated with the perception of vastness, that challenges existing mental frameworks. Despite its growing relevance in affective computing and psychological research, awe remains difficult to elicit and measure.
This raises the research questions of how awe can be effectively elicited, which factors are associated with the experience of awe, and whether it can reliably be measured using biosensors.
For this study, we designed ten immersive Virtual Reality (VR) scenes with dynamic transitions from narrow to vast environments. These scenes were used to explore how awe relates to environmental features (abstract, human-made, nature), personality traits, and country of origin. We collected skin conductance, respiration data, and self-reported awe and presence from participants from Germany, Japan, and Jordan.
Our results indicate that self-reported awe varies significantly across countries and scene types. In particular, a scene depicting outer space elicited the strongest awe. Scenes that elicited high self-reported awe also induced a stronger sense of presence. However, we found no evidence that awe ratings are correlated with physiological responses.
These findings challenge the assumption that awe is reliably reflected in autonomic arousal and underscore the importance of cultural and perceptual context.
Our study offers new insights into how immersive VR can be designed to elicit awe, and suggests that subjective reports—rather than physiological signals—remain the most consistent indicators of emotional impact.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Awe is a multifaceted emotion often associated with the perception of vastness, that challenges existing mental frameworks. Despite its growing relevance in affective computing and psychological research, awe remains difficult to elicit and measure.
This raises the research questions of how awe can be effectively elicited, which factors are associated with the experience of awe, and whether it can reliably be measured using biosensors.
For this study, we designed ten immersive Virtual Reality (VR) scenes with dynamic transitions from narrow to vast environments. These scenes were used to explore how awe relates to environmental features (abstract, human-made, nature), personality traits, and country of origin. We collected skin conductance, respiration data, and self-reported awe and presence from participants from Germany, Japan, and Jordan.
Our results indicate that self-reported awe varies significantly across countries and scene types. In particular, a scene depicting outer space elicited the strongest awe. Scenes that elicited high self-reported awe also induced a stronger sense of presence. However, we found no evidence that awe ratings are correlated with physiological responses.
These findings challenge the assumption that awe is reliably reflected in autonomic arousal and underscore the importance of cultural and perceptual context.
Our study offers new insights into how immersive VR can be designed to elicit awe, and suggests that subjective reports—rather than physiological signals—remain the most consistent indicators of emotional impact.
This raises the research questions of how awe can be effectively elicited, which factors are associated with the experience of awe, and whether it can reliably be measured using biosensors.
For this study, we designed ten immersive Virtual Reality (VR) scenes with dynamic transitions from narrow to vast environments. These scenes were used to explore how awe relates to environmental features (abstract, human-made, nature), personality traits, and country of origin. We collected skin conductance, respiration data, and self-reported awe and presence from participants from Germany, Japan, and Jordan.
Our results indicate that self-reported awe varies significantly across countries and scene types. In particular, a scene depicting outer space elicited the strongest awe. Scenes that elicited high self-reported awe also induced a stronger sense of presence. However, we found no evidence that awe ratings are correlated with physiological responses.
These findings challenge the assumption that awe is reliably reflected in autonomic arousal and underscore the importance of cultural and perceptual context.
Our study offers new insights into how immersive VR can be designed to elicit awe, and suggests that subjective reports—rather than physiological signals—remain the most consistent indicators of emotional impact.
Jansen, Anna; Morev, Nikita; Steininger, Melissa; Müllers, Johannes; Krüger, Björn
Synthetic Hand Dataset Generation: Multi-View Rendering and Annotation with Blender Proceedings Article
In: 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 809-810, IEEE Computer Society, 2025.
@inproceedings{jansen2025c,
title = {Synthetic Hand Dataset Generation: Multi-View Rendering and Annotation with Blender},
author = {Anna Jansen and Nikita Morev and Melissa Steininger and Johannes Müllers and Björn Krüger},
url = {https://www.computer.org/csdl/proceedings-article/ismar-adjunct/2025/934700a809/2bKcNnpvzTG},
doi = {10.1109/ISMAR-Adjunct68609.2025.00201},
year = {2025},
date = {2025-10-06},
urldate = {2025-10-06},
booktitle = {2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
pages = {809-810},
publisher = {IEEE Computer Society},
abstract = {Pose estimation is a common method for precise handtracking, which is important for natural interaction in virtual reality (VR). However, training those models requires large-scale datasets with accurate 3D annotations. Those are difficult to obtain due to the time-consuming data collection and the limited variety in captured scenarios. We present a work-in-progress Blender-based pipeline for generating synthetic multi-view hand datasets. Our system simulates Ultraleap Stereo IR 170-style images and extracts joint positions directly from a rigged hand model, eliminating the need for manual labeling or external tracking processes. The current pipeline version supports randomized static poses with per-frame annotations of joint positions, camera parameters, and rendered images. While extended hand variation, animation features, and different sensor-type simulations are still in progress, our pipeline already provides a flexible foundation for customizable dataset generation and reproducible hand-tracking model training.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Pose estimation is a common method for precise handtracking, which is important for natural interaction in virtual reality (VR). However, training those models requires large-scale datasets with accurate 3D annotations. Those are difficult to obtain due to the time-consuming data collection and the limited variety in captured scenarios. We present a work-in-progress Blender-based pipeline for generating synthetic multi-view hand datasets. Our system simulates Ultraleap Stereo IR 170-style images and extracts joint positions directly from a rigged hand model, eliminating the need for manual labeling or external tracking processes. The current pipeline version supports randomized static poses with per-frame annotations of joint positions, camera parameters, and rendered images. While extended hand variation, animation features, and different sensor-type simulations are still in progress, our pipeline already provides a flexible foundation for customizable dataset generation and reproducible hand-tracking model training.
Mustafa, Sarah Al-Haj; Jansen, Anna; Steininger, Melissa; Müllers, Johannes; Surges, Rainer; von Wrede, Randi; Krüger, Björn; Helmstaedter, Christoph
Eyes on Cognition: Exploring Oculomotor Correlates of Cognitive Function in Patients with Epilepsy Journal Article
In: Epilepsy & Behavior, vol. 173, iss. December 2025, no. 110562, 2025.
@article{alhaj2025,
title = {Eyes on Cognition: Exploring Oculomotor Correlates of Cognitive Function in Patients with Epilepsy},
author = {Sarah Al-Haj Mustafa and Anna Jansen and Melissa Steininger and Johannes Müllers and Rainer Surges and Randi von Wrede and Björn Krüger and Christoph Helmstaedter},
doi = {10.1016/j.yebeh.2025.110562},
year = {2025},
date = {2025-06-30},
urldate = {2025-06-30},
journal = {Epilepsy & Behavior},
volume = {173},
number = {110562},
issue = {December 2025},
abstract = {Objective
This study investigates the relationship between eye tracking parameters and cognitive performance during the Trail Making Test (TMT) in individuals with epilepsy and healthy controls. By analyzing ocular behaviors such as saccade velocity, fixation duration, and pupil diameter, we aim to determine how these metrics reflect executive functioning and attentional control.
Methods
A sample of 95 participants with epilepsy and 34 healthy controls completed the TMT while their eye movements were recorded. Partial correlations, controlling for age, sex, education, medication count, seizure status and epilepsy duration, examined associations between eye tracking measures and cognitive performance derived from EpiTrack and TMT performance.
Results
In the patient group, faster TMT-A performance was associated with shorter fix- ation durations (r = 0.31, p = 0.006). Lower minimum saccade velocity correlated with slower performance on both TMT-A (r = −0.35, p = 0.002) and TMT-B (r = −0.40, p<0.001), whereas higher peak saccade velocities were linked to worse performance (TMT-A: r = 0.45, p<0.001; TMT-B: r = 0.41, p<0.001). Pupil diameter findings indicated that slower TMT performance was associated with smaller minimum pupil sizes (r = −0.23 to r = −0.36), wich may indicate increased cognitive effort and attentional load. Higher EpiTrack scores also correlated with a smaller minimum pupil diameter − but only during the more demanding TMT-B − and with a more restricted saccade velocity range, reflecting greater motor control and attentional stability. No significant correlations emerged within the control group.
Conclusion
These findings highlight the potential of eye tracking as a non-invasive tool for assessing cognitive function in epilepsy. Efficient cognitive performance was characterized by stable and controlled eye movements, whereas impaired performance involved erratic saccade dynamics and prolonged fixations. Importantly, eye tracking parameters provide additional information beyond simple speed measurements, potentially enhancing the differential diagnostic capabilities of the TMT in epilepsy. The observed associations between oculomotor parameters and cognitive performance were not present in the control group, suggesting that these relationships may be specific to epilepsy. Future research should investigate whether both basic and advanced metrics of search strategies are sensitive to disease dynamics and treatment effects in epilepsy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Objective
This study investigates the relationship between eye tracking parameters and cognitive performance during the Trail Making Test (TMT) in individuals with epilepsy and healthy controls. By analyzing ocular behaviors such as saccade velocity, fixation duration, and pupil diameter, we aim to determine how these metrics reflect executive functioning and attentional control.
Methods
A sample of 95 participants with epilepsy and 34 healthy controls completed the TMT while their eye movements were recorded. Partial correlations, controlling for age, sex, education, medication count, seizure status and epilepsy duration, examined associations between eye tracking measures and cognitive performance derived from EpiTrack and TMT performance.
Results
In the patient group, faster TMT-A performance was associated with shorter fix- ation durations (r = 0.31, p = 0.006). Lower minimum saccade velocity correlated with slower performance on both TMT-A (r = −0.35, p = 0.002) and TMT-B (r = −0.40, p<0.001), whereas higher peak saccade velocities were linked to worse performance (TMT-A: r = 0.45, p<0.001; TMT-B: r = 0.41, p<0.001). Pupil diameter findings indicated that slower TMT performance was associated with smaller minimum pupil sizes (r = −0.23 to r = −0.36), wich may indicate increased cognitive effort and attentional load. Higher EpiTrack scores also correlated with a smaller minimum pupil diameter − but only during the more demanding TMT-B − and with a more restricted saccade velocity range, reflecting greater motor control and attentional stability. No significant correlations emerged within the control group.
Conclusion
These findings highlight the potential of eye tracking as a non-invasive tool for assessing cognitive function in epilepsy. Efficient cognitive performance was characterized by stable and controlled eye movements, whereas impaired performance involved erratic saccade dynamics and prolonged fixations. Importantly, eye tracking parameters provide additional information beyond simple speed measurements, potentially enhancing the differential diagnostic capabilities of the TMT in epilepsy. The observed associations between oculomotor parameters and cognitive performance were not present in the control group, suggesting that these relationships may be specific to epilepsy. Future research should investigate whether both basic and advanced metrics of search strategies are sensitive to disease dynamics and treatment effects in epilepsy.
This study investigates the relationship between eye tracking parameters and cognitive performance during the Trail Making Test (TMT) in individuals with epilepsy and healthy controls. By analyzing ocular behaviors such as saccade velocity, fixation duration, and pupil diameter, we aim to determine how these metrics reflect executive functioning and attentional control.
Methods
A sample of 95 participants with epilepsy and 34 healthy controls completed the TMT while their eye movements were recorded. Partial correlations, controlling for age, sex, education, medication count, seizure status and epilepsy duration, examined associations between eye tracking measures and cognitive performance derived from EpiTrack and TMT performance.
Results
In the patient group, faster TMT-A performance was associated with shorter fix- ation durations (r = 0.31, p = 0.006). Lower minimum saccade velocity correlated with slower performance on both TMT-A (r = −0.35, p = 0.002) and TMT-B (r = −0.40, p<0.001), whereas higher peak saccade velocities were linked to worse performance (TMT-A: r = 0.45, p<0.001; TMT-B: r = 0.41, p<0.001). Pupil diameter findings indicated that slower TMT performance was associated with smaller minimum pupil sizes (r = −0.23 to r = −0.36), wich may indicate increased cognitive effort and attentional load. Higher EpiTrack scores also correlated with a smaller minimum pupil diameter − but only during the more demanding TMT-B − and with a more restricted saccade velocity range, reflecting greater motor control and attentional stability. No significant correlations emerged within the control group.
Conclusion
These findings highlight the potential of eye tracking as a non-invasive tool for assessing cognitive function in epilepsy. Efficient cognitive performance was characterized by stable and controlled eye movements, whereas impaired performance involved erratic saccade dynamics and prolonged fixations. Importantly, eye tracking parameters provide additional information beyond simple speed measurements, potentially enhancing the differential diagnostic capabilities of the TMT in epilepsy. The observed associations between oculomotor parameters and cognitive performance were not present in the control group, suggesting that these relationships may be specific to epilepsy. Future research should investigate whether both basic and advanced metrics of search strategies are sensitive to disease dynamics and treatment effects in epilepsy.
Steininger, Melissa; Jansen, Anna; Welle, Kristian; Krüger, Björn
Optimized Sensor Position Detection: Improving Visual Sensor Setups for Hand Tracking in VR Proceedings Article
In: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 1388-1389, 2025.
@inproceedings{steininger2025b,
title = {Optimized Sensor Position Detection: Improving Visual Sensor Setups for Hand Tracking in VR},
author = {Melissa Steininger and Anna Jansen and Kristian Welle and Björn Krüger},
url = {https://ieeexplore.ieee.org/document/10972713},
doi = {10.1109/VRW66409.2025.00340},
year = {2025},
date = {2025-03-12},
urldate = {2025-03-12},
booktitle = {2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
pages = {1388-1389},
abstract = {Hand tracking plays an important role in many Virtual Reality (VR) applications, enabling natural user interactions. Achieving precise tracking is often challenged by occlusion and suboptimal sensor placement. To address these challenges, we developed the Sensor Positioning Simulator, a versatile tool designed to optimize sensor placement. To demonstrate its utility, we simulated scenes from the VIRTOSHA project, a VR-based surgical training platform. Evaluations show that the tool effectively positioned sensors to achieve maximum hand surface visibility and full hand movement area coverage, even in occlusion-heavy environments. Future developments include support for animated simulations and validation through real-world experiments.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Hand tracking plays an important role in many Virtual Reality (VR) applications, enabling natural user interactions. Achieving precise tracking is often challenged by occlusion and suboptimal sensor placement. To address these challenges, we developed the Sensor Positioning Simulator, a versatile tool designed to optimize sensor placement. To demonstrate its utility, we simulated scenes from the VIRTOSHA project, a VR-based surgical training platform. Evaluations show that the tool effectively positioned sensors to achieve maximum hand surface visibility and full hand movement area coverage, even in occlusion-heavy environments. Future developments include support for animated simulations and validation through real-world experiments.
2024
Otsubo, Hiromu; Lehnort, Marvin; Steininger, Melissa; Marquardt, Alexander; Dollack, Felix; Hirao, Yutaro; Perusquía-Hernández, Monica; Uchiyama, Hideaki; Kruijff, Ernst; Riecke, Bernhard; Kiyokawa, Kiyoshi
First-Person Perspective Induces Stronger Feelings of Awe and Presence Compared to Third-Person Perspective in Virtual Reality Proceedings Article
In: ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction, pp. 439 - 448, 2024.
@inproceedings{Otsubo2024,
title = {First-Person Perspective Induces Stronger Feelings of Awe and Presence Compared to Third-Person Perspective in Virtual Reality},
author = {Hiromu Otsubo and Marvin Lehnort and Melissa Steininger and Alexander Marquardt and Felix Dollack and Yutaro Hirao and Monica Perusquía-Hernández and Hideaki Uchiyama and Ernst Kruijff and Bernhard Riecke and Kiyoshi Kiyokawa},
doi = {https://doi.org/10.1145/3678957.3685753},
year = {2024},
date = {2024-11-04},
urldate = {2024-11-04},
booktitle = {ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction},
pages = {439 - 448},
abstract = {Awe is a complex emotion described as a perception of vastness and a need for accommodation to integrate new, overwhelming experiences. Virtual Reality (VR) has recently gained attention as a convenient means to facilitate experiences of awe. In VR, a first-person perspective might increase awe due to its immersive nature, while a third-person perspective might enhance the perception of vastness. However, the impact of VR perspectives on experiencing awe has not been thoroughly examined. We created two types of VR scenes: one with elements designed to induce high awe, such as a snowy mountain, and a low awe scene without such elements. We compared first-person and third-person perspectives in each scene. Forty-two participants explored the VR scenes, with their physiological responses captured by electrocardiogram (ECG) and face tracking (FT). Subsequently, participants self-reported their experience of awe (AWE-S) and presence (IPQ) within VR. The results revealed that the first-person perspective induced stronger feelings of awe and presence than the third-person perspective. The findings of this study provide useful guidelines for designing VR content that enhances emotional experiences.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Awe is a complex emotion described as a perception of vastness and a need for accommodation to integrate new, overwhelming experiences. Virtual Reality (VR) has recently gained attention as a convenient means to facilitate experiences of awe. In VR, a first-person perspective might increase awe due to its immersive nature, while a third-person perspective might enhance the perception of vastness. However, the impact of VR perspectives on experiencing awe has not been thoroughly examined. We created two types of VR scenes: one with elements designed to induce high awe, such as a snowy mountain, and a low awe scene without such elements. We compared first-person and third-person perspectives in each scene. Forty-two participants explored the VR scenes, with their physiological responses captured by electrocardiogram (ECG) and face tracking (FT). Subsequently, participants self-reported their experience of awe (AWE-S) and presence (IPQ) within VR. The results revealed that the first-person perspective induced stronger feelings of awe and presence than the third-person perspective. The findings of this study provide useful guidelines for designing VR content that enhances emotional experiences.
Marquardt, Alexander; Lehnort, Marvin; Otsubo, Hiromu; Perusquia-Hernandez, Monica; Steininger, Melissa; Dollack, Felix; Uchiyama, Hideaki; Kiyokawa, Kiyoshi; Kruijff, Ernst
Exploring Gesture Interaction in Underwater Virtual Reality Proceedings Article
In: Proceedings of the 2024 ACM Symposium on Spatial User Interaction, pp. 1-2, 2024.
@inproceedings{marquardtLehnort2024,
title = {Exploring Gesture Interaction in Underwater Virtual Reality},
author = {Alexander Marquardt and Marvin Lehnort and Hiromu Otsubo and Monica Perusquia-Hernandez and Melissa Steininger and Felix Dollack and Hideaki Uchiyama and Kiyoshi Kiyokawa and Ernst Kruijff},
doi = {https://doi.org/10.1145/3677386.3688890},
year = {2024},
date = {2024-10-07},
urldate = {2024-10-07},
booktitle = {Proceedings of the 2024 ACM Symposium on Spatial User Interaction},
pages = {1-2},
abstract = {An underwater virtual reality (UVR) system with gesture-based controls was developed to facilitate navigation and interaction while submerged. The system uses a waterproof head-mounted display and camera-based gesture recognition, originally trained for abovewater conditions, employing three gestures: grab for navigation, pinch for single interactions, and point for continuous interactions. In an experimental study, we tested gesture recognition both above and underwater, and evaluated participant interaction within an immersive underwater scene. Results showed that underwater conditions slightly affected gesture accuracy, but the system maintained high performance. Participants reported a strong sense of presence and found the gestures intuitive while highlighting the need for further refinement to address usability challenges.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
An underwater virtual reality (UVR) system with gesture-based controls was developed to facilitate navigation and interaction while submerged. The system uses a waterproof head-mounted display and camera-based gesture recognition, originally trained for abovewater conditions, employing three gestures: grab for navigation, pinch for single interactions, and point for continuous interactions. In an experimental study, we tested gesture recognition both above and underwater, and evaluated participant interaction within an immersive underwater scene. Results showed that underwater conditions slightly affected gesture accuracy, but the system maintained high performance. Participants reported a strong sense of presence and found the gestures intuitive while highlighting the need for further refinement to address usability challenges.