VIRTOSHA

About the VIRTOSHA Project

Authoring software will enable the creation of various scenarios, where tissue and bone properties can be parameterized to simulate different patients. By analyzing real operations on patients and specimens, the steps and options of the procedure are captured, including the behavior of tissue, implants, and tools, as well as the haptic feedback. Using the authoring tool, different processes and avatars can be linked together to create new training sequences. Correct simulation of the tissues and the precise integration of hand movements are crucial for immersion. The Krüger group’s main focus is on improving hand tracking, based on VR tracking information, sensor readings, and potentially cameras as additional data sources. This effort aims to enhance the accuracy and responsiveness of hand motion capture within the virtual environment, thereby increasing the realism and effectiveness of the training. The haptic interaction with tools is based on a drilling and screwing simulation to be developed, which will be implemented with haptic arms and allow for realistic manipulations. Further features for an educational experience include the representation of tissue characteristics during manual and tool-guided interaction, as well as realistic rendering. All components will be connected via a data hub that manages all processes and structures and serves as the system’s interface. The entire training environment is to be developed at the UKB (University Hospital Bonn) to create a meaningful teaching tool in close collaboration with medical users.