XRoads (Cross Reality On A Digital Surface) explores novel and multimodal interaction techniques for tabletop games. It is a Mixed Reality platform which combines touch, speech, and gestures as input modalities for turn-based and real-time strategy games.
XRoads is built on top of the in-house software framework Simulator X [Latoschik & Tramberend, 2011].
The utilization of Intelligent Realtime Interactive Systems (IRIS) plays a decisive role in the area of virtual reality, augmented reality and mixed (or rather cross) reality. These fields yield assistive technology for numerous everyday situations, offering the user intuitive options to interact with the system. Besides many other application areas, the development of novel games benefits from such interactions.
Funding and Collaboration
2012-2013 a mixed reality version of the role-play-based board game Quest: Zeit der Helden was implemented in the context of student projects: A multitouch table with object recognition capabilities (Samsung’s SUR40) serves as a shared interaction space for tangible game elements (pawns, cards, …). The system augments these real objects with multimedia content, according to the ongoing game. The integration of portable devices allows for private interaction spaces. This prototype unifies different interaction techniques, like touch input or tangible interaction, in order positively affect the user experience [Giebler-Schubert, Zimmerer, Wedler, Fischbach & Latoschik, 2013].
Until now the Quest-prototype has been enhanced with additional multimodal interfaces [Fischbach, Zimmerer, Giebler-Schubert, Latoschik, 2014]:
A dice shaking- and throwing-gesture, detected via the Leap Motion sensor, allows the player to roll a virtual dice that bounces off the real pawns standing on the table.
A simple speech interface provides an alternative to the existing touch interface for all actions a player can choose from at any point during the game.
Recently the prototype has been extended by a location-based adventure played on a mobile-device [Fischbach, Lugrin, Latoschik, Fendt, 2014; Zimmerer, Fischbach, Latoschik, 2014]. The extension devides the gameplay into two phases an adventure phase, where players follow a non-linear storyline by visiting various real-world locations to acquire rewards and information and a skirmish phase, where strategic plans and collaboration is required to defeat enemies in a turn-based battle.
During the skirmish phase, the mobile device is utilized to extend the interaction possibilities with the interactive table:
additional private interaction space, e.g. for playing secret traps
use of device sensors, e.g. shaking the mobile device activates a dice roll
This enhanced versions has been demonstrated at several events where user feedback was continuously collected.
Conceptual and technical implementation of additional (multimodal) interaction techniques, in particular speech and gesture interactions using consumer hardware like Microsoft Kinect, Razor Hydra or DICE+.
Implementation of further existing (tabletop) game mechanics
Exploration of new game mechanics, that are solely feasible in a mixed reality setup
Comparison of alternative interaction techniques (e.g. rolling a via touch vs. gesture vs. DICE+)
Continuative studies that research the difference in the user’s experience between the original tabletop game and an cross reality version.
M.Sc. Martin Fischbach
Telefon: 0931 31 86314