Human-Computer Interaction

Simulator X

Simulator X is a research testbed for novel software techniques and architectures for Real-Time Interactive Systems in VR, AR, MR, and computer games. It uses the central concept of semantic reflection based on a highly concurrent actor model to build intelligent multimodal graphics systems.


Simulator is developed using the Scala programming language and runs on Java Virtual Machines (JVMs). Hence, it is platform independent and can be used under Windows, Linux as well as MacOS X. Developers can easily reuse Java libraries or write Java Native Access wrappers for native shared libraries. The Scala language allows to program as well following the object-oriented as the functional programming paradigm. The first one is well-understood by most software developers and provides a smooth transition from the OOP paradigm. The functional paradigm allows for concise program code and thus reduces development time for advanced developers.

It comes with modules for OpenGL rendering, OpenAL sound rendering (via LWJGL), JBullet physics and VRPN support. In addition, a tool for interactively inspecting the current application state as well as basic examples are included, providing developers a smooth start.

The Simulator X project has been funded from 2009 to 2012 by the BMBF in the program IngenieurNachwuchs.

Concepts and Architecture

Simulator X’s highly modular, component-based architecture builds upon the ACTOR model by Hewitt. This eases the development of highly concurrent applications without the need for implementing synchronization mechanisms. Currently, Akka’s Actor library is utilized by Simulator X.

On top of the ACTOR model the concept of State Variables is developed. The message-based access to a state variable is mediated by an actor (its owner). An owner stores the current value of the state variable, whereas other actors keep references to the state variable and can request the current value, initiate updates, and request update-notifications. This concept (which is visualized in figure 1) allows for concurrent access without the need for application of synchronization mechanisms. Using such state variables, a (virtual) shared state is created.

Figure 1: State Variable concept

To provide a more comprehensible view on the current world state, entities that are composed of aforementioned state variables are used. Entities are instantiated according to Entity Descriptions which are composed of aspects. Aspects describe the functionality of an Entity.

Simulation modules are represented by components which can consist of one or more actors. Each component can provide its own aspects (e.g., a physics engine would provide a physical aspect). Adding such aspect to an entity-description makes the component aware of each entity instantiated through this description. The component can then include the entity in its simulation loop. This approach, in combination with symbol based access and automatic type conversion between components furthers reusability and extensibility of Simulator X applications. These concepts are visualized in Figure 2.

Figure 2: Simulator X concepts

Entity as well as component properties and features are represented in an ontology to provide a knowledge-based access based on a semantic reflection paradigm. This allows for automated code transformation, extended type and feature checking, as well as code highlighting.

Get Simulator X

Simulator X is available on GitHub. Clone the simulator-x repository and all its submodules (i.e. in the command line add the –recurisve option or use the respective option of your graphical git client).

It currently requires a Java JDK 1.6 or higher. The project directory contains settings for an IntelliJ IDEA project. To set it up, it requires plugins for scala and the simple build tool (sbt) which can be installed using IDEA’s integrated plugin manager. All dependencies will be downloaded on first build using sbt’s integrated maven support.

For further instructions see the getting started document and the doc folder in the repository.


The clips found below demonstrate some of Simulator X’s features and show demo applications, which were developed using that framework.

General Simulator X Concepts

Selected course results

Prototypes made with Simulator X that were developed during pratical exercises of master level lectures.


Big Bang - Unreal4 Integration and Multimodal Interaction in VR
XRoads - more information can be found at the XRoads project page
MagiCol Window - a collaborative mixed reality (MR) setup
smARTbox- a multi-touch enabled surface with stereoscopic rendering

Techical Demonstration

Can Knockdown - basic VR requirements validation

Fishtank - an actor benchmark

Technologies used in Simulator X:

Simulator X Core:

Modules wrapping third party software:

Additional modules:

Funding and Collaboration




Martin Fischbach

Legal Information