Daily Archives: May 10, 2012

Danse Neurale: NeuroSky + Kinect + OpenFrameworks

Posted on by

This performances makes use of the NeuroSky EEG sensor as well as the Kinect. Visuals and music are driven by EEG and registered with the performers body using the Kinect. It seems their system runs under OpenFrameworks. In fact, I noticed this video in the OF gallery. The second half of the video consists of an interview with the technical team and performer.

This performance uses off-the-shelf technology but is cutting edge in more than one sense. No one can accuse these guys of lacking commitment.

A project page may be found here: Danse Neurale.

They generously list the code used to acquire signals from the NeuroSky server in the OF forum. This part of the system is written in P5 (Processing).

Here are a few details on the technical background of the work, given by one of the creators in the OF forum:

Sensors:

– breath: it’s sensed with a wireless mic positioned inside Lukas’ mask. its signal goes directly through a mixer controlled by the audio workstation

– heart: it’s sensed with modified stethoscope connected with a wireless mic; signal works just like the breath (we’re not sure, but in the future we may decide to apply some DSP on it)

– EEG: we use the cheaper sensor from NeuroSky; it streams brainwaves (already splitted into frequencies) via radio in a serial like protocol; these radio packets arrive to my computer where they’re parsed, conveted into OSC and broadcasted via wifi (we only have 2 computers on stage, but the idea is that if we have an affine hacker soul between the public, he/she can join the jam session 🙂 )

– skeleton tracking: it’s obviously done with ofxOpenNI (as you can see in the video we also stage the infamous “calibration pose”, because we wanted to let people understand as much as possible what was going on)

The audio part maps the brainwave data onto volumes and scales, while the visual part uses spikes (originated i.e. by the piercings and by the winch pulling on the hooks) to trigger events; so, conceptually speaking, the wings are a correct representation of Lukas’s neural response and they really lift him off the ground.

Important: New URL

Posted on by

Important: I have changed the url of this blog to http://zemi.kasrl.org.

Only a small percentage of the members of the zemi are doing work in the field of what I would consider to be “NIME” (= New Interfaces for Multimodal Expression).

Please note however, that the zemi is in fact in the “Technology” field of the Eizo Faculty.

In other words, members are expected to undertake challenging research related to image arts and sciences technology, with an emphasis on real-time, interactive, expressive multimodal interfaces.

My main area of interest remains NIME (= New Interfaces for Multimodal Expression). That is my major area of expertise and the area in which I am most interested in supervising projects. Work in this area requires knowledge of computer programming, or at least a strong will and effort to learn how to program. This is stated clearly in the official description of my seminar. For whatever reason, it seems that some students have the impression they can join this seminar without the ability or will to learn how to program. That is incorrect.

Thank you for your understanding.

Michael