12/14/2009

PComp Final – OpenEEG BCI

For my final Physical Computing project I am currently constructing an EEG machine based on the design of the OpenEEG project at sourceforge. The idea is to use signals that are known to cause predictable spikes to trigger some action, what exactly that is still needs to be determined.

For a general overview of EEGs consider the Wikipedia entries, they give a (rather enthusiastic) overview of the device.

1. EEG – Overview

So far I've ordered the openEEG boards from Olimex and made active electrodes. However, it seems as if one channel (2 electrodes) isn't working properly, which may due to the whole soldering and desoldering they went through. A great guide on how to build your own active electrodes can be found here:http://www.dcc.uchile.cl/~peortega/ae/



Here is a flic of the whole project at work, so far


In the long run, I think I would like to go for something ambitious. Next semester I will be taking Learning bit by bit and will hopefully learn about writing classification algorithms (regressive ones in particular because they facilitate one-dimensional classification) using varies EEG potentials (slow sensory motor cortex activity, P300 response, N400 & P600 respectively).
However, more work is still to come and to be done.

11/12/2009

#_9 Kitchengame




Kitchengame was a game that started from the idea to have people interact with live music. MinWoo had outlined a project earlier that aimed at visualizing live music in public spaces via user (audience) interaction (his blog for more info).
As we were figuring out how we would imagine the audience to interact directly with a musical performance, we thought of laser-tripwires that would could capture the crowd dynamic in a visual way. After we went through some scenarios, we changed our initial plan and started to think of a game scenario we could use this basic principal for. So, our idea was to use photocells (photo resistors) and lasers to trigger sound instead of visuals. Also, instead of passively registering the audience's interaction we used laserpens as "guns" that would have to be shot at the photocells, thereby enabling direct interaction with the interface.

The basic game setup so far: a one way interaction with two teams trying to shoot photocells with laserpointers and thereby triggerin sounds. We still wanted to include some real world musical element so we decided to add another player to each team that would have to play some instrument, which would 'activate' the photocells (i.e. make them read values) and control a visual output. This as outline for our mediacontroller project, we started to organize materials and test some small setups:
10 laserpens (eBay 10$)
12 3,5" Photocells (electronic goldmine ~20$)
2 Arduinos (wherever ~60$)
(1 Arduino and multiplexer also possible)
cardboard (free – 18th Street Fellowship)


After we had tried various game scenarios, we decided to set up two Arduinos, one [1] reading six photocells and the other[2] reading values from piezzo sensors that were attached to the physical instruments (cardboardboxdrum). The analog values read from the piezzos on [2] were linked to a Max Patch and whenever they would receive readings above a threshold (I think 50 for our setup) a digital signal was communicated to [1], which would then start reading from the three photo cells matching the piezzo source signal to the appropriate team (if drummer of team A drums, field players of team A can score).
Here is a video MinWoo made of the game being played at the presentation (thanks to the fierce players: Dawn, Cody, and Sarah Jenny on the one side, Chika, Leo, and Rory on the other side):

The critique afterwards was pretty cool because many detailed comments where made that showed that the basic concept worked but that many details weren't really fleshed out. The most important one, I thought, was that we should have introduced a scoreboard. Also the connection to the MaxPatch could've been clearer or at least been projected. All in all, I am happy with the outcome as I never designed a game before (especially no physical one).


Here are the codes we used for the photo cell sound communciation.
***********************************************
***********************************************
Arduino
*********************************************** ***********************************************
int analogOneA = 0; int analogTwoA = 1; int analogThreeA = 2;  // analog input
int analogOne = 3; int analogTwo = 4; int analogThree = 5;

int digitalOne=12,digitalTwo=13;//digitalPins to communicate with the other Arduino

int sensorValue = 0;  // reading from the sensor

void setup() {
  // configure the serial connection:
  Serial.begin(9600);
  // configure the digital input:
  pinMode(digitalOne, INPUT);
   pinMode(digitalTwo, INPUT);
}

//we were having trouble with the for-loop
//which is why we call from each pin with full declarations of evey pin


void loop() {
 //if signal arrives from the piezo buzzer
 if(digitalRead(digitalOne)==1)
 {

  // read the sensor:


  sensorValue = analogRead(analogOne);
  // print the results:
 // Serial.print("analogOne-");
  Serial.print(sensorValue, DEC);
  Serial.print(",");
 
  // read the sensor:
  sensorValue = analogRead(analogTwo);
  // print the results:
 // Serial.print("analogTwo-");
  Serial.print(sensorValue, DEC);
  Serial.print(",");
 
  // read the sensor:
  sensorValue = analogRead(analogThree);
  // print the results:
  //  Serial.print("analogThree-");
    Serial.println(sensorValue, DEC);
 }

if(digitalRead(digitalTwo)==1)

{
sensorValue = analogRead(analogOne);
  // print the results:
 // Serial.print("analogOne-");
  Serial.print(sensorValue, DEC);
  Serial.print(",");
 
  // read the sensor:
  sensorValue = analogRead(analogTwo);
  // print the results:
 // Serial.print("analogTwo-");
  Serial.print(sensorValue, DEC);
  Serial.print(",");
 
  // read the sensor:
  sensorValue = analogRead(analogThree);
  // print the results:
  //  Serial.print("analogThree-");
    Serial.println(sensorValue, DEC);
}
 
}




***********************************************
***********************************************
Processing code
*********************************************** 


import processing.serial.*;

import ddf.minim.*;
import ddf.minim.signals.*;


Serial myPort;

int counterA=0, counterB=0;
// Init objects
Minim minim;

AudioPlayer win;

AudioPlayer oneA;
AudioPlayer twoA;
AudioPlayer threeA;

AudioPlayer oneB;
AudioPlayer twoB;
AudioPlayer threeB;

void setup() {




  myPort = new Serial(this, Serial.list()[0], 9600);

  minim = new Minim(this);

  win = minim.loadFile("winner.mp3");

  // first team
  oneA = minim.loadFile("1.wav");
  twoA = minim.loadFile("2.wav");
  threeA = minim.loadFile("3.wav");
  //second team
  oneB = minim.loadFile("4.wav");
  twoB = minim.loadFile("5.wav");
  threeB = minim.loadFile("6.wav");
}

void draw() {

  // First Photocell
  if (oneA.isPlaying()) {
    //counterA +=1;
  }
  else {
    oneA.rewind();


  }

  //Second Photocell
  if (twoA.isPlaying())
  {
    //  counterA+=1;
  }
  else {
    twoA.rewind();

  }
  //third Photocell
  if (threeA.isPlaying()) {
    //counterA+=1;
  }
  else {
    threeA.rewind();

  }

  //AND THE WINNER IS
  if (counterA>10)
  {
    //win.play();
  }

  // First Photocell
  if (oneB.isPlaying()) {
    //counterB +=1;
  }
  else {
    oneB.rewind();


  }

  //Second Photocell
  if (twoB.isPlaying())
  {
    //  counterB+=1;
  }
  else {
    twoB.rewind();

  }
  //third Photocell
  if (threeA.isPlaying()) {
    //counterB+=1;
  }
  else {
    threeB.rewind();

  }

  //AND THE WINNER IS
  if (counterB>10)
  {
    //win.play();
  }



}


void serialEvent(Serial myPort) {

  String myString = myPort.readStringUntil('\n');
  println(myString);

  if (myString != null) {
    myString = trim(myString);
    int sensors[] = int(split(myString, ","));

    for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) {
      print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t");
    }
    println();

    //first team
    if (sensors[0] > 200) {
      oneA.play();
    }


    if (sensors[1] > 200) {
      twoA.play();
    }

    if (sensors[2] > 200) {
      threeA.play();
    }

    // second team
    if (sensors[3] > 200) {
      oneB.play();
    }


    if (sensors[4] > 200) {
      twoB.play();
    }

    if (sensors[5] > 200) {
      threeB.play();
    }
  }
}

void stop()
{
  win.close();
  oneA.close();
  twoA.close();
  threeA.close();
  minim.stop();

  super.stop();
}

11/11/2009

#_X Electrodes and OpAmp


Yesterday I finally got started with (basically) what I came for to ITP; developing and creating BCIs (Brain Computer Interface). 2nd year Mustafa has been working on it all semester long for his course "Rest of you" taught by Dan O'Sullivan so we decided to join forces on this one.

So far, I managed to hook up electrodes to a simple Op-Amp differential circuit and get some reading of body activity. However, the results so far are not closely something that could match any medical machines, which is mainly due to the bad amplification a single Op-Amp offers and also the rather arbitrary application of the electrodes to my body – not even close to 10-20 or any other systematic way. However, that was the first step and I am happy to keep going with this stuff.