American Sign Language (ASL) Interpreter

Greetings Minions! I am embarking on a mission to develop an ASL interpreter based on the Candescent NUI for the Kinect. I will probably start with the ASL alphabet first. If you are signer please let me know what your obstacles are on a regular basis with people who cannot sign.

XNA Game Studio 4.0 and Graphics Cards

So if you have a beat up old computer like I do and is planning on developing games in XNA Game Studio, you are most likely affected by this error .

After much frustration and hair-loss I figured out a solution. Apparently XNA 4.0 HiDef profile only works on graphics devices that support Direct X 10 and up, I have an ancient Geforce 6800 Go Ultra.

So the work around; click on your project's properties window and select the "Reach" profile. How simple is that?



Voila!

Creating your first XNA Game

So say you want to create your own game and you're looking for a good place to start, you've come to the right place. I am going to try to break this down to the best of my knowledge assuming that you have a little comprehension of the C# programming language. To get started you need to download and install the Windows Phone SDK, then download and install the Windows Phone Developer Tools January 2011 Update and the Windows Phone Developer Tools Fix. (Note: According to Microsoft's website the January 2011 Update should only be applied after you've installed the Windows Phone Developer Tools.)

Once you've got Visual Studio installed and running click on File menu > New Project. From the list on the left
Give it a name and a location and Visual Studio should create a template for you. As it is you have a working game, although it may not do much. To run your game just hit F5 on the keyboard or click on Debug>Start Debugging. You should see a blue screen much like this:


Congratulations! You've got your first game running.

Talking to your Arduino via Bluetooth!

If you've been working with Arduino long enough, you might have come to the conclusion that for the average hacker these are indispensable. Most often these micro-controllers don't offer the flexibility or mobility for certain projects. In my attempt to minimize size and permit mobility, I have moved on to a Arduino Pro Mini (5V) instead of my most faithful Arduino Diecimila which talks to the computer via BlueSmirf (a bluetooth modem).

I shall walk you through the steps I went through just to save you the trouble. I am assuming you have an Arduino Diecimila or NG with you because the Arduino Pro Mini does not come with any connections and your old board can be used to connect your Pro Mini board to the computer. I am also assuming you already have a bluetooth dongle or computer with a bluetooth module embedded.

For this tutorial, you will need:

1. An Arduino Pro Mini (5V or 3.3V, I will be using the 5V one)
   
2. A BlueSmirf Gold Bluetooth Modem
3. An Arduino Diecimila
   

First we need to pair the devices, i.e. your computer and the BlueSmirf modem.
Make the following connections.

BlueSmirf	Arduino
VCC	5V
GND	Ground
TX-0	RX-0
RX-1	TX-1

Set the jumper on your Arduino to receive power from your power adapter, this is because you cannot attach the BlueSmirf directly to a serial port so we avoid using the USB cable. Once you power it up, a red LED starts blinking. Search for bluetooth devices in the vicinity and you should see the BlueSmirf pop up. Mine was listed as FifeFly-ABCD with ABCD being the last four digits of the MAC Address. I did not need authentication to pair it up but the default passkey is 1234. Your new connection should pop up as a Serial Port Profile which basically create a COM port on your computer. You will be able to talk to your Arduino via serial much like you did when you had connected it via usb.

Now that you've completed pairing your device, you can disconnect your bluetooth modem and set the jumper setting back to USB on the Arduino board. To get your Arduino Pro Mini going, you will need to take off the AVR chip from your Arduino Diecimila or NG. Then make the following connections.

Diecimila	Mini
5V	VCC
GND	GND
TX-0	TX-0
RX-1	RX-1

I had trouble uploading sketches to my Arduino Pro Mini and one solution I found out that worked was holding down the reset button on the board right before uploading.

I uploaded the following code to my Arduino Pro Mini

char val;         // variable to receive data from the serial port int ledpin = 13;  // on-board LED

void setup()
{   pinMode(ledpin, OUTPUT);  // pin 48 (on-board LED) as OUTPUT
  Serial1.begin(115200);          // start serial communication at 115200bps
}
void loop()
{
  if( Serial1.available() )         // if data is available to read
  {  
     val = Serial1.read();          // read it and store it in 'val'
  }
  if( val == 'H' )                     // if 'H' was received
  {
     digitalWrite(ledpin, HIGH);  // turn ON the LED
  }
  if( val == 'L' )
  {
     digitalWrite(ledpin, LOW);   // otherwise turn it OFF
  }

  delay(100);                         // wait 100ms for next reading
}
The following processing code was from Mitchell Page's blog

//import class to set up serial connection with wiring board
import processing.serial.*; 
Serial port;    //button setup
color currentcolor;
RectButton rect1, rect2;
boolean locked = false;  

void setup() {  
//set up window  
size(200, 200);  
color baseColor = color(102, 102, 102);  
currentcolor = baseColor;     

// List all the available serial ports in the output pane.   
// You will need to choose the port that the Wiring board is   
// connected to from this list. The first port in the list is   
// port #0 and the third port in the list is port #2.   
println(Serial.list());      

// Open the port that the Wiring board is connected to (in this case 1  
// which is the second open port in the array)   
// Make sure to open the port at the same speed Wiring is using (115200bps)   
port = new Serial(this, Serial.list()[10], 115200);     

// Define and create rectangle button #1  
int x = 30;  
int y = 100;  
int size = 50;  
color buttoncolor = color(153, 102, 102);  
color highlight = color(102, 51, 51);   
rect1 = new RectButton(x, y, size, buttoncolor, highlight);     

// Define and create rectangle button #2  
x = 90;  
y = 100;   
size = 50;  
buttoncolor = color(153, 153, 153);  
highlight = color(102, 102, 102);   
rect2 = new RectButton(x, y, size, buttoncolor, highlight);

}  

void draw() {  
background(currentcolor);  
stroke(255);  
update(mouseX, mouseY);  
rect1.display();  
rect2.display();
}  

void update(int x, int y) {  
if(locked == false) {       
rect1.update();    
rect2.update();  
} else
{    
locked = false;  
}     

//Turn LED on and off if buttons pressed where  
//H = on (high) and L = off (low)  
if(mousePressed) {    
if(rect1.pressed()) {            //ON button      
currentcolor = rect1.basecolor;      
port.write('H');     }
else if(rect2.pressed()) {    //OFF button      
currentcolor = rect2.basecolor;      
port.write('L');     }   } }  

class Button {  
int x, y;  
int size;  
color basecolor, highlightcolor;  
color currentcolor;  
boolean over = false;  
boolean pressed = false;        
void update() {    
if(over()) {      
currentcolor = highlightcolor;    
}
else {currentcolor = basecolor;}}
boolean pressed() {    
if(over) {
locked = true;      
return true;    
}
else {      
locked = false;      
return false;}}

boolean over() {
return true;   
}     

void display() {}

}

class RectButton extends Button
{RectButton(int ix, int iy, int isize, color icolor, color ihighlight)
{
x = ix;
y = iy;
size = isize;    
basecolor = icolor;    
highlightcolor = ihighlight;    
currentcolor = basecolor;  
}     
boolean over()
{    
if( overRect(x, y, size, size)
) {      
over = true;      
return true;    
} else {      
over = false;      
return false;
 }   }     
void display() {    
stroke(255);    
fill(currentcolor);    
rect(x, y, size, size);   } } 
boolean overRect(int x, int y, int width, int height)
{   if (mouseX >= x && mouseX <= x+width &&       
mouseY >= y && mouseY <= y+height) {    
return true;  
} else {    
return false;   } } 

Multiple points of reference for tracking

Beta testing revealed that the use of one IR LED for tracking served the purpose but did not exactly provide enough reference points to simulate shapes efficiently. To solve this issue two more LEDs were added to the front of the glove (see pictures below).

Three LEDs as opposed to one lets the prototype capture left and right limits to the user's hand and a center reference point.

The motors trigger upon hitting obstacles as before, but the intensity increases the deeper the user is within an obstacle

Prototype V3 Test

Feedback intensity and patterns changed, 3 LEDs instead of 1 used to reference points.

Degrees of freedom for tilt, Prototype V3

The following video illustrates the freedom of tilt the glove permits the user.