"He was like..."

While certainly not the technology focused topic I usually post, I definitely wasted a few minutes trying to sing the following chorus in the true spirit of procrastination. Try to sing along (if you can). If you are linguistically incapable, just reading along is amusing enough.

"he was like
she was all
he was all
they were like
we were all,

like oh my god
like totally

we were like
that was all
they were all
he was like
she was like

all totally
like oh my god"



If this was not educational enough for you, the following "anthropological introduction to YouTube" has a boring title, but is an incredibly fascinating and entertaining discussion of the cultural and social phenomena within the depths of YouTube... and relevant to the video above. Like totally. (warning: 1 hour talk, but definitely one of the better uses of 1 hour in my life).

Nice Pen-based Input Research

One of the things I enjoy to using this blog for is to share cool projects from Human-Computer Interaction (HCI) research. This post highlights projects by Gonzalo Ramos (or "Gonzo" for short) and his co-authors. He has worked on several projects demonstrating how much better pen input software could be. These are just a few I like.

1. The Zlider - A pressure sensitive slider widget that adds additional navigation and control capability to standard slider interactions. Academic research video below. Quick demo montage at beginning, but the demo meat is at 3:07


2. Using a Pen to Effortlessly Bridge Displays. Using a stylus, you can simply drag documents between computer screens or mobile devices. The pen motion also implicity defines the orientation of the displays relative to one another. Academic video below. Demos at the beginning and more mobile screen scenarios at around 2:43


3. Rolling the Pen as Input Using an external tracker and a Wacom tablet, rotating the pen in your fingers can be used to control another parameter without moving the stylus. Academic video below, demo meat at 2:19


You can check more of his projects on his website.

Some great Wiimote IR tracking projects

I've decided to collect some of my favorite projects I've seen people do with my Wiimote projects, derivatives of them, or distantly inspired (through the creator's own admissions). It's a surprise, and flattering to see how many people seem happy to credit me. Thanks all! The list gets more "unusual" the further you go down.

Two Wiimote Whiteboards to make a competative relay race:


Great IR wands for the Wiimote whiteboard. I've been meaning to make these, but I haven't gotten to it yet.


Some nice two handed, two finger pinching systems:




Wiimote Wheelchair art. Unfortunately, no video but more information at this link.


Head tracking prototypes with Anime assets. The effect of the girl coming out of the screen (about half way through the video) is very nicely done with the "haze" layer. His other videos are also worth checking out. I don't know what he does for a living, but he's good at it.


Wii Theremin gallantly created/performed by Ken Moore:


Finally, a video on "chicken head tracking". It doesn't use the Wii remote, but was posted as a response to my video and I love it!

Scratch Input and Low-Cost Multi-spectral material sensor

Chris Harrison, a PhD Student at my old program at CMU, presented a couple projects of his at UIST 2008 that I really really like. The first is his "Scratch Input" device. The basic idea is that if you place a senstive microphone on the bottom of a mobile device. Any large, hard surface you put it down on can now be used as an input gesture surface. A variety of gestures can be distinctly and reliably detected with some simple machine learning. Video (academic) below include a nice demo where he turns his entire wall into an MP3 player controller:



The other project he presented was a simple, cheap multi-spectral sensor for recognizing various materials. It includes an IR LED, UV LED, RGB LED, a photoresistor, and a TSL230 TOAS optical sensor. With these, he read the reflectively under different illuminations to recognize 27 different materials with 86.9% accuracy, be this your jeans, your backpack, your desk at home, your desk at work. This means coarse location awareness of mobile devices for cheap, some opportunities for more intelligent power management, and implicit security behaviors when placed on familiar or unfamiliar surfaces. Very nice work.

SurfaceWare - sensing glasses for Surface

My colleague, Paul Dietz, in the Applied Sciences group released a video of one of his first projects he did when he joined Microsoft. These glasses use the transparent material of the glass as prisms that sense the amount of liquid in them by watching the amount of internally reflected IR light. Check out the video:



If you aren't familiar with how Surface works, it is a rear projected table that also has a bright IR emitter inside that illuminates objects placed on the surface which are then visible to an IR camera. The video does a good job explaining how the glasses work.

This is actually a revisit of an older project of Paul's called iGlassware. That one used passively powered RFID sensor tags in the base of the glass to capacitively measure the liquid level. The table had a big RFID antenna in it. Paul was also a key developer of Mitsubishi Electric Research Lab's Diamond Touch table being skillfully demonstrated by Ed Tse below.



Ed is currently at Smart Technologies, who helped push out their new touch table:

Andy Wilson

I was re-watching some videos of work done by one my colleagues Andy Wilson, and I don't think his work gets as much attention as it deserves given how amazing it is. If you think my stuff is cool, you should bow down to his greatness... or at least watch these videos.

Working with the PixArt camera directly

This has been a pretty whirlwind past few months. Lots of things have happened, almost none of which procrastineering related which is why I haven't posted anything here. But, one of the things that I have poked at in the past few weeks was creating a PixArt to USB-HID device which allows the camera from the Wiimote to appear as a relatively easy to access USB device. This addresses several problems with using the Wiimote such as running off batteries for extended periods and flakey platform specific Bluetooth drivers. It's also possible to read from the Pixart cam at over 100Hz if you read directly via I2C as well as track visible dots once you remove the IR filter. Of course, none of this was discovered by me. All credit belongs to the numerous individuals who have contributed thier knowledge to the various Wiimote hacking websites. Normally, this project wouldn't be worth a post, but all the information on how to do this is pretty scattered and difficult to follow. So, I figured I would contribute by trying to making this all a bit clearer.

This project is fairly advanced. You must be comfortable with working with microcontrollers. Several simpler devices such as the Arduino or the Basic Stamp may work, but I used the 18F4550 PIC Microcontroller which provides built-in full-speed USB capabilites. But first, let talk about the PixArt camera:

Here's the pinout thanks to kako and a PCB picture. The Reset pin is active low, so use a pullup resistor to Vcc. The Wiimote runs the camera with a 25Mhz clock, but it also works with a 20Mhz clock so you might get away with fudging this a bit. The I2C communication is fast 400Khz and the slave device address is 0xB0. Most microcontroller development platforms should include I2C communication capabilities. If yours doesn't, get a better dev kit =o). Desoldering the camera can be hard with so many pins. But, careful use of a hot air gun will do the trick. The first part is to initialize the camera over I2C. Here's the pseudo code for initializing to maximum sensitivity (actual CCS C code in comments):

1. write(hex): B0 30 01
2. wait 100ms
3. write(hex): B0 00 00 00 00 00 00 00 90 //sensitivity part 1
4. wait 100ms
5. write (hex): B0 07 00 41 //sensitivity part 2
6. wait 100ms
7. write(hex): B0 1A 40 00 //sensitivity part 3
8. wait 100ms
9. write(hex): B0 33 03 //sets the mode
10. wait 100ms
11. write(hex): B0 30 08
12. wait 100ms

It's still somewhat mysterious to me what all these mean, but in this mess is the sensitivity and mode settings described at Wiibrew. The above code uses the sensitivity setting suggested by inio "00 00 00 00 00 00 90 00 41, 40 00" experssed in the 2nd, 3rd, and 4th message. The wait times are conservatively long. After you initialize, you can now read samples from it:

1. write(hex): B0 37 //prepare for reading
2. wait 25us
3. write(hex): B1 //read request
4. read 8 bytes
5. wait 380us
6. write(hex): B1 //read request
7. read 4 bytes

This yeilds one sample from the camera containing 12 bytes, 3 for each of the 4 potential points. The format of the data will be the Extended Mode (X,Y, Y 2-msb, X 2-msb, Size 4-bits). The wait timings approximate what the Wiimote does. I've called this routine 1000 times per second without ill effect. Though, I doubt this is actually scanning the sensor and instead is just reporting the contents of an interal buffer. But, people claim 200Hz updates are possible. So, you can use that as a suggestion.

Hooking this up to your microcontroller is pretty straight forward. Give the camera 3.3v power using a voltage regulator, ground, a 20-25Mhz clock, and connect the SDA and SCL lines (don't forget your pull up resistors), and pull up the reset pin.

The CCS C Compiler for the PIC18F4550 includes USB-HID sample code. It's simply a matter of stuffing the data you got from the PixArt camera into the input report buffers for the USB. With this, you could actually create a USB mouse profile and make it control the cursor without any software or drivers at all. If set it up as a full speed device, it's possible to get 1ms reports providing extremely low latency updates. CCS provides relatively affordable PIC programmers as well. Explaining how to set all this up is not within the scope of this post, but it should be plenty to get you started. If you want to make a PCB, you can try ExpressPCB which can get you boards in-hand for as low as $60.

Update 9/6/08: Just a note about the clock. Since my PIC was using a 20Mhz resonator, I just piggy backed the Pixart clock pin off the OSC2/CLKO pin of the PIC which seemed to work fine. Also, Kako has more details (in Japanese) on doing this with an Arduino