"Congrats"

I just wanted to say, "congrats". To everyone involved at Xbox, everyone at the game studios. For a really great showing at E3 last week. For all the blood, sweat, and tears that got us to this point. And for helping to bring to market the most sophisticated consumer device ever created.

"Congrats"

Laptops + Pico projectors = Bonfire

Some colleagues of mine at Intel Research/University of Washington recently got permission to post a video of thier research project from UIST 2009. It's a really nice idea that I wish I had done myself and, in my opinon, is one of the better applications of cheap pico projectors that I've seen.

Using pico-projectors and cameras mounted on a laptop, you can take advantage of nearby table space around the laptop to effectively expand your workspace at will. The video is a little stiff but, like many good ideas, you get the jist of it pretty quick.

Guitar Hero without a guitar

This year, I'm helping the User Interface Software and Technology (UIST) conference put their proceedings videos online so that more people can access them. So far, I've gotten most of the videos from this year uploaded to the conference YouTube account. If you've closely followed the tech media coverage, you might recognize projects like the pressure sensitive keyboard and Mouse 2.0.

However, one of my favorite projects this year was a muscle sensing system that (among other things) allows you to play Guitar Hero without a guitar. It directly senses the electrical signals in your arms and maps those to the appropriate button presses. This was done by Scott Saponas, a Phd student at the University of Washington exploring a variety of biometric sensing techniques for input.



I also really like this project which combines a large touch table with other physical input devices such as multiple mice and keyboards all working together nicely. This prototype was done by Bjoern Hartman who has recently joined UC Berkeley's faculty.



I like systems that combine many modes of input so that you can dynamically choose the right device for the job and can gracefully scale to multiple people simultaneously. We are pretty far past having a 1:1 ratio between between people and computers. Yet, most systems today are still designed with 1 device and 1 user in mind.

Pressure Sensitive Keyboard

Some of my colleagues in the Applied Sciences group in Microsoft Hardware have recently gotten some media exposure for one of their recent projects: the pressure sensitive keyboard



Congrats! It's a very nice prototype, and I look forward to seeing what the students at UIST cook up in the innovation contest.

If you aren't familiar with UIST (User Interface Software and Technology), it is a conference dedicated to new interface research. It is one of my favorite conferences to attend. It's where I demonstrated my past work like Automatic Projector Calibration, Foldable Interactive Displays, where Chris Harrison (recently known for Physically Changing Displays) showed off Scratch Input, Andy Wilson showed early pre-Surface touch tables, Hrvoje Benko showed Spherical Surface, MSR-Cambridge presented Second Light, and where Jeff Han first demonstrated FTIR. ...and that's just a small sample from the last 2-3 years.

So if you are interested in new interface technology (or you are part of the tech media) I would encourage you to attend UIST in October this year. You'll get to see what the latest interface researchers are doing all over the world and get to talk to the people behind the ones you have read about.

Rhonda - 3D drawing

It's always good to give people reminders of what is possible when you don't stick with just a mouse and keyboard. This is a very nice piece of interface work for 3D drawing. The system is called Rhonda. The drawing is a bit on the abstract art side, but it's easy to see the level of control he has.



The great thing about 3d drawing is that the current tools are awful, so new ways of doing it are always interesting. Unfortunately, the bad thing about 3d drawing is that there are a relatively small number of people on the planet that really want to do it. So, it's unlikely these interfaces become widespread outside the domain.

Project Natal

If you've been wondering why my project blog has been pretty quiet, I can finally say it is because I have been helping Xbox with Project Natal. If you haven't seen the vision video, it is definitely worth checking out:



Now, I should preface by saying I don't deserve credit for anything that you saw at E3. A large team of very smart, very hard working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard working people involved. The other thing I should say is that I can't really reveal any details that haven't already been made public. Unfortunately.

Speaking as someone who has been working in interface and sensing technology for nearly 10 years, this is an astonishing combination of hardware and software. The few times I’ve been able to show researchers the underlying components, their jaws drop with amazement... and with good reason.

The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera.


But once you have the 3D information, you then have to interpret that cloud of points as "people". This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I've seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world coming together to make this happen.

We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.

Xbox and Microsoft deserve an enormous amount of credit for taking on such an ambitious project. It’s one thing to say “Wouldn’t it be cool if…”, but it’s another thing entirely to say, “let’s dedicate the resources to really make it happen inventing whatever needs to be invented along the way.” I have to say it's pretty neat building the future.

Inspiring robots set to nice music

Since, I'm on a bit of a mini-robot kick at the moment. I thought I would share some of the videos I've seen lately that at least inspired it. The elegance of some of these movements and the music remind me of the child-like imagination that we perhaps once had before becoming more jaded with age. At least, the dreams of a little engineer.







This last one is just fun.