If you've been wondering why my project blog has been pretty quiet, I can finally say it is because I have been helping Xbox with Project Natal. If you haven't seen the vision video, it is definitely worth checking out:
Now, I should preface by saying I don't deserve credit for anything that you saw at E3. A large team of very smart, very hard working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard working people involved. The other thing I should say is that I can't really reveal any details that haven't already been made public. Unfortunately.
Speaking as someone who has been working in interface and sensing technology for nearly 10 years, this is an astonishing combination of hardware and software. The few times I’ve been able to show researchers the underlying components, their jaws drop with amazement... and with good reason.
The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera.
But once you have the 3D information, you then have to interpret that cloud of points as "people". This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I've seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world coming together to make this happen.
We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.
Xbox and Microsoft deserve an enormous amount of credit for taking on such an ambitious project. It’s one thing to say “Wouldn’t it be cool if…”, but it’s another thing entirely to say, “let’s dedicate the resources to really make it happen inventing whatever needs to be invented along the way.” I have to say it's pretty neat building the future.
Monday, June 1, 2009
Project Natal
Subscribe to:
Post Comments (Atom)
Posts
287 comments:
1 – 200 of 287 Newer› Newest»and the presumed price is...?
Excellent work; when is the expected release date?
what's the chance of seeing 3d head tracking as done with the wiimote?
Simon, Lionhead have already incorporated the 3D head tracking into their Milo demo and James Cameron announced that Avatar is the first major fully 3D game.
Cool in a lot of ways, but I'm always suspicious of the big corps. vs academia. I mean, we're potentially talking a revolutionary set of techs for the advancement of useful functional AI and robots, and it's being used as a proprietary marketing tool for a games machine in order to attempt to compete with another games machine.
I'm sure it connects via USB and I'm wondering if microsoft is going to limit functionality to games on the Xbox, or more positively open it up to open PC/hobbyist/Academic developement with an available SDK containing all those algorithms. I'm guessing the former. Imagine that one day we create computer sentience and a new life form is copyright Microsoft. Still, I'd rather it comes from MS than DARPA. I think.
On another note, on the one hand I'm impressed they've made an affordable time of flight camera that captures 3d. On the other hand I'm depressed they've made an affordable time of flight camera that captures 3d. Why? Because a whole bunch of smart people are now going to ask what the point is anymore in making better stereoscopic vision systems.
wow astonishing indeed !
> The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision research.
Have any papers been published by the Natal team? Do they plan to publish any of it?
Johnny, that's awesome. I love your blog and projects, and I got a little vicarious thrill for you when I saw that you were working on something so (seemingly) perfect for you. I hope it's a ton of fun.
I could play Devil's Advocate here: The video and demonstrations during the press conference reminded me of the Wii's early concept footage and presser demos.
That being said, the Wii didn't quite deliver the total immersive experience of swinging a sword like Link and ducking behind your couch cause you in such a great shooter.
The technology looks very robust but in the end, it'll depend on software support. Will this be the next best thing in gaming or will it be buried in mini game compilations?
Oh, and thanks for the technical insights too. You guys are doing something revolutionary.
So, What is the meaning of the name Natal?
Tadeu: exactly what you think, it's because of the city in Brazil (plus its meaning in Latin).
Johnny! I was the skateboard kid from the E3 Natal video. I knew you were working on it!!
When I auditioned for the job they mentioned "Minority Report" which to me immediately got me ecstatic because I already made my own "Minority Report" gloves using IR and your method.
Anyway, if you aren't super busy, drop me an email or something. You are one of my greatest inspirations.
~Ka'i
kaikau@gmail.com
Wow. This is truly amazing--I have followed the various DARPA challenges for years, always wondering how long it would take for someone to finally develop the technology that can efficiently do what the human visual system does. I studied neuroscience alongside engineering, and I always marveled at how capable our visual system is at object detection and identification. What you guys are doing is nothing short of revolutionary. Here's to hoping that y'all will have the opportunity to develop these systems further. I'm just imagining all of the dangerous places we could take people out of and replace them with expensive (but replaceable) machines. Bravo, sir!
(Oh, and the gaming possibilities are absolutely /mind-bendingly/ awesome. Again, Bravo!)
Wow.
Can't help but wonder... will we be able to use the Natal hardware for our own projects?
(like we've done with the wiimotes?)
That would be really great...
Check the image, it looks like a stereoscopic camera system to me:
http://www.keralanext.com/news/images/34.games_e3_microsoft.jpg
Nice. Now I know this project will succeed. Good luck and bring us some nice inventions :)
So Nintendo missed their chance with you. Natal seems like a great fit for your skills. Looks incredible!
Until I saw this blog post and your crediting for natal I immediately changed my impression of Natal from "yeah its gonna be cool" to "OK this is a technology I MUST SPEND TIME LEARNING".
I'm sure the natal team deserve alot of credit, and they will get it in time. BUT you've just given this beast legs my friend by lending your name to it.
NICE and look forward to building things with natal
Jose (advertboy.wordpress.com)
wow.. incredible!
This is the interface of the future (now!). I can imagine this kind of interface in all of hardware that need an interaction with us.. mobile phones, computers, cars?, houses?
I try the wii-pc but I don't like the result.. I can't wait to test Natal :)
Thank you for your work!!
Hey,
I don't know if this video was just a promotional one, showing only ideal scenarios, but still, is one of the most impressive things I ve seen last years. I am also in the Computer Vision field and I can really REALLY understand what a difficult task this is.
However, I have two concerns.
1) How robust the system is and
2) as far as the technologies are concerned, are they generically applicable or you assume pretty much "static" objects, confined in a specific space?
Anywayz,
just unbelievable ...
Stratis
Thank you soo much for experimenting with this stuff and actually getting it into a product, very inspiring! - can't wait for a sdk or a reverse engineering of it! :)
Dare we hope for a Dance Dance Revolution game where the dreaded pad errors are a thing of the past? Or will we just be substituting lag and cam errors? :)
This product makes me laugh! While I very much appreciate the programming and hardware that goes into Natal, I have to ask: Do you think Wii Fit users will give it up for Natal?
I ask that particular question because that is inevitably the only (gamer) market for such a device. Bear with me ...
How many gamers do you know?
Okay, how many gamers do you know that enjoy aerobics?
Okay, how many gamers do you know that will give up their rumble seats and VR headsets to jump around the room, waving their arms and ducking?
I think you get my drift. Comic Book Guy from The Simpsons certainly ain't gonna buy one of these devices. Same goes for every hard core gamer I know or have ever been exposed to. This is simply too sloppy, and always will be, given the time delays inherent in motion to pixel translation. It's only for slow games that do not rely on reflex. In other words: The boring games favored by the ladies (love ya, but you don't buy many games) and the wee children (same).
This (admittedly cool) version of this type of tech (around since at least 1990) will appeal to a limited gamer audience because it involves much more physical activity than hanging out, drinking Rockstar and wasting bugs does.
It really feels like Microsoft spent a bunch of resources trying to unseat Wii ... a little late, once again.
I predict that the true end game for this product is not going to be gaming. Maybe something along the lines of remote communications (controlling avatars) or something. It won't be for remote controlling machinery, because it's far more efficient to manipulate a hardwired prototype, so it won't be for medicine or security work, the traditional targets of biomechanical interfaces. Hard to tell where this will end up.
It would be good for Microsoft to stop worrying about following the leaders and to remember that gamers prefer sitting on the couch to jumping behind it.
Dude, I'm going to have to play it to believe it.
James Butler there doesn't realize that people are much more willing to change their habits than it may seem at first. Who will want to miss out on this? Besides, you saw the car racing game, you can play while sitting on the couch!
And why am I still stuck with only one camera?! I hope this thing is very hackable, like the wiimote :)
(BTW so this thing translates as... "Christmas"?)
This is amazing technology.
A wii and PS3 killer.
I hope its not vaporware
Someone linked me this video on Facebook the other day. It's obviously a concept demo, something an expensive marketing team put together rather than a group of tinkerers.
Just now, I saw your name in a headline and clicked-through to find a Natal on-stage demo at E3. As a developer for Google's just announced Wave system, I can't help but wonder how much of this live demo was faked. There's got to be someone driving offstage picking those colors as the guy's talking. There are probably plenty of other things that are baked in to the demo but should be set in real-time.
As you are evidence of yourself, Microsoft has some amazing researchers doing really cool things. I hope more of this demo is factual than cheated. That concept vid makes it look like vaporware, but I trust you guys are working on some really cool stuff under the covers.
I must wonder though, at the end of the day, how many parents are going to want their kids shouting colors at a TV? ;-)
Very exciting stuff. Games are cool, but the potential for completely immersive training experiences, and communication is really exciting. Can't help but be a little jealous that you get to work on such cutting edge tech, but even more jealous that you understand it!
Great job, I've been following you since the $14 stabilizer days and you've inspired a lot of my work.
Will Natal only be available with an Xbox developers license or will it be released for use with XNA?
This is not just for games. This very natural device for inputting 3d cloud data. Whatever is in front of the camera gets mapped. The Photosynth and Seadragon technologies are making an appeance hear and my mind just can't imagaine the potential applictions this is going to unlock. This Natal part with a body language inteface is just one potential application. I like the idea of the camera being able to recognise things I move around in front of it. Object search for Bing - find me another one of these. :)
This makes me wonder what the game corporations will invent next... a street? a football field? a shopping mall? Everything starts to resemble real life more and more, so why just not live the original real life instead? Get out instead of becoming integrated with your couch!
Following James Butler's comment, I feel it's quite unfortunate that this is directed to a very small target group of people. But then again, being a computer vision scientist myself, I know what great potential the technology itself has (and it is quite impressive, I agree). So let's hope this will get out of the reach narrow-minded, short-sighted, for-profit managers can envision.
All these tech are Time of flight .
Johnny Lee , It is very cool when knowing that you are one member in Project Nathal .
Maybe Microsoft will be a winner in this nextgent console war .
I have some theories which I posted on nui forum http://nuigroup.com/forums/viewthread/5886/ want to share with all :
Anyway , I wonder that if Microsoft want to take cover all licenses on Time of Flight tech ? I know that Microsoft bought 3DV (one small company has Time of flight tech) http://en.wikipedia.org/wiki/ZCam . Big M$ also bought licensees from GestureTek on Time of flight tech , in this webpage http://www.gesturetek.com/3ddepth/introduction.php , look at demo , it is the same Project nathal , exactly , and scrool to the bottom of the webpage you can see EYETOY and XBOX360 logo . So , what can I say ? I think sony bought licensees for ps2 eyetoy , but that gesture tech version is the old one, just 2D gesture . In the case of project Nathal , It can sense 3d object , so this gesture version is the new version . And I think big M$ bought exclusive licensees for long term (maybe 10 years ..) , adn they also bought 3DV company . So, Micrsoft will be the winner in next generation console war . Oh man ! who know . Sony and Nintendo can't purchase Time Of Flight tech due to exclusive licensees .
Impressed beyond words.
Can't wait to get my hands on project natal. I hope it will be as good and cool as we've seen so far i demos.
ignore the stupid fucks and keep up the good work, johnny chung lee
My cautious optimism has gotten slightly more optimistic. I'm glad someone of your stature is involved with this seemingly ground-breaking project. Obviously, it will be a year or more before this tech lands in our hands, but the strides we have made in this console generation are astounding. Good luck and Godspeed to you guys!
Also, for people who don't quite believe in this tech yet, here is a hands-on preview from a reputable, unbiased source: Brian Crecente from Kotaku played Burnout: Paradise (not a game designed for Natal) using Natal controls. http://kotaku.com/5276789/from-stick-figure-to-mind+blowing-controls
Good stuff, I'm extremely jealous of people who can and are contributing to the development of cutting edge technology
This fantastic future of Xbox will change my life! Thanks for developments!
xboxnatal.ru
I have to say I was meh on Project Natal, but seeing how you're involved in the project now has me excited. You've done some amazing things with the Wiimote, and to see even a fraction of those ideas translated to Project Natal would make it above and beyond what's on the market now.
Personally, I think the guys are onto something here!
RT
www.real-anonymity.pro.tc
Great read.
Too many people are just assuming this is the same as the Vision/Eyetoy cameras available now. I really think Microsoft must make it crystal clear that this is very much the next generation of interaction.
So when this really becomes revolutionary, we will refer to the time prior to this great achievement of man kind as "Pre-Natal".
*zing*
Its good to see your transition from tinkering with the now basic wii-mote translate into something that will change the face of HCI in a way that everyday users such as myself can use. Well done!
Johnny, any time I'm in the mood to completely blow away anyone (techie or otherwise) with the future of videogames, I show them your VR head-tracking video. I don't doubt that you have many other involvements with Natal and head-tracking may not even be one of them, but I really hope to see something similar to your technology in games sooner than later. Please???
Not to be a downer, but there are a number of things wrong with this idea. Not everybody has a 40+" screen to play on, nor do they have fifteen square meters of floor space to occupy. People aren't going to want to have to rearrange their furniture to play a game when they know damn well they could be holding a controller. Further, its still going to be a gimmick, much like the Wiimote was for the first while. It had its moments (Trauma Center!) but it was mostly waggling (Any game with a sword). Besides which, if the wiimote couldn't do Red Steel properly (gestures were used instead of button combos to input moves, rather than direct translation) then what are the odds of this working properly? If I'm driving, will it actually accept handoverhand movements, or consider the steering wheel to be on the opposite side because of the orientation of my hands vs. the invisible virtual wheel (like MarioKart - soon as you turn past 180degrees, your wheel goes from all-right to all-left, and you lose control) Could it determine between me holding a sword, and me spinning said sword (which would appear to be me using nunchuks, without a physical object that the camera can recognize)? How would we alter the camera angles ingame without an extra stick? Where is the Big Green X button going to be?
All of this tells me that despite the wonderful and fantastically life-changing technology (just like the wiimote five years ago, remember) we will still be playing our games with a controller. Because god knows that I am simply incapable of doing any kind of high kick with my actual body - this is why I play games in the first place.
Whoa dude, I wasn't sure about this project... but now that I know you are working on it I might have to go get a 360.
Okay, now I have to get one.
Imagine Fight Night 4 with this tech, no more complext moves just precie combos.
maybe they could have a subtle mode, where you can use the tech sitting down.
This reminds me of the Rumours before the Wii was anounced, i forgot the name, but the specualtion was insane after fan made a video.
this tech plus 3d projection is the future of gaming.
as someone interested in the field of computer vision, I'd be very interested in seeing some of the techniques used published.
HELLO!!
People are missing the bigger picture here.
We are talking about a device with the additional add-on of an earpiece and some object recognition. It could recognize a walk sign from a don't walk sign. Keep a person on a sidewalk and between the lines in a crosswalk. It wouldn't be easy at first being overwhelmed with constant audio info about objects and distances being constantly relayed to the ears but we humans have the amazing ability to adjust to new circumstances.
This could replace the seeing eye dog.
No more canes
After seeing your videos on Youtube, I knew someone had to pick you up. Anyone would be crazy not to hire you!
Good luck with all the bleeding edge tech work!
Been a fan since the $14 Steady-Cam, keep up the good work!
To all the fellows who say they're jealous of Johnny Lee -- just study harder! :)
"This makes me wonder what the game corporations will invent next... a street? a football field? a shopping mall? Everything starts to resemble real life more and more, so why just not live the original real life instead?"
Arguments of this time are easily countered by the fact that in real life you can't really fight aliens. (Or can you?... No, you can't!)
Hi,
Can we combine natal with Vuzix Wrap 920AV ?
It will be nice, the augmented reallity stuff goes to market
so we can type virtual keyboard seen on eye glasses, while natal recognize the movement of our finger
Hope you and your team catch the idea
This is simply amazing. Being one of the hardcore gamers I'll have to admit, it will be difficult leaving the controller behind. But also being athletic (like several of my friends on xbox live) I think it will be well worth it.
However I do have concerns:
1. How accurate are the vision sensors and recognition algorithms?
It could be real frustrating in the middle of a fight to have to redo the same command because it wasn't recognized the first time.
2. How durable is it? I've noticed lately that things have grown far more fragile than they were in the past when that certainly should not be the case. I wouldn't want this device to get knocked of the t.v. and bust due to all the jumping around you will most likely be doing.
3. Will gaming be reduced to a bunch of cartoony characters and childish mini-games? Will the leading game producers be able to incorporate this new device into their games easily and effectively?
Just figured I'd throw these ideas out there.
-- Dj
I'm guessing based on the wikipedia page that they're actually using IR light sources with polarizing lenses to project a grid, then using the deformation of that grid to calculate 3d coordinates. The technology is actually very old but there hasn't been much done with it. I have a robotics book from the 90s that talks about it.
Please combine
project natal
+
Vuzix Wrap 920AV
= true augmented reality !
also .net framework wrap !
maybe we need circular treadmill too
What you don't say is that this technology was born inside an Israeli company called 3DV systems who were purchased by Microsoft.
@ dorbie
To the contrary, MS exec Shane Kim stated yesterday that this technology was developed in-house and was in progress for "a long time" before 3DV was purchased.
http://www.eurogamer.net/articles/e3-natal-not-derived-from-3dv
My guess would be that MS bought 3DV to avoid potential patent issues and, if they had something good, smooth the road for future adoption of pieces of their technologies.
Anyone who doesn't think this can appeal to hard core gamers, obviously hasn't ever met an *actual* hard core gamer. Uber-nerds have been using things like the TrackIR for years. These are the same people that spend $700 on joysticks. If MS can bring that sort of functionality to the merely "serious" gamer, it would be incredible.
Now add a few party/sport games to grab the casuals, and boom, you've appealed to the entire gamer market, top to bottom.
Here's a little video that demos that TrackIR thing. Natal should be able to do similar things without breaking a sweat:
http://www.youtube.com/watch?v=9wXx3vMy_AQ
You are the best !
I love how the first thing people ask is when is it coming out and how much does it cost.
It will be a LONG time before you see this on store shelves, this is brilliant technology and probably has a bit of a ways to go. That said I am uber-excited to see what comes out from this.
I'm curious as to why everyone assumes that, because there's this new "camera" controller, that your regular controllers just vanish. I forsee games using the stock XBox 360 controller along WITH the camera system, if appropriate. Awesome to see you on the project... even more excited now!
...my only fear is "Totem Ball 2"
So is the Milo demo in 3D then?
Very impressive work. I'm really impatient to try it.
Just for fun, check what could have been the answer of Nintendo :
http://www.youtube.com/watch?v=wnujjhZkPPs
Simply awesome.
I don't think this wouldn't work well with television gaming. It will replace some game types but this wouldn't revolutionize the game industry. Some people wouldn't buy this because those people wants to just relax, not move around and get tired. lol
Target and Sell this to fat people who wants to lose weight!
I might enjoy seeing people kicking their television and breaking them in YouTube videos. lol
But this is a great invention to revolutionize way we control computers, this might extinct mouse controllers. lol
any chance for a pc port of this?
im drooling at the thought of modifying it for pc use.
Constantly I see mocap and video games with live mocap done in 3rd person with no developement for first person, ie. if you can do cost affordable live mocap using a device like this, this means all you have to do is put on glasses,re-orient the view from 3rd person to 1st (computer camera view inside characters face, like a fps view) and you have a websuit similiar to that used on the movie the lawmowerman, only without the websuit...
glasses would be vr glasses like used in the movie the lawmowerman, this would give the sensation of actually game emmersion, ie when you move your hand, youd see your characters hand move as your own.
if you havent seen the lawmowerman with jeff fahey and pierce brosnan, def worth a lookat.
thats the lawnmowerman, new line cinema. ahh my keyboard. lol
"Natal" means "Christmas" in Portuguese. Unless i'm mistaken. Coincidence? Christmas seems like a very good time of year to release it.
@Alan
Yes, "Natal" means Christmas in Portuguese; but in this case it's because it's the hometown of the project's mentor.
As in Natal, Brazil. :)
> The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I've seen in academic research, never mind a consumer product.
Are they going to publish their work?
Really cool video and work. You could build a whole new world of applications that you could interact with on the XBOX. Imagine games like interactive World of Warcraft, where people will actually have to fight to win. The possibilities are endless. Great work!!
Wowsers!!
Can't wait to get one of these.
If you're looking forward to Project Natal, check out project-natal.com for the latest news around the web.
This is pretty much the first step to a motionresponsive holographicdisplay, like in Ironman.
I'm getting all warm and fuzzy inside thinking about writing grames for this.
Now to get this working on my PC when it's out... :D. Won't be buying the xbox, since I already have a state-of-the-art PC.
... Hopefully GlovePIE will be able to do it, or maybe some other program. Or maybe Microsoft themselves will release it for the (windows) PC?
This will clarify some recent discussions surrounding GestureTek in the 3D space. GestureTek is the inventor, pioneer and world leader in video gesture control technology for both 2D and 3D cameras. Our software works with 3DV, Mesa, Prime Sense, Canesta and other 3D depth cameras. For more than 20 years, we have been creating 3D tracking technology and evangelizing video gesture control as the next step in computer human interaction. With more than 4000 public installations worldwide, we have a robust library of video gesture control technologies, applications and patents. GestureTek has licensed these patents and technologies in various ways to a multitude of consumer electronics providers, including for PlayStation, Xbox 360, Hasbro, etc. GestureTek is proud of the trail we’ve blazed in the area of 3D depth tracking and control and our current leading position in the market today.
Here’s more on GestureTek’s 3D tracking and control software: http://www.gesturetek.com/3ddepth/introduction.php .
Here are some examples of work that GestureTek was showing more than a year ago, before Natal:
- Beijing Olympics Flight Simulator http://www.gesturetek.com/3ddepth/businesscases/3dexplorer.php
- Two-Handed Control Driving Demo http://www.gesturetek.com/newscenter/media.php?media=58
- Rock em Sock em Robots http://www.gesturetek.com/3ddepth/introduction.php
- Our technology is also being used in a 3D depth sensing interactive digital signage system for Sprint.
Patti Jordan
+1 for more info on the hardware and software solutions presented here.
I'd sincerely hope MS Research give back to the Computer Vision community as much as they've received from it.
The video was awesome, but was I the only one to spot the Natal system had miraculously obtained details of the underside of the 'skateboard that was scanned' that were occluded by the rider's fingers? Ok - I guess they could assume Markov Random Properties and texture fill the gaps...somehow I just get this feeling it was more a staged video effort instead. I would LOVE to be proved wrong though since it does look very very good.
If you play boxing and get punched, can you feel the impact of that punch but not be hurt?
If you get shot can you feel like you've been shot but not be hurt?
If you play tennis can you feel the impact of the ball on the racket but not smash a window?
If you kick a ball can you feel what it's like when you really kick a ball but not have to go and fetch it?
Are any of the experiences offered by this "new" technology really any different or just a different way to experience the same "old" technology.
If the former, I'm in.
If the latter, I'm out.
And I'm 31, cash-rich and bored!!
It seems like the holodeck is "invented" by the videogame people every few years. The holodeck is a fantasy, which reality can only asymptotically approach. WoW, which uses none of these "technologies", has paradoxically achieved a much greater immersion factor than any game on the Wii or VirtualBoy or what have you. At the same time, Mr. Lee is awesomely cool and I wish him, and Microsoft, the best of luck with this product, which, let's face it, will be pretty hard to sell.
"The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain... ...At times, working on this project has felt like a miniature “Manhattan project”..."
Because eventually, robots will use this technology to identify and destroy all humans. :)
Awesome tech.. can;t wait ....
I wish I can test this baby....NOW!!!
Suhas
suhasharma@gmail.com
I dug around your VITA but could see anything, but is this closely related to active appearance modeling by Cootes? I'm a grad student that is slowly getting more and more into the HCI world and that's where my focus is leading me
Are there a relationship between Natal , 3dV and GestureTek ? I know that Micsrosoft and Sony bought licensees from GestureTek. I don't believe Natal is in-house development.
For those begging for more info, I hate to say it but you probably ain't gonna get it - any disclosure at this point would likely invalidate patent applications (it's a reasonably common thing to see researchers rush off to present and/or publish papers only to discover that they've just destroyed their IP protection).
I suspect, given MSoft's buying up of 3DV, that this is going to be handled as a submarine patent to gain trade secrets protection too - filing of a provisional application and then repeated extensions to prevent that patent from being completed (and publicly available) until threatened by a rival technology. If that is the case, then SDK information will likely be through an intermediary interface.
Pure speculation, of course, but given the money that MSoft seems to have thrown in this direction they'd be rather stupid not to protect their investment.
Oh ! I posted some theories on Time of Flight tech (sorry , post is not in english) http://lethanhhung.blogspot.com/
I believe that Microsft used Time of Flight tech on Project Natal.
VR Display with head tracking!
Tell MS to let you sell Natal buy doing tricked out cool stuff with it on Youtube!
I;m amazed that Nintendo didn't snap you up first - congrats!! +1 sale :D
you're part of this?!
sick
Hey johnny, its nice to get a bit of technical info on Natal, I'm also in the image processing field kinda fell into by accident on my way to other topics.
I presented a paper at the ICPR (international conference on Pattern Recognition) conference (which to my knowledge is one of the premier computer vision conferences) in dec last year.
I didn't see anything remotely close to what natal claims to be capable of, there is no way you would able to do that with normal cameras. The use of a special camera that generate those point sprite image does make it seem a lot more feasible.
I'd love to get more info on the camera, I'm sure a lot of the computer vision researchers would kill to get their hands on it.
Even so, I still think to a large extent the system is still a little over hyped, I would love to see a real demo of it with people walking behind/in front of the players and see if the system will still maintain its target points.
Just to clear things up, industry is usually miles ahead of academia in regards to real world applications, and most of the research they do never gets released due to NDAs and so on.
at the end of the day the proof is in the pudding as they say, I'll be interested to see it in action, and not some "semi-faked" closed door presentations.
very, very skeptical. while me and my fellow nerds will no doubt enjoy the system in it's final stage, it will not create the family atmosphere that is shown in the commercial. the very first time it fails to recognize my mother or father or pretty much anyone who is not a Nerd they will never try it again. keep that in mind when you make statements like "this will revolutionize gaming."
...holy crap man. No WONDER Microsoft picked you up! That's absolutely insane. When was expected release / expected pricepoint?
I just remembered something from the distant past:
Back to the Future Part II - Marty plays Wild Gunman in 2015 in the Cafe 80s. Girl walks by:
"You mean you have to use your hands? That's like a baby's toy!"
Freaky.
My God this is awesome!! It seems almost impossible and you guys succeeded! Super awesome! Be proud Johnny. You're actually part of something really ingenious..
I suppose that 3d camera and chip were the last piece of the puzzle. Maybe the most important one even, since voice and facial recognition were in development at least for a dozen years and works well nowadays even on PCs.
In any case, MS is probably the only company in the world who have the resources and talent to put all pieces together.
Soo, simultaneous multiplayer.
This gaming system is totally amazing. Hello...George Jetson.
did not work 100%
This is some sort of black magic. I'm at the same time conspiracy level suspicious, and childishly optimistic. If this comes to fruition at the same level of immersion as it's being presented with, then that may be the first time that a product has reduced me to tears of joy.
Good luck with the implementation. I'm optimistic, but skeptical.
Wow, Your Headtracking VR system with the wiimote would now be possible with Natal - WITHOUT - the IR Glasses.
Go Johnny. Go Johnny, go go go.
To everyone who is a part of this, I am excited beyond words.. holodecks indeed.
KEEP IT UP! I am so glad to read about what we are all doing Beyond 2000 :D
I love comments about how people won't like Natal and there are no good uses for it and living rooms aren't big enough, etc.
Why can't you just appreciate the opportunity for new creative possibilities and use your brain power to build instead of just raining negativity. It is so small minded to categorically say that people won't like it.
Why don't all of you nay sayers get together and create an experience where all you do is rank on things. That should be fun. And please close the door so we don't have to listen to it.
From what I've seen so far, Natal looks great!
I hope the 3d sensor has the ability to work in very low light conditions. I use a projector and it would be great if this worked in near darkness.
It will work even in absolute darkness. Those 3d cameras are based on lasers and receptors.
The camera is not even an important component here, the important is the chip that gives out 3d map in real time.
Congratulations! I've been following your blog for a while as an enthusiastic Wii user and been consistently impressed with your experiments with that tech. Clearly someone at M-soft noticed it too - congrats on landing an exciting (and well-deserved) position. :)
I saw your video on head tracking with the wiimote (dude awesome) you have been a geekhero of mine since then and to see your name pop up with this project awesome. I am amazed that the software is able to track a person in what seems like a split second. I saw a pic of the demo with Jimmy Fallon and the red jump suits was that just a gag or was that used so the camera wouldnt slip with all the bright lights.
Dude you keep rocking, I am not a code guy but I do understand the wonderment that you guys can create and whatever project you are apart of I am putting money down on it because I know it is going to be great. I am sorry I am kind of star struck you da man.
It's genuinely going to be immense,
As a gamer: I am very excited to see the possible applications that this technology can deliver to enhance my experience. I, among many others, play games to escape from the real world from time to time, and this will (possibly/hopefully) further immerse me into my imagination.
As a game designer/developer: The possibilities are limitless. While some of the projected uses are "neat", in the short amount of time since the announcement of this project I have already envisioned a vast array of applications for this technology. I, for one, will most definitely push hard for further development with/for this tool for my future projects, hopefully inspiring others to also find unique and practical uses for it.
As a technologist: While the technology itself is not new, Johnny says it the best himself: "... providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost." This will undoubtedly make this type of technology more accessible in a casual market. Even if an SDK isn't provided "right away" its only a matter of time before its reverse engineered. I wouldn't be surprised if in 10 years (or less) this technology is completely mainstream, where businesses can set up displays to usher customers into their establishment without having to hire extra help.
In the posted video, the first scene shows the teen interacting with an AI that follows his movement around the room, "talking" to him. Using this same sort of application, I immediately envisioned walking through a mall and seeing a television displayed in a shop window. While walking by the window, the virtual usher greets me with an advert, or possibly a clever or witty hook, all while "following" me. This experience of course feels very personal as it seems the interactive AI is talking to me and not a generic community. With this in mind, the AI could react to my interaction to its introduction, such as if I walk into the shop I would be welcomed in kindly. Or, if I chose to ignore the introduction, the AI would respond differently, perhaps something that may incite emotion (i.e. guilt). And because of this project, this type of advertising may become very popular, not only because the team found a more cost effective way to do this, but also because of the popularity of video games in this generation will spark interest among marketing teams to invest their resources to make this happen.
The applications are endless, in the video game market or otherwise. Awesome job to Johnny and everyone else working hard to bring this to life.
I wonder what would happen if someone were to place a mannequin in front of the camera, if Natal would think it was a player. :)
I've seen comments about Natal all across the net, from "Xbox vs. Wii and PS3" to "It's a shame that this tech is being wasted on games that even gamers won't want to play".
I shouldn't be, but, I'm completely surprised at the limited scope of these commentators.
Project Natal, including the games (Project Milo, etc.) that will eventually support it, is a way to perfect this tech, recoup the R&D expenses, push the Xbox platform, and get the population accustomed to interacting with software in ways that most of us would currently find uncomfortable if witnessed by even our loved ones.
To think that Microsoft doesn't have plans outside of gaming for this tech is absolutely ridiculous. Take what they learn from Project Milo 5-10 years into the future. A personal database that you interact with visually and vocally, whose avatar is completely customizable, that you could put on multiple devices, that's tied to Bing and MS's other services. If you can't envision the possibilities that this would allow just trust others when they say it's mind-boggling.
Natal, as a gaming device, is the perfect cultural Trojan Horse for Microsoft to get us ready for what lies ahead..
Maybe now someone can build a home-alarm motion sensor that can distinguish between a human burglar and my cat! (I'm not sure what it would do with a "cat burglar.")
A week after I saw your wiimote head tracking demo, I Threw together a hack with your code and OpenCV to do head tracking; and I was blown away at the sense of immersion I got.
I think I speak for a lot of people when I say, this HAS to be done with project Natal.
In fact I may just fire up XNA now and toy with the idea :)
2d tracking was too hard, so now they have moved to 3d tracking to more easily separate objects, but it's still got a looooong way to go. I suspect that, like the wiimote, it will only sorta-kinda work.
Nice video i found it so interesting
Chikabebe
I've got a few questions:
How can you walk and look around (a must for, for example, an FPS) without a controller? you can't expect people to walk in place or do a moonwalk thing to walk through the game...
For looking around you could make the screen scroll when you're pointing your hands/fingers to the sides of the screen but then you'll loose the realistic effect because then the game doesn't do exactly what you're doing anymore and you won't be able to shoot accurately.
also how do you run, jump, roll, move the camera, and do things like controlling speed, or play a game like mirrors edge?
what if you're para-jumping in the game? How do you control that? Or when you have to swim or dive somewhere?
even when you have your own Holodeck you won't be able to swim and para jump etc. with the same movements.
I can't think of a way how you'd solve this problem without taking something away from the game experience.. or of course using some kind of controller anyway..
You share valuable information and excellent design you got here! I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs up. Please come visit my site Long Beach Yellow Page Business Directory when you got time.
I was thinking of looking up some of them newspaper websites, but am glad I came here instead. Although glad is not quite the right word… let me just say I needed this after the incessant chatter in the media, and am grateful to you for articulating something many of us are feeling - even from distant shores. Please come visit my site Sacramento Yellow Page Business Directory when you got time.
You may have not intended to do so, but I think you have managed to express the state of mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site Santa Ana Yellow Page Business Directory when you got time.
Hmmmm. Spatial and visual depth and positioning determination is one aspect of the Human to Machine interface. What is still missing is the tactile pressure feedback or resistance to movement which humans would meet in real life. Until this is incorporated it is still "virtual" reality, the ability to feel the pain (via controlled electrical stimulus perhaps) would make it less virtual and more realistic allowing each individual to achieve independent personal limits (not those pre-programmed by a young techie geek). If a "holo deck" is the ultimate goal then the introduction tactile feedback is a must to ensure physical barriers can be detected and determined ie walls, doors stairs etc. It would be interesting to see a device that can stimulate a human response to an action to "virtualize" climbing stairs and present the gamer with "tired" legs and exercised muscles after the climbing event.....now that would make games sell.
Hmmmm. Spatial and visual depth and positioning determination is one aspect of the Human to Machine interface. What is still missing is the tactile pressure feedback or resistance to movement which humans would meet in real life. Until this is incorporated it is still "virtual" reality, the ability to feel the pain (via controlled electrical stimulus perhaps) would make it less virtual and more realistic allowing each individual to achieve independent personal limits (not those pre-programmed by a young techie geek). If a "holo deck" is the ultimate goal then the introduction tactile feedback is a must to ensure physical barriers can be detected and determined ie walls, doors stairs etc. It would be interesting to see a device that can stimulate a human response to an action to "virtualize" climbing stairs and present the gamer with "tired" legs and exercised muscles after the climbing event.....now that would make games sell.
Project Natal, if similar to the tech demo shown at E3, will revolutionize user experience all across the board.
Didn't get to read half the comments, but the possibilities are astounding.
As shown in the demo, it would be great if a game were developed (though more of a athletic simulator) which acted more like a personal trainer for working out (with/without weights)... or perhaps a martial arts game were developed to help teach people self-defense or meditation techniques while monitoring the person's form.
Though it would be great if a person were scanned and projected within a game. It would be neat if the scanner also acted as a projector to project the simulation onto a wall surface. (case someone doesn't have a big tv) or the scanner were used in conjunction with virtual goggles to create a truly 3d virtual simulation.
Above all, we all look forward to your groundbreaking research at Microsoft and the products that come forth.
It is a very nice website with all those good videos. Enjoyed it.
My Blog : earn money chao!
Hello
Natal project Great really so Nice
No more comment excellent work.
Best Wishes
Archibald
http://www.infysolutions.com/resources/resources.html
WOW! Is this real? no controller required? I think this will forever transform gaming and entertainment with a whole new way to play.
Amanda
my site
I am not really sure if best practices have emerged around things like that, but I am sure that your great job is clearly identifed. I was wondering if you offer any subscription to your RSS feeds as I would be very interested and can't find any link to subscribe here Please come visit my site Norfolk Business Directory when you got time. Thanks.
I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site?
Please come visit my site Chesapeake Business Directory when you got time. Thanks.
puma mens shoes
nike air max ltd
NIKE air shoes
nike max ltd
puma womens shoes
nike shoes
puma shoes
cheap nike shoes
cheap puma shoes
I like your page very much. It is really wonderful. The way you talk about this is very cool.
puma mens shoes
nike air max ltd
NIKE air shoes
nike max ltd
puma womens shoes
nike shoes
puma shoes
cheap nike shoes
cheap puma shoes
米蘭情趣用品我情趣用品我情趣我飛機杯我自慰套我充氣娃娃我AV女優我按摩棒我跳蛋我潤滑液我角色扮演我情趣內衣我自慰器我穿戴蝴蝶我變頻跳蛋我無線跳蛋我電動按摩棒我情趣按摩棒我丁字褲我G點我SM我後庭
情人趣味用品我柏拉圖情趣用品我成人論壇我情色論壇我一葉情貼圖片區
視訊美女我視訊做愛
網頁設計
crystal jewelry
wholesale crystal jewelry
wholesale jewelry
jewelry wholesale
cheap jewelry
wholesale gemstone jewelry
crystal wholesale
rock crystalri
I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site? Please come visit my site Prince Edward Island Directory when you got time.
As a Newbie, I am always searching online for information that can help me. Thank you. Please come visit my site Northwest Territories Directory when you got time.
Provides the best tiffany jewellery , including Necklaces, Pendants, Bracelets, Earrings,
Rings at the lowest prices. tiffany jewelry is the best online United Kingdom jewelry stores
where you can buy the cheapest tiffany jewelry Co silver jewelry.
Our huge selection of tiffany uk and
tiffany jewelry and tiffany jewelry.
Im 18 years old and as a struggling student who enjoys racing ugg boots ultra short Le its
becoming extremely expensive and harder to attend and enjoy a days racing. Sure a student
with a part time uggs upside job would have to save for a few weeks to consider
attending a raceday as expensive as this. I see that Fairyhouse are racing on the 19th of this
month and are charging uggs whitely boots 5e entry fee. Its up to other tracks to follow suit
and use some initiative to offer better value to the normal racegoer…!Pretty good air jordan
post. I just found your site and wanted to say that I have really enjoyed browsing your
Bailey Button ugg boots posts. In any case I’ll be subscribing to your blog and I
hope you nike sb post again soon!
Every girl wants to make herself more beautiful and charming. For that purpose, you also
can choose wear bangles, bracelets(like Tiffany Bracelets ), charms earrings, necklaces
accessories, sets (like Tiffany Sets ), pendants and rings(like strong>
Tiffany
Rings ). Tiffany jewelry onlinestore provides a variety of tiffany jewellery, including
tiffany bangle, tiffany bracelet, tiffany charm, tiffany necklace , tiffany pendant and tiffany
ring.
Pervert! Let's get out of here. There are so many perverts here.So what? You are such a
bad person Who's gonna like you? Tiffany Bracelets Don't mind him. Don't mind him.
He's just playing. Forget him. He saw me steal the diamond. Tiffany Sets ,
Tiffany
Rings ,tiffany earrings,Tiffany Necklaces & Pendants You're dreaming! In your
dreams. links of london Wake up and smell the coffee.Where'd you get it!Good idea.Think
it over.You can never tell about this sort of thing.Take a closer look.When did I say that!
You have to be on time. Don't be late.
Our huge selection of Tiffany Jewelry. Tiffany Bracelets JewelryEarrings, Tiffany Rings ,
Tiffany Sets )
is the best online links of london charms stores where you can buy the cheapest Tiffany
jewellery and links london. Hope everyone will like these things!
Hi. I'm sure it connects via USB and I'm wondering if microsoft is going to limit functionality to games on the Xbox, or more positively open it up to open PC/hobbyist/Academic developement with an available SDK containing all those algorithms.
us drugstore
ed hardy Shoes
ed hardy jacket
ed hardy jeans
Well, nice article buddy… Someone will love to read this infor if I tell her about this. She’s really interested in this subject. Thanks again… Please come visit my site ear nose & throat when you got time.
It is very interesting article and quite impressive and more informative and looking forward to read such article. Please come visit my site Miami Yellow Page Business Directory when you got time.
. Since I’m new to blogging, these articles are greatly appreciated; very useful and informative blog and every body must visit this blog. Please come visit my site Tampa Business Directory when you
I am excited for all of the Project Natal Games that will be coming out.
Thanks ever so much, very useful article. If you do not mind, please visit my article related to pandeglang district in Banten, Indonesia at Kenali dan Kunjungi Objek Wisata di Pandeglang and also related to a leadership at Mengembalikan Jati Diri Bangsa and Oes Tsetnoc or Oes Tsetnoc the second. iklan baris gratis | jaringan iklan gratis baris | iklan baris gratis | pasang iklan baris gratis | submit iklan baris gratis | media pasang iklan gratis | promosi gratis iklan baris gratis | iklan baris gratis | pasang iklan baris gratis
I have always stayed away from microsft Xbox technology. Maybe it's ok, but ...
project free tv
You got a really useful blog I have been here reading for about an hour. I am a newbee and your success is very much an inspiration for me. Please come visit my site Fresno Business Directory when you got time.
Hello mate, I want to thank you for this nice blog. Would you mind telling me some secrets for a succesful blog ? Which could attract some visitors than it normally does. Please come visit my site Sacramento Business Directory when you got time.
I like your blog. Thank you. They are really great . Ermunterung ++ .
Some new style Puma Speed is in fashion this year.
chaussure puma is Puma shoes in french . Many Franzose like seach “chaussure sport” by the internet when they need buy the Puma Shoes Or nike max shoes. The information age is really convenient .
we have lots of polo shirts
Abercrombie Fitch clothing
Abercrombie Fitch hoodile
ralph lauren shirts
Columbia Jacket
north face jackets
spyder ski jacket
polo shirts
Post a Comment