Thoughts on Project Natal, Milo and the coming Motion Tracking Videogame and Technological Revolution

Bill Loguidice's picture

Amidst all of the usual software-centric sequels and somewhat tired continuations of long running series at E3 was a clear, present and somewhat surprising focus on hardware accessories, and, more specifically and perhaps most exciting, next generation motion tracking and control systems. This wasn't just an attempt to copy Nintendo's almost-there original Wii Remote technology, but rather an attempt to redefine the technology once and for all and influence videogames and the technological world at large for generations to come (think integrated touch and motion controls in your 2015 laptop).

Of course Nintendo's approach was already made known and will be out shortly - the Wii Motion Plus - which is a small snap-on attachment for their present Wii Remote that offers pretty much 1:1 tracking, enhancing the precision of the sometimes hit-or-miss controller to a very high degree. I'll take a more detailed look at the add-on after its release.

Sony's PlayStation Eye-powered motion technology aims to turn a light emitting wand controller into any virtual item imaginable on-screen (suggestions included swords, tennis raquets, whips, whiskey bottle, etc.). Of course the PlayStation Eye is their pre-existing camera - which I have - and which already can work with real-world objects, a la the Eye of Judgment, which I also have, but have yet to put through its paces (Microsoft has a camera of their own for the Xbox 360, but it's not quite as sophisticated). Anyway, the unnamed light emitting wand controller is expected to see release some time in the spring of 2010, so expect it to go through some refinements between now and then assuming Microsoft's stuff (also a 2010 release) - discussed in the next paragraph - continues to take shape like it is.

As for Microsoft, in regards to at least this type of technology, they went above and beyond and stole the show with Project Natal. As they say, "no controller required". Essentially using a smallish bar that sits above your TV, you and all of your movements are tracked in real time. It also naturally has camera and audio (microphone) technology, making it a complete real-time tracking and control system. Based on the demos and comments from developers who have early dev kits, this actually works, and has lots of people excited, including videogame enthusiast Steven Spielberg, who most recently dabbled in the videogame world on the Wii with the excellent Boom Blox.

Perhaps most intriguing and showing Natal to its best effect was the Lionhead demo for Milo, essentially a virtual boy/playmate type of thing, though practically to the level of science fiction. You really have to see the demo to really appreciate all this has to offer. What's so big about this is that I've been watching for years how university researchers have been dabbling in this type of technology, both virtually and through "emotional" robots, meaning robots that can show human-like facial expressions. What's so interesting about Milo is that it looks like it one-ups all of that type of stuff, meaning we've taken that next generational step in the research, allowing for goodness-knows-what future possibilities, and all this in the mainstream rather than academia, where it can actually reach people in a reasonable timeframe.

Lionhead's Milo Project (the beginning of this is the tail end of the general Project Natal demo, so just be patient before the Milo part begins):

Unlike the past, these things have a real chance of becoming integral to our gaming (and even general computing) experiences, an inextricable companion to our well worn and proven controller methods. Why do these things have a chance this time versus the past? Well, in the past, these have been all one off not-quite-there-technology, a la Broderbund's U-Force or Mattel's Power Glove, but this time the technology is to a highly usable point - and, more importantly - the big three - Nintendo, Sony and Microsoft with their vast resources - are behind it all. These are three of only a handful of companies on the planet that can successfully launch a mainstream videogame console, and they're certainly more than capable of reimagining the industry itself.

As for Milo and the other stuff being just a fad, gimmick or unnecessary, just remember that the same thing was said about e-mail - why send an electronic letter to someone when you can just as easily pick up the phone? Sometimes the application of this stuff can take on an unexpected utility and become integral to society itself. Don't count out the latter as a strong possibility.

One final thought and something I think will have to be addressed at some point is the lack of real-time, physical feedback. At some point, assuming partial body suits or gloves and the like are not a realistic option (and they're probably not), some type of practical audio projection system will have to be developed, one that hits you with sound waves where you can feel when you're touching or hitting an object (or you are in turn "hit" in a game or receiving a hug from a friend). While the "phantom limb" effect is somewhat there when playing controllerless games, a la Sony's Eye Toy for the PlayStation 2 and the karate mini-game where you punch on-screen enemies - you somewhat "feel" the impact through clever use of visual clues and bass heavy sounds - it's not enough for truly profound feedback. Something does need to actually touch you or at least FEEL like it's touching you to complete the effect.

Anyway, I'd love to hear your thoughts on all this. I certainly think since there's a brief "the future" section in our in-development feature film documentary, that this will be worthy of a mention or two, as I really think this is a big part of that future.

Comments

Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Great post, Bill. I've

Great post, Bill.

I've never been particularly impressed with motion sensing technology, though I could easily see how it could work for certain kinds of games. I think it's just easier using a controller from the comfort of your couch than gesticulating and gyrating in front of a sensor (or waving your arms around, even).

I do love the special seats that give you a more visceral feedback. I didn't like the vibrating controllers very much, but moving that tech to the chair does make a noticeable difference. Of course, the costs are much too high for consumers, but who knows what may happen in the future. The old "body suit" thing gets mentioned a lot. Maybe they could just use mild shocks or electric pulses to simulate touch, but that could be dangerous and who knows how all that electricity would affect you over the long-term. It's about the only way I could think of though that you could get a fine-tuned touch response that could really be jacked up. It's just too big of a pain to have to put on something like that just to play a game. Sitting in a chair is much easier.

I just watched Top Gun for the first time last night (gasp, I know) and now watching the special features. One thing that struck me was how much they talked about how flying a fighter jet is a physical, athletic experience. People have no idea how it actually feels to fly in one of those (the g's, the stomach leaps, etc.) Obviously, no matter how good the graphics get, this a realm of realism that's only possible with some kind of special hardware.

n/a
Bill Loguidice
Bill Loguidice's picture
Offline
Joined: 12/31/1969
Refined virtual reality
Matt Barton wrote:

I've never been particularly impressed with motion sensing technology, though I could easily see how it could work for certain kinds of games. I think it's just easier using a controller from the comfort of your couch than gesticulating and gyrating in front of a sensor (or waving your arms around, even).

As you know, as part of my collection, I have a vast array of "yesterday's" technology, some of which appeared or was discussed in our Vintage Games book in the Dance Dance Revolution chapter. The big problem with a lot of that technology was always some time of fudging to make it work or cost cutting to keep the price reasonable. The first really functional technology at a reasonable price was Sony's original Eye Toy, though even that had its limits. The original Nintendo Wii Remote is pretty good when used right, but without the addition of the aforementioned soon-to-be-released Wii Motion Plus add-on, it's nowhere near 1:1 tracking, which is really necessary for that sort of thing.

I agree with you about comfort. I prefer using a regular wireless controller from my couch. I do other physical things - I don't necessarily need them from my videogames. Again, though, we're not talking a replacement, we're talking a complement to existing technology. Just like touch screens, multi-touch, etc., have revolutionized interfaces by finally becoming reasonably ubiquitous, I believe that this new motion stuff will have a profound influence and be incorporated more and more into everything.

Matt Barton wrote:

I do love the special seats that give you a more visceral feedback. I didn't like the vibrating controllers very much, but moving that tech to the chair does make a noticeable difference. Of course, the costs are much too high for consumers, but who knows what may happen in the future.

The reality is, most controllers today are just rumble controllers, though they're often misnamed as force feedback. I have a true force feedback controller, one from Microsoft that makes use of the old game port and required its own power supply. This is the type of joystick that actually fight you back. You're pulling g's in a virtual aircraft and you need to fight to move the stick in the opposite direction. This tech is not cheap and can have potential reliability problems which is why it's not used more, but that's the kind of stuff that I would have actually really wanted to see supported. Rumble as-is is fine and can have its place - for instance, in Hasbro's family game night on Xbox Live Arcade when its your turn your controller shakes to indicate its time for you to go - it's a concession technology more than anything.

Frankly, we'll see integrated motion tracking long before any type of motion chairs or other similar devices, which have been planned for the home since the mid 80's. The costs are just prohibitive for the latter, not to mention storage space needs and all of the other issues. Again, once profound motion tracking is successfully paired with sonic projection to make us feel the virtual stuff we're interacting with, then we'll have the end-all, be-all technology.

Matt Barton wrote:

I just watched Top Gun for the first time last night (gasp, I know) and now watching the special features. One thing that struck me was how much they talked about how flying a fighter jet is a physical, athletic experience. People have no idea how it actually feels to fly in one of those (the g's, the stomach leaps, etc.) Obviously, no matter how good the graphics get, this a realm of realism that's only possible with some kind of special hardware.

True, but again, it's just a matter of time for a combination of holography, sonic projection and this motion/technology we're talking about - where in theory it could all come from a projector bar - for this to become a reality. It's about removal of physical barriers - the physical is always where the cost is at - and doing as much as you can virtually.

Books!
Bill Loguidice, Managing Director | Armchair Arcade, Inc.

n/a
Bill Loguidice
Bill Loguidice's picture
Offline
Joined: 12/31/1969
Noah Falstein, an industry

Noah Falstein, an industry pro who we interviewed for our upcoming feature film videogame documentary, commented on this tech in a new blog post as well: http://www.gamasutra.com/blogs/NoahFalstein/20090605/1122/What_Use_Is_A_...

Books!
Bill Loguidice, Managing Director | Armchair Arcade, Inc.

n/a
Mark Vergeer
Mark Vergeer's picture
Offline
Joined: 01/16/2006
Microsoft Force Feedback flightstick...
Bill Loguidice wrote:

....
The reality is, most controllers today are just rumble controllers, though they're often misnamed as force feedback. I have a true force feedback controller, one from Microsoft that makes use of the old game port and required its own power supply. This is the type of joystick that actually fight you back....

I actually had the same Microsoft Flight Stick, this featured true force feedback and it was great with flight simulator. I believe it actually housed a true 486-cpu and it was pretty high-tech for the time. It doesn't work with Windows XP or Vista I believe.

Xbox 360: Lactobacillus P | Wii: 8151 3435 8469 3138
Armchair arcade Editor | Pixellator | www.markvergeer.nl

n/a
Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Yeah, Noah's comments seem

Yeah, Noah's comments seem pretty apt to me. As far as exergaming is concerned, this stuff is right on the money.

I've been reading a bit about Japanese game development, and there is this big thing about how they conceptualize games from the "controller in," whereas we go from the "controller out." In other words, the Japanese spend a lot of time thinking about the control mechanism and what kind of games can be designed that really take full advantage of it. We're more likely to think of a game first and worry later about how it will be controlled. I think the Japanese way is vital for this new technology. Right now people are seeing only the obvious stuff that's been done before. However, someone who really put some thought into it could probably come up with some "killer app" type stuff that is straight out of left field--not dancing or exergaming, but something else that will make the rest of us go "d'oh! Why didn't I think of that??"

Heck, you would probably do best not even to go to gamers or devs for inspiration.

Hmm...Maybe it could be used for sign language?

n/a
Catatonic
Offline
Joined: 05/20/2006
I have not been impressed by

I have not been impressed by the original Wii controller. For the most part, it just takes regular joypad controls and translates them from your thumbs to your arms. These new controllers look like they might start to deliver something really different - new types of entertainment that can co-exist with videogames as we know them today.

On the other hand, don't listen to Peter Molyneux, he is a major B.S. artist.

Bill Loguidice
Bill Loguidice's picture
Offline
Joined: 12/31/1969
Controlling hype
Catatonic wrote:

I have not been impressed by the original Wii controller. For the most part, it just takes regular joypad controls and translates them from your thumbs to your arms. These new controllers look like they might start to deliver something really different - new types of entertainment that can co-exist with videogames as we know them today.

On the other hand, don't listen to Peter Molyneux, he is a major B.S. artist.

Molyneux tends to overhype, but the reality is he always delivers on at least some of those promises, which is more than can be said for others who don't even try to be overly ambitious and play it safe.

And yes, I agree that the original Wii controller was overhyped as well, but it do something important and that's legitimize another method of control like the iPod/iPhone legitimized touch.

Books!
Bill Loguidice, Managing Director | Armchair Arcade, Inc.

n/a
LarryLaffer (not verified)
This stuff is bad, bad bad.

This stuff is bad, bad bad. i do not want a cemera on me, no thanks. these violate privacy. I do not want to log to facebook and see me karoiking or dancing. Can you imagine? BILL you are dreaming if you think this will become popular. Rocktard and Guitar Idiot will never last, the idea is so dum i don't know why people buy into it. I can't explain it. Just give me a colecovision and I'm happy.

Mark Vergeer
Mark Vergeer's picture
Offline
Joined: 01/16/2006
Laffer?
LarryLaffer wrote:

This stuff is bad, bad bad. i do not want a cemera on me, no thanks. these violate privacy. I do not want to log to facebook and see me karoiking or dancing. Can you imagine? BILL you are dreaming if you think this will become popular. Rocktard and Guitar Idiot will never last, the idea is so dum i don't know why people buy into it. I can't explain it. Just give me a colecovision and I'm happy.

Mr Larry Laffer - how did you manage to escape the Lesuire Suit Larry Universe? Cameras can be turned off and if you value your privacy than don't join any online community - especially not facebook.

Who are you really? Who were you before?

"Karoiking" ? Wow I just learned a new verb. Or perhaps my non-native English brain just isn't up to Karoiking. I wonder why I always get Boo'ed away....

"The ants are my friends, they're blowing in the wind. The ants are blowing in the wind......"

;-)

Xbox 360: Lactobacillus P | Wii: 8151 3435 8469 3138
Armchair arcade Editor | Pixellator | www.markvergeer.nl

n/a
Mark Vergeer
Mark Vergeer's picture
Offline
Joined: 01/16/2006
The Milo demo has struck me as something amazing

Of course heavily scripted (MILO) - I can't imagine an AI being able to interact with a person and hold a a meaningful intelligent spontaneous conversation at the same time at the level that was demoed here. But in the basics this is perhaps were things are going. As a psychiatrist I am very interesting in a thing like Milo. Perhaps with my psychological knowledge of how the mind works I could actually contribute to something like that in a meaningful way - by adding more depth to a personality like that. If something like Milo would cross my path - the opportunity to work on something like that I would go for it instantly! Amazing - the conversation and the fact that the AI is able to recognize emotions is what really draws me in.

On Natal. I think Natal could actually turn out to be better than Nintendo's Wii controller. The various Natal demo-videos out there are of course heavily enhanced and scripted but even the realtime demos look fantastic.

Xbox 360: Lactobacillus P | Wii: 8151 3435 8469 3138
Armchair arcade Editor | Pixellator | www.markvergeer.nl

n/a

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.