Friday, January 18, 2008

Cool Shit Alert: simulating a 3D environment on the Wii


Johnny Chung Lee, a Carnegie Melon student and known Wii-hacking supergenius, has developed something that's not only an amazing technology hack, but simply an amazing concept in general. It strikes me that this would be revolutionary, no matter what technology it was exhibited on... the fact that the Wii makes it easy is just a testament to the versatility of Nintendo's hardware.

Here's the video. Be patient for the first minute and a half, as Johnny zips through a short explanation. Once you get to the demo of the display, it becomes rather mind-blowing.

When we think of 3D displays, we usually think of filtered glasses, allowing a screen to split an image into a "left-eye" version and a "right-eye" version. This creates the illusion of depth as the brain synthesizes the two images. Though it's also a two-dimensional take on 3D, Lee's display is working from a completely different paradigm.

Instead of simulating depth from a fixed point of view, Lee's program is simulating space by adapting a 2-D image to the position of the viewer. This is, in fact, more advanced than the traditional fixed-point approach used in movies like Beowulf. Here, the user can interact with the simulated space by moving around the frame and processing multiple viewing angles.

This could really be the next generation of displays for consoles and simulation. Even from the video, you can tell that it's mind-blowingly immersive, and it promises new heights of simulation and interaction.

I think it's also a testament to technology that this could be developed and disseminated by a man who is essentially an amateur, working with pre-existing tools and an incredibly innovative brain. He didn't need a room full of engineers to sit around and develop this with him, nor a corporate sponsor to give him financial backing and public exposure... he created it and publicized it himself, and nobody can ever take that credit away from him. That's a kind of visionary independence that's never been possible in any other culture or era.

Quickly, I'd like to throw in my own thought for expanding on this innovation. Perhaps somebody else has already suggested this, but I figure I may as well record it for posterity.


Okay, at the end of the segment, Johnny mentions that this will only work for one person at a time. This is because the display has to adapt to the user's position and adjust the image accordingly, and the same image can't accommodate two different points of view simultaneously. I'm not an inventor, and for me this is all speculative, but I have an idea of how to solve this particular problem.

I've heard of a technology that uses interlacing and wavelength filtering (i.e. through filtering glasses) to display two different images on the same screen at the same time. The screen would just interlace two images (image A and image B) that are projected for two different wavelengths, and the users (user A and user B) would each wear a different pair of glasses (glasses A and glasses B). The final result: user A would only see image A, and user B would only see image B. This would be a lovely alternative to split-screen viewing in two-player video games. Both players would be able to use the entire screen to steer their Kart.

Combining this technology with Lee's head-tracking wouldn't be difficult. He himself used a pair of safety goggles to track his head movement. If you just put filtering lenses in two pairs of goggles, you could give each of two users their own individualized content on the same screen. Thus, you could have the same scene, adapting to the positions of two different people at the same time, and you could create two-player games where each player got their own unique 3D experience.

COME ON, NINTENDO!!! DO EET!!! This is the future!

No comments: