Living Game Worlds day one

Lasy night, being up checking the web and writing about Bruce’s book at 2am meant that I barely woke up in time today to make it to our tour of the very cool stuff being done at Georgia Tech in the games, augmented reality, and interface areas. Boy, am I glad I woke up. Here, take a brief illustrated tour wih me — warning, a bunch of photos may make this slow to open.

I saw so much stuff that it is definitely going to all turn into mush shortly. I didn’t upload any photos of the cool stuff that say, Ian Bogost is doing, but we looked at that. And Ian and the guys behind Façade are working on a game that does justice to the controversies of abortion (they don’t think small here).

I didn’t snap any shots of what I’d call “thematic world generation,” which is procedural worlds built based on what the player has already done, so that every time you play the game you get a coherent world and story based purely on what you’ve done before. And I won’t try to replicate the graphs and discussion surrounding the different algorithms for plausible story building.

I will mention (though didn’t take a picture of it) the very cool procedural platform-game world generator. 🙂 Still very early stages, but a great application of procedurality to a a very practical problem.

All that stuff was part of the quick run down of projects by faculty, grad students, and a few undergrads (I think) in only ONE of the programs here.

Elsewhere, at the Aware House, we got to see all kinds of slightly freaky stuff. The house is wired in spades — there were at least four giant air conditioners outside to keep it from overheating with all the electronics in the house. Cameras were embedded in the ceiling every few feet, and the house was capable of knowing where everyone was, and approximately what they were doing.

One panel on a side table showed a feed of data like the Dow Jones, the weather, traffic, etc, all rendered as a cartoon street scene — or a beach scene, or whateve scene you’d like — meant to be hung on a wall, so you could quickly get a sense of the gestalt of the world that interested you at a glance. The higher the Dow, the more fruit in the store; the better the weather, the clearer the sky. The forecast colder tomorrow? No problem, the store awning turns blue…

There were also flat panels hung above the chimney that served to display a very wide array of feeds selected from among all the students at the College of Computing. These threw up everything from comics and news to recent Netflix rentals.

The real point of the Aware House, however, seemed to be applications towards medical purposes. A touch-table game taught autistic children how to take turns. The camera system offered a circular buffer of the last half hour of activity, so that if there were a medical problem or a behavioral issue with a child, the caregiver could instantly press a keyfob, and have the last thirty minutes of video sent automatically off to medical personnel. My dad is working with remote home monitoring of children with chronic care needs, so this resonated for me.

Yes, the privacy implications are scary… but the benefits were also clear. I kept flashing on David Brin’s Transparent Society, as well as Bruce’s spimes.

The cameras also served to do social datamining of the folks in the house. An activity monitor chopped up the house into a few dozen zones, down to the level of individual couches. You could look at the graph and see how people clustered — or not, and it wouldn’t take much effort to datamine out social network information, attentional data towards specific tasks or events within the house, etc. Even better, though, they had used the data to build abstract art that somehow represented the “social state” of the house in the last five minutes. A lot of the images very much worked as art, in my opinion.

They also had pretty good chocolate chip cookies.

We then went on to a couple of other labs. The next stop did a lot of stuff using tabletop interfaces: using glass pawns to control images on an interactively projected tabletop screen; a tiltable screen that you could roll rendered balls around on; a system that used cameras to detect footprints on a surface and then render them back onto the surface so you could see a history of where people had walked. There was even a “real life Sims,” with RFIDs tagged onto real-world objects, which then reflected into a Sims-like display.

I kept getting distracted by the guy who has been wearing an augmented reality system for a few years now. That eyepiece gives him a heads-up display, and he wears the computer at his hip…

There was also a wonderful system for improving the sign language use of deaf kids, by giving a videogame a sign language input and an embedded tutor to teach you the proper signs to use. We also got to try on an augmented reality headset ourselves — one time was for a greenscreen “pit” that looked like a gaping hole in the floor, leading down to another office on a lower floor, while you had the headset on. Even if you “knew” that it was actually a solid surface, the motor instincts kicked in when you tried to walk across a very narrow plank over the “hole,” and our arms started to windmill… I also got to meet Jay Bolter, he of hypertext fame, during this portion.

For those Façade fans out there, though, perhaps more impressive was the use of augmented reality to replicate that game experience within a real-world duplicate of the apartment from the game. Grace and Trip, the characters you interact with, are projected onto the heads-up display on your headgear, so you see them “in the room,” and then play as you normally would. You speak your words, and someone offstage types them in for you, so you get as close to a true virtual experience as possible.

Will Wright, of course, had to try this out. I know there are some folks out there who don’t get Will,who wonder if he really is a robot sent from the future or something. Well, those folks will likely not be reassured by this photo of Will being reprogrammed while his batteries are recharged…

I didn’t even touch on Irfan Essa‘s amazing displays of video technology… he showed us a hamster running around in an enclosure, and then the process of mining that video to get a synthetic video hamster indistinguishable from the real thing, that would run around as a controllable character, completely photorealistic. Or the nontradtional renderers that did brush strokes in 3d on real video, to stunning effect. Or the awesome video mosaic music video for “Mad, Mad World”… all in all, it was a day full of rather amazing stuff.

For more info on the Living Game Worlds Symposium, check out the website — but don’t try to stop by, because I am told it’s already full up, and standing room only. 🙂

PS, my apologies to the many folks whose name I didn’t quite catch, or failed to write down. 🙁