UW News

September 12, 2000

Readers become part of the action through high-tech mixture of traditional storytelling and virtual reality in UW’s ‘Magic Book’

Imagine reading a biology textbook and coming to a chapter on frogs. The illustrations on the page are helpful, but the flat images can’t communicate the sort of information that would come from watching a real frog.

Now imagine donning a pair of specialized glasses. You can still see the book and your surroundings, but suddenly a 3-D image leaps off the page and floats above the book. You can rotate the book to see detail from different angles and the image moves like a real frog. But you want to know more. So you “leap” into a scene in the book. The outside world disappears while you explore a pond that the frog calls home, or even fly into the frog’s mouth for an inside look at its gullet.

Sound like magic? That’s exactly the term University of Washington researcher Mark Billinghurst and his colleagues are using to describe their latest project, dubbed “Magic Book.”

Magic Book was featured in demonstrations of emerging technology at the recent SIGGRAPH 2000 conference, the Association of Computing Machinery’s international computer graphics meeting that attracts more than 50,000 participants from academia and industry. The response was enthusiastic, Billinghurst said.

“People from Disney were interested, as were some from Universal Studios, and several training companies and museums,” he said. “There are a lot of very innovative and useful things you could do with this.”

A Magic Book looks like a traditional book – it has text and colorful pictures. But look at it through a lightweight viewer and moving, three-dimensional images jump off the page. People can read the book together, interacting as they read the story and simultaneously viewing the virtual images from different angles.

But the experience doesn’t end there. With the turn of a switch, readers can “fly” into a scene to become part of the action. If more than one person steps into the story, participants can see one another as “avatars,” characters that represent people in the virtual world.

“What we’ve done is create a continuum that people can move along, from the real world to a combination of real and virtual, to an entirely immersive virtual experience,” said Billinghurst, a doctoral student in electrical engineering who studies virtual reality in the UW’s “HIT Lab” – short for Human Interface Technology Laboratory. “What makes our technology so unusual is the ease with which people can move from one point to the other. It’s quite seamless.”

The applications for Magic Book range from storybooks to technical manuals and textbooks.

In terms of storybooks, the HIT Lab and partners ATR MIC Labs in Japan and Hirokazu Kato of Hiroshima City University have so far put together about eight books. One features the tale of a Japanese princess, a samurai warrior and an evil ninja who steals the princess’ looking glass. As each character is introduced, readers are treated to moving foot-high 3-D images standing above the page. The princess weeps because her mirror is gone. The samurai shows puzzlement because he can’t find his horse. And the ninja goes through combat exercises.

There are frequent invitations to join the story. Readers can enter the princess’ bedroom and help her look for the missing item. They can help the samurai by searching his farm for his lost horse. And they can hunt down the ninja and find the looking glass in a feudal village. In each case, readers simply look around to take in their surroundings. Pressing a button on the viewer provides forward movement.

The technology also works well for technical material, Billinghurst said.

Imagine opening an anatomy textbook to the chapter covering the human heart. Students would be greeted by a detailed, three-dimensional heart, floating above the page. They could examine it from different angles and study it while it beat.

“Then you could switch to an immersive environment and actually enter the heart and go through it like you were a blood corpuscle,” Billinghurst said. “And the whole time you would still be interacting with other students in your study group.”

Here’s how Magic Book melds the virtual and the physical: Each viewer contains a small camera that provides continuous video feedback to a computer of what’s being viewed. Incorporated into the pictures of the book are unique patterns that are recognizable by the computer. When the computer sees a pattern, it projects a corresponding image onto the page. The software continuously calculates the position and orientation of the camera relative to black borders around the pictures so the images appear to stay fixed in relation to the pages of the book. As the book is rotated and tilted, the images move in sync.

When readers enter a virtual scene, tracking switches from visual to a tracker in the viewer. The tracker keeps tabs on head orientation to allow the reader to move smoothly through the virtual surroundings.

Although there’s no set timetable, Billinghurst calls commercialization of the technology “the ultimate goal.”

“It is very conceivable that the HIT Lab could enter into a consultant relationship with some of the groups interested in this,” he said. “A spinoff company is also possible, or an arrangement in which the lab would enter into a licensing agreement with an interested company. We’ll see what happens.”


For more information, contact Billinghurst at (206) 543-5075 or grof@hitl.washington.edu. A Web site about the Magic Book project, which includes digital video clips of the technology in action, can be viewed at www.hitl.washington.edu/magicbook.