Images could soon be beamed straight to eyeballs

by Jon Marmor | September 1, 1994

Take a good long look at your laptop computer screen. It soon could become a collector’s item. And it will have company. Your TV screen might be a goner, too. Same goes for that silver screen you watch slides on.

Thanks to the eye-popping research coming out of the Human Interface Technology Lab at the University of Washington, those items could end up on the 8­-track-tape pile of innovation.

Until now, when images were created, they had to be projected somewhere: namely, a screen. Those screens required a lot of power, weren’t always clear, and were often too heavy or too small. Some screens weren’t easy to move and cost a lot of money to operate, let alone purchase.

Years back, Tom Furness saw the light. A UW professor of industrial engineering and director of the Human Interface Technology Lab, he came up with an intriguing idea: How about projecting an image, any image, right into your eyeball? And forget those bulky, Star Trek-like virtual reality helmets common today. He envisioned cool-looking eyeglasses that would be inexpensive and require minimal power.

Today, his vision exists—in prototype form—in the bowels of the concrete­-gray Washington Technology Center, and it has scientists, the medical community, Hollywood and many others buzzing.

“What we have done,” explains Richard S. Johnston, director of engineering for the center’s Virtual Retinal Display development team, “is create a display where there is no screen. Think of any other type of display—a television, computer, a slide projector. You need a screen for that.

“A screen takes up space, requires a lot of power, and is flat. With virtual retinal display, we not only eliminate the screen, and the huge amount of power to run it, but we are creating a direct interface of the image and the viewer’s eye.”

Now that’s interfacing.

This new technology not only will create a better picture and cost very little, but it looks to have many far-reaching applications. It could revolutionize the way surgery is performed, the way a person uses a laptop computer, even provide an entertainment experience unknown to humanity.

“This will render much of conventional, flat-panel display technology obsolete,” says Furness, who came to the UW in 1989 to head up the University’s research into computer technology and virtual reality. In other words, the cathode-ray tube could be toast.

And that would be most welcome by UW Surgery Professor Mika Sinanan. He says VRD, as it is known, could have the same impact in medicine as anesthesia and antibiotics.

Compared to X-rays and CT scans, “it promises to have much higher resolution than anything we use now,” says Sinanan. “And the portability will allow us to improve the ergonomics of surgery greatly. Now, we have to look at the TV monitors in the operating room at awkward positions. With VRD, each person on a surgical team could have their own VRD hookups, and get an equally superb image to work with.

“That means you won’t have to crane your neck to see the screen, and surgical instruments could be arranged better. It is very promising.”

And that is just one of the possible benefits of Virtual Retinal Display. Others include:

MEDICAL EDUCATION. Ophthalmology Chair Robert Kalina says it could improve the way eye surgery is taught. “Currently, most surgery is learned by students performing surgery on people under supervision,” he says. “Since most ophthalmological surgery is done under a microscope, it is a pretty artificial environment to begin with. And with VRD, you might be able to create that through imagery and learn through that, and not on humans or animals.”

MEDICAL IMAGING. Images currently derived through magnetic resonance imaging, CT scans and X-rays might be projected directly into the eye of the surgeon, allowing him or her to use them as a map while performing a procedure. No more stopping in the midst of a procedure to review a film. “The information the surgeons require is in the X-ray, but the X-ray is in another place,” Furness says. “That would change.” (One long-range idea of Furness—a science fiction aficionado—is that surgeons could take a “walk” through the insides of a patient, viewing organs from every angle during an operation, by connecting advanced ultrasound scanners to computers.)

COMPUTERS. Hate those fuzzy displays on your laptop computers? Tired of the neckache trying to look at that miniature screen? They could be history. The screen image could be shot directly into your eye. Just plug in a pair of VRD eyeglasses and you are set.

INDUSTRY. Production workers installing intricate parts, such as wiring for a Boeing airplane, could just wear their VRD eyeglasses while they work and use the diagrams projected into their eyes to guide them. No more stopping to review paperwork along the way.

ENTERTAINMENT. Virtual reality-based games could be played easily and become more lifelike, given VRD’s incredibly high resolution. That has caught the attention of entertainment giants like Paramount, Lucasfilm and Disney, the people who brought us Star Trek, Star Wars and animation classics.

“The potential is very exciting,” adds UW Electrical Engineering Professor Yongmin Kim, who serves on the project’s science advisory board. “When it comes to medical approaches, we must be very cautious. There are a lot of clinical applications that sound great—especially image processing—but we must be sure it is the right way to go.”

David Hunter is already sure this is the right way. The Vancouver, B.C., businessman, who answers his cellular telephone calls before the first ring, heads up a small Seattle company called MicroVision. Last year, MicroVision was granted the exclusive license to manufacture and sell VRD technology in exchange for royalties and equity, and a $5.1 million grant to the UW to support VRD research.

That deal is just one of many put together by the UW-based Washington Technology Center (WTC), a state agency created in 1983 to develop partnerships between university-based research and private industry. MicroVision joined the growing list of start-up companies in Washington state that have sprouted to life with the WTC’s help, resulting in more than 150 new jobs.

“A company isn’t going to risk a lot of money on something that isn’t proven,” says Bob Center, the executive director of the WTC. “That’s where we come in. In this case, we proved whether VRD would work or not, and MicroVision came in with the money and interest to get involved.”

Proving whether VRD would work didn’t just impress Canadian investors or Hollywood dream factories. It also showed to the engineering world that Furness—who came to the UW to build the Human Interface Technology Laboratory into “the world’s most advanced facility for coupling humans to machines”—was on to something special, “even though many pooh-poohed it,” says Center.

It was an idea that percolated in Furness’ mind back in the 1980s, when he was still a researcher with the U.S. Air Force, charged with developing advanced jet cockpits. Back then, he was leading a team of scientists, engineers, psychologists and physicians to create a computer system to help fighter pilots overcome a huge problem—being overwhelmed by information during combat.

“In the Vietnam era, we began to find that what made or broke a mission was electronics,” Furness says. “Pilots were overwhelmed by the information being fed to them by the increasingly computerized devices that were supposed to help them. They complained that under high-stress conditions their brains were oozing out of their fingertips.”

Thus, he created something called a “super cockpit,” which, through bulky headgear, created a synthetic but three-dimensional representation of the world outside the aircraft.

Through his love of science fiction, Furness has long set a goal on smoothing out the inherently clumsy and inarticulate way computers and humans communicate.

“Humans are basically spatial beings,” Furness says. “We touch things, hear things and see things in three dimensions.” Unfortunately, however, most computers provide information on a flat screen. It’s like “a two-dimensional peephole into a three-dimensional world,” he says.

When he first heard of Furness’ idea for making an eyeglasses-sized device to shoot a beam into the eye, Joel Kallin was skeptical. Kallin is an electrical engineer and optics wizard with an impressive background in holography at MIT. In 1990 Furness asked him to see if he could turn the idea into working hardware.

The main obstacle, as Kallin saw it, was “eye tracking.” Kallin, a research engineer whose long curly brown hair is tied into a ponytail reaching halfway down his back, feared that if someone moved his or her eyes, the image being sent into the eyes wouldn’t remain stable.

After months of trying different configurations, he discovered that as long as the image could be set at infinity, and as long as the light beam carrying the image entered the eye, it would remain stable. “Right then, I knew it would work,” Kallin says.

In the summer of 1991, he ordered $10,000 worth of optical parts, and by early 1992, the first crude prototype was built.

Since then, progress has come at quite a clip, thanks to the VRD development team, which was recently put together. It is led by Johnston, 38, a Georgia Tech-educated electrical engineer who spent six years at Boeing and another 10 years with high-tech start-up companies. Under Johnston are four full-time hardware engineers, a half-time software expert and several graduate students from the UW’s electrical engineering program.

The technology is patented, and better still, the VRD team has the luxury of knowing that at least for the next four years, its work will be completely underwritten. In return for its investment, MicroVision stands to become “the Intel of virtual worlds,” Furness says, referring to the Silicon Valley company that dominates the computer chip world. MicroVision hopes to have a working monochrome unit available within a year. More advanced versions are still some years off.

Those advances include creating the electronic image in color, in stereoscopic vision, and improving the current field of view. Kallin, the team’s resident optics expert, is also looking into developing a wider field of vision “to feel immersed in the environment,” he says.

“If the field of vision is too small, it is kind of annoying, as if you are trying to look through a knothole. We hope to create a field of vision of about 100 degrees, to make you feel like you are in your environment. If you can act and not think about it, then virtual reality is working.”

Right now, the prototype projects a monochrome image: a three-dimensional model of a rotating Space Needle with seagulls flapping around it. It is projected at a resolution of 900 lines by 1024 lines, somewhat higher than high-resolution computer displays. Eventually, the color images will be projected at resolutions of 3000 by 4000 lines—far superior to high-definition TV.

Even now, in its early stages, it is drawing look-sees from many quarters, including the UW Department of Ophthalmology. Kalina, the ophthalmology chair, recently met with Furness and the VRD engineers to “get to know each other better.”

“They are not biomedical people and don’t know how the eye works,” Kalina says. “And we are not engineers. By working together, we can share ideas to see how we can make the most of it.”

Virtual reality research on the medical front isn’t new. The Medical College of Georgia, for instance, is conducting research on using virtual reality techniques with ophthalmologic surgery. “It is still with a large virtual reality helmet, and when I saw it a few months ago, it was still a long way off from being useful,” Kalina says. “The HIT lab’s idea of using the retina as a screen is unique.”

Eye specialists are excited by the promise of VRD as a tool to aid people with poor vision. During a recent demonstration of the apparatus, a scientific adviser stopped by and peered into the viewfinder, which now resembles half-drill press and half-erector set. The adviser, who has no vision in one eye, saw the image perfectly with his bad eye.

Kalina, who says he learned of this surprising find when he read a newspaper account, explains that it makes perfect sense. Many people, especially the elderly, suffer from macular degeneration, where the sensitive part of the retina deteriorates. That curtails the ability to see small objects, forcing many older people to use a magnifying glass.

“When the image is directed into the eye, it allows for perfect viewing,” says Kalina, who hopes to add VRD to other computer devices to help people regain vision.

VRD isn’t dangerous to the eye, either, because of the low power it uses. “This could be a big help,” says Kalina.

The project is getting plenty of help from MicroVision, which raised $3.5 million in its initial fund-raising effort. Now, Hunter and his partner, Caisey Harlingten, are trying to raise an additional $12 million.

With the financial clout of MicroVision behind them, the VRD folks can concentrate on their project and not worry about funding running out—a common problem in the academic world.

“It is a win-win situation for all of us,” Hunter says. “The UW gets the funding to continue its research into a very exciting project and we get the exclusive right to market the products that will result. The state of Washington wins because new businesses start here. And this is technology that is unsurpassed in the world.”

Fall 1995 is the target date for a VRD product to be available on the market, probably in the medical arena. The once “pooh-poohed” technology now has drawn interest from health-care and electronics giants General Electric, Johnson & Johnson and Siemens, who might want to help out in the production of the product in a south Seattle manufacturing plant.

The cost of such equipment hasn’t been determined yet, but shouldn’t be very high. “Remember, we are talking about a computer chip, and we don’t have to create a screen. That alone is a huge saving of money,” says Hunter.

And this is one instance where, in the not too distant future, the eyes have it.

---

Jon Marmor is associate editor of Columns.