Stanford Researchers Use Lasers to See Around Corners
Stanford University researchers David Lindell and Gordon Wetzstein work on a NLOS experimental setup. Photo by L.A. Cicero/Stanford News Service.
Stanford University researchers have developed a new technique employing lasers to detect objects hidden around corners in foundational work that they hope will enable future autonomous vehicles to 'see' objects that are hidden from the normal vision of a car's occupants or its computerized navigation systems. Their findings were first reported in a 5th March edition of 'Nature' magazine and in Stanford news reports.
Imagine the following scenario: a driverless automobile is making its way following a winding neighborhood street, about to make a sharp turn onto a road where a child's ball has just rolled. Although no person in the car can see that ball, the car stops to avoid it. This occurs because the car is outfitted with extremely sensitive laser technology that reflects off nearby objects to see around corners.
This scenario is one of many that researchers at Stanford University are imagining for a system that can produce images of objects hidden from view. They are focused on applications for autonomous vehicles, some of which already have similar laser-based systems for detecting objects around a vehicle, but other uses could include seeing through foliage from aerial vehicles or giving rescue teams the ability to find people blocked from view by walls and rubble.
"It sounds like magic but the idea of non-line-of-sight imaging is actually feasible," said Gordon Wetzstein , assistant professor of electrical engineering and senior author of the paper describing their work.
Seeing the unseen
"A substantial challenge in non-line-of-sight imaging is figuring out an efficient way to recover the 3-D structure of the hidden object from the noisy measurements," said David Lindell, graduate student in the Stanford Computational Imaging Lab and co-author of the paper. "I think the big impact of this method is how computationally efficient it is."
For their system, the researchers set a laser next to a highly sensitive photon detector, which can record even a single particle of light. They shoot pulses of laser light at a wall and, invisible to the human eye, those pulses bounce off objects around the corner and bounce back to the wall and to the detector. Currently, this scan can take from two minutes to an hour, depending on conditions such as lighting and the reflectivity of the hidden object.
Once the scan is finished, an algorithm untangles the paths of the captured photons and, much like the mythical image enhancement technology featured in television crime shows, the blurry blob takes much sharper form. But thanks to the work of Lindell and Wetzstein, images do actually improve to considerable degrees, and this transformation occurs much more quickly than with existing systems. According to the researchers, resolution improves in less than a second; the algorithm is so efficient it can run on a regular laptop. Based on how well the algorithm currently functions, the researchers think they could speed it up so that it is nearly instantaneous once the scan is complete.
The team is continuing to work on its system so it can better handle the variability of real world situations and complete the scan more quickly. For example, the distance to the object and amount of ambient light present at the time of a scan can make it difficult for their technology to see the light particles it needs to resolve out-of-sight objects. This technique also depends on analyzing scattered light particles that are intentionally ignored by LiDAR technology that is currently seeing limited deployment in automobiles as part of driver assistance system tests.
"We believe the computation algorithm is already ready for LiDAR systems," said Matthew O'Toole, a postdoctoral scholar in the Stanford Computational Imaging Lab and co-lead author of the paper. "The key question is if the current hardware of LiDAR systems supports this type of imaging."
Before this system is road ready, it will also have to work better in daylight and with objects in motion, like a bouncing ball or running child. The researchers did test their technique successfully outdoors, but they worked only with indirect light. Their technology did perform particularly well identifying retro-reflective objects, such as safety apparel or traffic signs. The researchers say that if the technology were placed on a car today, that car could easily detect things like road signs, safety vests or road markers obscured from ordinary vision, although it might struggle with a person wearing non-reflective clothing.
"This is a big step forward for our field that will hopefully benefit all of us," said Wetzstein. "In the future, we want to make it even more practical in the "˜wild.'"
