Autonomous driving has long been on the to-do list for car manufacturers, but within the last year consumer interest has spurred inquisition in both the engineering community, as well as the legal arena. Google has been operating an autonomous vehicle for more than five years, and companies like NVIDIA are helping feed the innovation monster.
NVIDIA, which is famed for their graphics work and introducing the world to the GPU (graphics processing unit), has become engrossed in the automotive market as companies, like Google, Audi, BMW, and Tesla, are looking to make cars safer, ridding the highways of driver error.
Danny Shapiro, director of the Automotive Division at NVIDIA, explains, "The Google car has been around for several years, basically every major automaker has been working on this, whether publicly or in stealth mode.” Shapiro feels autonomous system size and power-consumption have been the biggest issues keeping it out of the limelight, until recently.
“You used to have thousands of watts being consumed to run PCs and specialized hardware to monitor what is going on inside the car, and interpret what’s coming in off the cameras, LIDAR scanner, or other radar,” he continues. “It’s turned into a bit of an arms race. Google may have helped fuel it, as they’ve been very public on what they’re doing, so everyone feels like they have to start unveiling what they’ve been working on for years.”
NVIDIA used its graphics know-how to drive the screens that began to appear in vehicles – some six years ago – for infotainment, navigation, and control systems. “It’s evolved into digital instrument clusters. Rather than physical buttons, dials, and gauges, they're pixels. We’ve developed an automotive-grade processor that can withstand the harsh operating conditions in a car.”
The company currently has a presence in about 4.5 million vehicles on the road, but Shapiro’s division has more than graphics in mind when developing technology and algorithms for cars.
A Driver-Free Future
The talk of driverless cars in the mainstream media has spurred interest in a division of the automotive industry that has been hard at work for some time. With the introduction of hardware technology, like NVIDIA’s Tegra, car-makers have seen the opportunity to begin capitalizing on research.
“We’ve basically brought this supercomputing power onto a tiny chip with the Tegra K1,” explains Shapiro. “Now it can go from this total experimental phase into serious production.”
According to Shapiro, it could be possible to have autonomous cars on the road now, or at least in the next five years, but a major challenge lies in infrastructure. “You need to take out the randomness from other drivers. So you can have an autonomous vehicle, but that’s not going to prevent somebody from smashing into it from the side of an intersection or rear-ending it.” He feels that there will be some time before the proper infrastructure and communication technologies are in place. “There’s a lot of experimentation happening on that front," he adds.
Giving vehicles the opportunity to speak with infrastructure technology (as well as each other) is essential to making cars completely autonomous. “NVIDIA has a modem product in the car that enables communication on different frequencies,” Shapiro says. “When your car knows what all the other cars around it are doing, can broadcast its intent, that’s where we get to a safer level of full autonomy.”
All About the Power
With the development of the Tegra K1, which was announced at CES 2014 and totes 192 cores, NVIDIA has started integrating a powerful processor into vehicles that sips power (single-digit wattage), rather than guzzling (two to five kilowatts), like the former automotive PCs. “[With the Tegra K1], a full driver assistance system could operate under 100 watts,” says Shapiro. “Maybe even closer to 50 watts… depending on what you’re counting, cameras, and other elements.”
Audi, which NVIDIA has worked closely with, has more than 2 miles (60 lbs) of wire in its vehicles, and some companies have more than 100 processors operating driver assistance controls. Power consumption can be a subjective number, but “it needs to be an order of magnitude less than it used to be, and that’s what we were able to deliver,” says Shapiro.
The NVIDIA GPU helps the vehicle process the massive amounts of data it receives, and Tegra allows any car maker to move from a heavy, energy-consuming computer system to a highly energy efficient supercomputing system in the car.
“Audi’s not saying, ‘we’re going to make a completely autonomous car,’ because it’s a nearly impossible task to say that you’re going to make a car that will not crash. But, they are taking the first steps by stating what the conditions and scenarios are when the car can take over," he explains.
NVIDIA is working with Audi to develop piloted driving or traffic-jam assist. Basically, the car gives the driver an opportunity to opt-in when traffic is heavy. “The car senses the vehicles around it. It will accelerate and brake to maintain a safe distance and the camera is reading the lane markings, so it will steer to stay in the lane,” Shapiro says. At this point, you can take your hands off the wheel and feet off the pedals and the car will drive itself until you stop it or until the traffic dissipates. “The threshold for traffic dissipation is something like 40 mph. So, when traffic is moving at 40 mph, it gives the control back.”
According to Shapiro, this is the first stage to the self-driving car, but the bigger issue is shrouded in the realm of responsibility and liability. “Certainly, there are legal issues, but that’s not to mention, getting people to be comfortable with handing control over to a robot. Trusting computers. People will start to recognize that self-driving elements are better drivers than any person can be.”
Many car manufacturers have put stakes in the ground for full autonomy by 2020 or 2025, but Shapiro remains skeptical. “I think it’s still a ways away from full autonomy, but I think we’re only a few years away from self-driving modes of operation,” he says.
This is where NVIDIA’s GPU experience and chip technology makes its presence known. A significant part of the company is involved in research. “At the core level, rendering technologies researchers are coming up with new ways to do rendering or model how light behaves, which can be leveraged across several industries,” Shapiro explains. “Automotive is definitely a focus for us. We're able to leverage rendering technology that we’ve done in gaming to scientific work we’ve done in medicine to some of the military and defense applications – it all converges in the car.”
NVIDIA’s processors can process data coming from a vehicle’s cameras and LIDAR in real time. The technology enables the cameras on the front of a car to even identify people and vehicles. “We’re running these algorithms that will send signals of the braking system to stop if it detects that there is a person in the road,” Shapiro explains. “It’s vital that the data is processed in real time; it doesn’t make any sense to detect a pedestrian after you’ve run them over.”
Whether we are five or 15 years from autonomously driven vehicles, the challenges of computing power, liability, and driver responsibility will continue to be intertwined. The driverless car is certainly coming, but perhaps, it's already here.