The automotive smart front camera is a unique sensor when it comes to ADAS and Automated Driving. The big difference to other sensor technologies like Radar, Lidar or Ultrasonic is that a smart camera can identify road markings, traffic signs, and traffic lights. Additionally, it can also recognize objects, provide data that facilitates their identification i.e. as cars or pedestrians, detect their motion path, and calculate their distances. This makes the smart camera a sensor that enables – under good visibility - features like automated emergency braking or provides the information necessary for automatic car steering.
Thus, front cameras play an important role for carmakers in complying with NCAP (New Car Assessment Program) standards, which are instrumental in reducing accidents with other traffic participants i.e. vehicles and pedestrians.
Smart cameras are expected to be used in high volume in future cars, which implies that they should be available at a low cost.
The challenges for a front camera include requirements for high-performance computer vision and high level of functional safety, moreover, at low power consumption. The latter is a stringent requirement, as the small smart camera housing, including the sensor itself and the SoC is mounted next to the windshield. At this position, the camera is exposed to direct sunlight, which raises the temperature in addition to the heat being generated by the components themselves. Therefore, the power dissipation by the components of the smart camera has to be as less as possible.
The V3M and V3H from the Renesas autonomy platformTM are designed for NCAP-compliant smart cameras, providing high-performance computer vision, low power consumption, and high functional safety.
The V3H SoC is optimized for use in stereo front cameras and achieves five times more computer vision performance than its predecessor R-Car V3M.
V3M and V3H implement a top-notch performance computer vision heterogeneous platform using different accelerators: a versatile pipeline engine (IMP) and a computer vision engine (CVE) supplemented by DMA to control memory transfers. Furthermore, both SoC's coming with an integrated IP for convolutional neural network (CNN). This makes the R-Car V3M and V3H suitable for computer vision algorithms, like object detection & classification, deep neural networks, and convolutional neural networks. In addition to the mentioned hardware accelerators, V3H has dedicated IPs for Dense Optical Flow, Dense Stereo Disparity and Object Classification.
The V3M and V3H come with an integrated ISP (Image Signal Processor) that enhances the quality of the raw image from the camera sensor and makes it thereby ready for machine vision. The integration eliminates the need for an external ISP component in the front camera or in the sensor itself. The system has a single DDR3L memory channel, which reduces the cost for memory components compared to other solutions for front cameras.
The V3M integrates a dual Cortex A53 and V3H a quad Cortex A53 for the application software and both have Cortex R7 Lockstep cores to run AUTOSAR.
This reasoned architecture reduces the need for external components and keeps the bill-of-material to a minimum, still offering high performance and ensuring safety at all times.
The V3M and V3H follow the principle of Renesas autonomy: it is an open platform supported by Renesas and its ecosystem of partners. The ecosystem of partners is further increasing and will complement Renesas' offer with additional hardware, operating systems, and partners for computer vision and on the application level. This gives customers the freedom to choose where to start development: based on the differentiating features of Renesas, or using a solution from Renesas or its ecosystem partners.
This choice allows Tier1 to maximize the creation of value in-house, or shorten time to market by working with partners - a new option in the market for smart cameras. This emphasizes the promise given with Renesas autonomy: "You decide what the future of driving will look like," with Renesas giving a choice to the developer.
Renesas Electronics Unveils Open, Innovative and Trusted Renesas Autonomy™ and Cutting Edge Front Camera SOC R-Car V3M