Surround view is a vehicle camera system that provides several views like top view, rear view and panorama view to assist the driver when parking, either manually or automatically. It offers the driver the view of the exterior of the vehicle to aid in maneuvering the vehicle to park and to alert the driver to obstacles in its path that may not be immediately visible. A display on the vehicle’s interior control panel shows surround view, which is typically composed of four wide-angle cameras that are mounted in the wing mirrors, at the front and rear of the vehicle.
Currently, surround view provides a view to assist the driver as he or she parks. In more advanced implementations, surround view will be paired with object detection for auto park, where the vehicle assists the driver to move from the traffic lane to a parking area while the driver is still inside the car. The system will further evolve to enable remote park, followed by valet park.
The simplest implementation today is a 360-degree view from above in 2D perspective and is called “top view”. Various camera views are stitched together with the correct geometric alignment. The brightness and color of each cameras’ videos are adjusted to enable a harmonized view.
This top view already assists the driver when parking, and can be enriched with additional information such as 2D overlays or a specific rear view using only the rear camera.
The assistance provided by the top view can be further improved through a 3D view with arbitrary viewpoints. This allows the driver to choose any viewpoint around the car and get a more detailed representation of the car’s surroundings.
It gives the driver a better judgement of the distance from objects when parking. Automated object detection, in addition to the visual representation, will increase safety as it can give automated warnings to the driver. It will also be possible to let the car brake automatically during parking, if an object is too near.
Auto park, remote park and valet park are the next evolutionary steps. Auto park, with the driver still in the car, will provide automated parking based on object recognition, including other cars and pedestrians as well as confining elements like curbs. The following step is remote park, where the driver can stand outside the car while it moves automatically into a tight parking spot such as a garage. Valet park will go beyond this – the driver will be able to leave the car at the entrance of a car park, and the car will find a space autonomously. This, however, will require investment in the infrastructure of the car park, as well as the sensors in the car.
Renesas Electronics has pioneered automotive SoCs with the R-Car automotive computing platform as well as innovative power management devices that support the surround view electronics.
The latest series of automotive SoCs, the R-Car Gen3, is based on the Arm® Cortex®-A57/A53 cores, which use the Arm 64-bit CPU architecture. They provide the performance capability to process the considerable amounts of data from multiple sensors around the vehicle.
There is a trade-off in terms of graphics and computer vision when developing entry level or high-end systems. Entry level systems are typically less demanding, whereas high end systems provide better graphical capability. Most OEMs will want devices that can support both of these key areas and which are scalable. The R-Car Gen3 SoC family ranges from the R-Car V3M, targeting entry level vehicles, to the R-Car H3 for the premium segment.
Targeting the high end of the automotive market, R-Car H3 achieves 40,000 Dhrystone million instructions per second (DMIPS). For best-in-class GPU performance, it also includes the Imagination Technologies PowerVRTM GX6650 3D graphics engine. The R-Car H3 supports the multiple cameras inputs (up to 8) and multiple displays of high end vehicles (up to 4K).
Building on its proven track record in the automotive space, the R-Car Gen3 has an on-chip IMP-X5 parallel programmable engine with advanced image recognition technology. The automotive computer vision hardware is optimized for parking applications that require optical flow and multi-object detection (see Figure 4).
Reducing the cost of the Bill of Materials (BoM) is a significant factor in the competitive automotive market. System integration can result in space and cost savings.
Image Signal Processing (ISP) integration processes the data from the sensor, which – together with on-chip automotive network support for CAN-FD (Controlled Area Network – Flexible Data-rate), Flexray and Ethernet buses – can reduce the total BoM cost. The R-Car Gen3 SoC system architecture introduces functionality and interfaces to the SoC to reduce the system cost. R-Car V3M is the first of R-Car Gen3 that integrates all these features.
For seamless development from PC to embedded systems, migration solutions can support the developer from the pre-development to the final development phase, with research and advanced engineering support, evaluation boards, simulation, or software in the loop, and the final embedded operating system (OS).
Starter kits, such as the H3 Starter Kits available from Renesas Electronics, can be used for tests in-car, and to check algorithms from the sensors to the processing unit. The R-Car H3 and M3 Starter Kit for the R-Car H3 SoC has both OpenCL and OpenGLES 3.1 support, as well as a powerful computing library.
Renesas delivers solutions for ADAS and Autonomous Driving to drive the future of mobility.