Recently, the majority of camera sensor input has become 2Mpix (full-HD) or 8Mpix (4K). However, when using high-resolution sensors, it is common problem that there are few choices of sensor modules that support ISP functionality. Also, that module with ISP is more expensive than the module without ISP. Therefore, the MPUs that support ISP are more desirable to manipulate image data and improve image quality.
This use-case does not require an external ISP chip, thus you can reduce the BOM cost of your product. On the other hand, ISP tuning is a time-consuming task because of manual parameters tuning and effectively it adds schedule delays and additional development.
Consider some of the challenges listed below:
- Multi-function implementation challenges
Typically, ISP converts image data and simultaneously adds image improvements:
e.g., WDR that blends two or more images to enhance the image, 2D or 3D noise reduction - Complexity challenges
The number of registers also increases with increased ISP functionality.
In some cases, there were thousands of registers in the ISP.
You must adjust these registers to get the desired image quality. - Sensor-specific performance and availability challenges
The best setting depends on the sensor type because these settings need to consider the unique characteristics of the sensor such as color.
There are also differences in settings depending on the lens, for examples shading and distortion correction.
Even for users who are familiar with ISPs, many challenges need to be overcome and iterations are needed. Therefore, at Renesas, we came up with a "Tuned ISP" concept to simplify the ISP functions of the RZ/V2M MPU to speed up the image improvement process and shorten development time. We use APIs to abstract each ISP function at a higher level without compromising the image quality of the ISP function.
The "Tuned ISP" has the following features:
- Various ISP functions are supported as API.
The several functions such as auto exposure or auto white balance require additional software processing.
The API includes the processing, thus you can realize the total function only by calling the API. - The image quality by default starts at a baseline that is optimum.
From this setting, you can apply the fine adjustments using the parameters of API. - You can use the adjustment tool when you install your product in the field.
This tool also supports many functions such as exposure and distortion correction.
The setting by this tool is referred to by API, but you can change the setting without much parameter tuning or programming.
You can "easily" and "immediately" use the input images from the sensor with preset settings and APIs.
This figure shows the support function of RZ/V2M ISP.
Renesas also provides multimedia-related APIs. These can be easily implemented on encoded video stream after ISP conversion.
To get started, Renesas has selected the number of sensors that are supported on RZ/V2M and APIs we provide. We continue to expand the list of supported sensors by RZ/V2M’s "Tuned ISP".
Please refer to the functionality of API and related software.
- RZ/V2M: The product with high-performance ISP and power efficiency AI inference
- RZ/V2M ISP Support Package: Supports ISP and multimedia API
- Introduction for Image Quality Functions listed the several API result
- RZ/V2M Linux Package: Base software for RZ/V2M
- RZ/V2M DRP-AI Support Package: supports AI inference function on RZ/V2M.