Dexterous Hand

This robotic hand control system is designed for precise and dexterous manipulation, featuring six pneumatic cylinders to control five fingers, along with additional actuators for enhanced flexibility and articulation. It includes an intuitive GUI that provides real-time feedback, making the system user-friendly, even for beginners. With multiple communication interfaces and seamless sensor integration, the system is highly versatile for applications in prosthetics, industrial automation, and robotics research. The firmware is compatible with Micro-ROS, enabling efficient communication within ROS 2-based robotic ecosystems.

Building on the base system, the AI vision-enhanced dexterous hand integrates the Renesas Vision AI MPU to enable intelligent gesture-based control. Running on Ubuntu 24.04 with ROS 2 Jazzy, this extension uses camera input to recognize hand gestures and control the robotic hand accordingly. Powered by DRP-AI accelerators, the AI model detects and publishes key points (e.g., landmarks and bounding boxes) as Foxglove image annotations for intuitive visualization. The system supports over 20 gesture commands, including grasp, pinch, thumbs up, call me, peace, and OK, providing a natural interface for human-robot interaction.

System Benefits:

• Supports multiple communication protocols, including CAN FD, ensuring reliable and high-speed data transfer.
• Integrates micro-ROS, bringing the advanced capabilities of ROS 2 to the MCU level.
• The 6-cylinder control mechanism ensures precise and independent movement of each finger, facilitating dexterous hand functions for various applications.
• Features a modular design with extensive sensor integration capabilities, allowing easy expansion to support advanced functionalities such as adaptive grip, tactile sensing, and environmental interaction.
• Powered by DRP-AI accelerators, the system achieves real-time object detection, segmentation, and pose estimation.
• Supports object detection models like YOLOX Pascal VOC, YOLOX Hand, and publishing a bounding box model in the standard ROS 2 message format.
• Compatible with pose estimation models like MediaPipe, HRNetV2, and RTMPose.