Skip to main content

Dexterous Hand

Overview

Description

This robotic hand control system is designed for precise and dexterous manipulation, featuring six pneumatic cylinders to control five fingers, along with additional actuators for enhanced flexibility and articulation. It includes an intuitive GUI that provides real-time feedback, making the system user-friendly, even for beginners. With multiple communication interfaces and seamless sensor integration, the system is highly versatile for applications in prosthetics, industrial automation, and robotics research. The firmware is compatible with Micro-ROS, enabling efficient communication within ROS 2-based robotic ecosystems.

Building on the base system, the AI vision-enhanced dexterous hand integrates the Renesas Vision AI MPU to enable intelligent gesture-based control. Running on Ubuntu 24.04 with ROS 2 Jazzy, this extension uses camera input to recognize hand gestures and control the robotic hand accordingly. Powered by DRP-AI accelerators, the AI model detects and publishes key points (e.g., landmarks and bounding boxes) as Foxglove image annotations for intuitive visualization. The system supports over 20 gesture commands, including grasp, pinch, thumbs up, call me, peace, and OK, providing a natural interface for human-robot interaction.

System Benefits:

  • Supports multiple communication protocols, including CAN FD, ensuring reliable and high-speed data transfer.
  • Integrates micro-ROS, bringing the advanced capabilities of ROS 2 to the MCU level.
  • The 6-cylinder control mechanism ensures precise and independent movement of each finger, facilitating dexterous hand functions for various applications.
  • Features a modular design with extensive sensor integration capabilities, allowing easy expansion to support advanced functionalities such as adaptive grip, tactile sensing, and environmental interaction.
  • Powered by DRP-AI accelerators, the system achieves real-time object detection, segmentation, and pose estimation.
  • Supports object detection models like YOLOX Pascal VOC, YOLOX Hand, and publishing a bounding box model in the standard ROS 2 message format.
  • Compatible with pose estimation models like MediaPipe, HRNetV2, and RTMPose.

Comparison

Applications

Block Diagram

Select a block to discover products for your design

Available on Lab on the Cloud - Use our PC-based GUI to instantly start configuring and testing.

Launch Lab on the Cloud

Dexterous Hand System

4:3 ratio Sheet.1 Sheet.2 Sheet.3 Sheet.4 WS077-1 WS077-1 WS077-1 Sheet.6 Parallelogram Parallelogram.14 Parallelogram.15 Parallelogram.16 Trapezoid.18 Trapezoid Trapezoid.20 Trapezoid.21 Trapezoid.22 Trapezoid.23 Trapezoid.24 Rectangle Rectangle.26 Semi Circle Sheet.21 Semi Circle.29 Sheet.23 Semi Circle.31 Sheet.25 Semi Circle.33 Sheet.27 Semi Circle.35 Sheet.29 Sheet.30 Parallelogram Parallelogram.41 Parallelogram.42 Sheet.34 Parallelogram Parallelogram.41 Parallelogram.42 Sheet.38 Parallelogram Parallelogram.40 Parallelogram.42 Sheet.42 Parallelogram Parallelogram.41 Parallelogram.42 Sheet.46 Parallelogram Parallelogram.41 Parallelogram.42 Sheet.50 Parallelogram Parallelogram.41 Parallelogram.42 Rectangle Sheet.98 Status LEDs.50 Status LEDs.75 Status LEDs.76 Status LEDs.77 Status LEDs.83 Status LEDs.84 Status LEDs.85 Status LEDs.86 Sheet.107 Status LEDs.50 Status LEDs.75 Status LEDs.76 Status LEDs.77 Status LEDs.83 Status LEDs.84 Status LEDs.85 Status LEDs.86 Connector dotted line.1133 Connector dotted line.120 Connector dotted line.121 Connector dotted line.122 Connector dotted line.123 Connector dotted line.124 Bi-directional.125 Bi-directional.126 Bi-directional Bi-directional.128 Bi-directional.129 Bi-directional.131 Connector Arrow.135 Connector Arrow 24V Input 24V Input 24V Input 3.3V Output 3.3V Output 3.3V Output Connector Arrow.147 Connector Arrow.148 Vertical Wire Bundle 6 6 6 Sheet.151 Vertical Wire Bundle.153 6 6 6 Sheet.155 Tx Tx Tx Rx Rx Rx USB Plug Sheet.159 Sheet.160 Sheet.161 Sheet.162 Sheet.163 Bi-directional.164 Laptop RS-485 RS-485 RS-485 Bi-directional.170 Vertical Wire Bundle.171 2 2 2 Sheet.173 Pressure Sensor Pressure Sensor Pressure Sensor Bi-directional.175 Vertical Wire Bundle.176 2 2 2 Sheet.178 Micro-ROS Micro-ROS Micro-ROS Bi-directional.181 Vertical Wire Bundle.182 2 2 2 Sheet.184 Extend DI/DO Extend DI/DO Extend DI/DO Bi-directional.186 Connector dotted arrow Graphical User Interface Graphical User Interface Graphical User Interface Communication Interface Communication Interface Communication Interface Sheet.190 MCU MCU MCU UART UART UART VDD VDD VDD USB USB USB UART UART UART I2C I2C I2C CAN FD CAN FD CAN FD LDO LDO LDO
Exiting Interactive Block Diagram

Dexterous Hand System with AI Vision

Web Sheet.1 Sheet.2 Sheet.3 Sheet.4 Connector 2 Connector Arrow.17 24V 24V 24V Connector Arrow.19 3.5V 3.5V 3.5V Connected signals arrow Connector Arrow.22 0.8V 0.8V 0.8V Bi-directional SD Card Sheet.29 Sheet.30 Sheet.31 Sheet.32 Sheet.33 Sheet.34 Sheet.35 Sheet.36 Sheet.37 Sheet.38 Bi-directional.39 Connector Arrow.42 Bi-directional.45 Ethernet PHY x 2 Ethernet PHY x2 Ethernet PHY x2 Bi-directional.51 USB 2.0 x 2 USB 2.0 x2 USB 2.0 x2 Bi-directional.54 RS-485 RS-485 RS-485 Bi-directional.57 USB 3.2 USB 3.2 USB 3.2 Camera (Up to 4) Camera (Up to 4) Camera(Up to 4) Bi-directional.65 Bi-directional.66 Camera Sheet.68 Sheet.69 Connector dotted line Connector Arrow.71 USB communication is optional USB communication is optional USB communication is optional Acquiring Image Data Acquiring Image Data AcquiringImage Data WS077-2 WS077-2 WS077-2 Connector Arrow Vertical Wire Bundle 9 9 9 Sheet.79 Vertical Wire Bundle.80 2 2 2 Sheet.82 Vertical Wire Bundle.83 2 2 2 Sheet.85 Vertical Wire Bundle.86 2 2 2 Sheet.88 Sheet.113 Parallelogram Parallelogram.14 Parallelogram.15 Parallelogram.16 Trapezoid.18 Trapezoid Trapezoid.20 Trapezoid.21 Trapezoid.22 Trapezoid.23 Trapezoid.24 Rectangle Rectangle.26 Semi Circle Sheet.128 Semi Circle.29 Sheet.130 Semi Circle.31 Sheet.132 Semi Circle.33 Sheet.134 Semi Circle.35 Sheet.136 Bi-directional.138 Dexterous Hand Dexterous Hand Dexterous Hand PMIC PMIC PMIC Buck Regulator Buck Regulator Buck Regulator Clock Generator Clock Generator Clock Generator PCIe 2-lane PCIe 2-lane PCIe2-lane Buck Regulator Buck Regulator Buck Regulator DDR DDR DDR Sheet.141 MPU MPU MPU LPDDR4 LPDDR4 LPDDR4 SDIO SDIO SDIO Clock Clock Clock PCIe Gen 3 PCIe Gen 3 PCIe Gen 3 Ethernet Control x 2 Ethernet Control x2 Ethernet Control x2 USB 2.0 Gen 2 x 2 USB 2.0 Gen 2 x2 USB 2.0Gen 2 x2 UART UART UART USB 3.2 Gen 2 x 2 USB 3.2 Gen 2 x2 USB 3.2Gen 2 x2 MIPI CSI-2 x 4 MIPI CSI-2 x4 MIPICSI-2 x4 Power Power Power
Exiting Interactive Block Diagram