The Highly-Integrated Gesture Recognition Solution from PixArt

Gesture Recognition technology allows humans to communicate and interact with machines (HMI) more intuitively and efficiently without the need for intermediary devices. Driven by the wide public interests in contactless control solutions since the outbreak of COVID-19, the demand for gesture sensing applications – that allow users to control using simple hand gestures without physically touching any devices – has grown considerably.

At present, there are several gesture recognition technology in the market, including capacitive, infrared transceiver, and vision-based recognition. For simple hand movement detection such as swipe left/right, rotation and push, PixArt Imaging Inc. has utilized the infrared image technology and further integrated a wide range of commonly used hand movement gestures in one compact and highly integrated module – the PAJ7620U2/PAG7631J1 products from our Gesture Recognition series.

For solutions that aim to achieve highly intuitive human-machine communication via gestures by identifying rather complex hand/finger movements such as grab, pinch, and finger rotate, the visible-based gesture technology is undoubtedly the most suitable technology; however, the technology comes with a cost. Visible-based gesture technology is by far one of the best methods to achieve multi-finger detection, yet it generally involves running a complex gesture algorithm behind which occupies a lot of the system’s computing resources. Such characteristic makes the processing ability of the system processor critical, thus a higher-end MCU (with a higher cost of course) is usually collocated with the sensor to support the computing load.

More challenges await for the system developers: apart from the sophisticated hardware and optical designs, complicated software codes would need to be implemented and integrated into the system before the product could get to work. Considering all factors, implementing products with visible-based gesture technology could almost feel like a painful adventure.

As one of the industry leaders in gesture recognition, PixArt is not only good at what we do but takes it one step further in making our customers’ design work a whole lot easier. PixArt has worked our way to tackle the technical barrier of vision-based technology by providing multi-finger gesture recognition solutions with the predefined algorithm that can be easily implemented and applied to customers’ desired applications.

A Simple Way to Realize Multi-Finger Gesture Control Function

PixArt’s Multi-Object Gesture Recognition set consists of a Global Shutter sensor (PAC7640LT) with a high frame rate, and a standalone ISP SoC (PAG7661QN) with full algorithmic support to offer our customers a painless journey in end-product development.

Figure 1. PAG7661QN ISP SoC

Built-in with the handshape detection algorithm, circuit and processor, the PAG7661QN ISP SoC serves to recognize and detect sophisticated multi-finger gestures with the minimum computation and software implementation effort. The identified hand, finger, and palm information, as predefined by the built-in algorithm shown in Figure 2, can be directly output via its I2C, SPI, or USB interfaces.

Figure 2. List of the predefined gestures that comes with the PAG7661QN SoC

To take a closer look, the PAG7661QN ISP SoC features the proprietary handshape detection algorithm built-in to analyze the IR image captured from the PAC7640LT sensor. All preprocessed gesture information from PAG7661QN can be directly output through its I2C/SPI/USB1.1 interface. The hardware algorithm circuit of PAG7661QN that takes care of most of the complicated gesture interpretation can not only foster a fast detection response time, but significantly lift the computation load of the system’s MCU to allow more room for the application-related algorithm.

Note that apart from image data, PAG7661QN can also output the processed information (e.g. position, size, and brightness of fingertips and palm) in real-time, allowing system developers to utilize the data and customize it into their desired gesture values. All these efforts are made to ensure that our customers can develop their gesture-controlled applications with maximum flexibility and support.

The advantages of PixArt’s Gesture Recognition solution are summarized in Figure 3.

Figure 3. Advantages of the Gesture Recognition Solution Provided by PixArt Imaging Inc.

Block Diagram of Evaluation Kit

Figure 4 shows the block diagram of the PAG7661QN evaluation kit. This kit demonstrates the features of the PAC7640LT+PAG7661QN products together, to provide a complete turnkey, in-air gesture solution. The evaluation kit contains a PAC7640LT sensor with a matching lens (FOV(H) = 78 degrees), a PAG7661QN chip, and 2 LEDs. It is powered by a 5V power supply which means it can be easily plugged into a USB port for power supply. With this configuration, the operating distance of the kit is between 5cm and 40cm. The communication interface of the kit can either be I2C, SPI, or USB1.1, which can be used to output interpreted gesture results and set up control settings. In addition, this kit can output a 50×50 image through USB1.1 UVC mode. Complete reference design guide and sample code will also be provided by PixArt along with the kit set.

This all-in-one evaluation kit solution is aimed to provide fast prototyping, simple system integration, and initial product feature evaluations. It is made for anyone who is interested to play around, evaluate, and find out the beauty of PixArt’s Gesture Recognition solution with the minimum effort.

Figure 4. Block diagram of the PAG7661QN evaluation kit

Figure 5 shows the top and bottom view of the PAG7661QN evaluation kit respectively. The compact 55.5mm x 10mm size makes it easy to be designed into almost any consumer product, including monitors, toys, interactive display, kitchen, and automotive infotainment system.

Figure 5. 4L PCB (size: 55.5mm x 10mm)


Unlike most of the vision-based gesture recognition sensors in the market, PixArt’s multi-finger gesture recognition products provide highly integrated solutions for gesture-controlled applications and systems that aim to reduce system developers’ efforts in system design (both mechanically and software-wise) and implementation. Not to mention the benefits that our fast-responsive, built-in algorithm could bring when it comes to intuitive controlling and user experience.

With features of small form factor, cost-effectiveness, and low power consumption, PixArt’s Gesture Recognition series can be applied to a wide variety of applications such as interactive display, lighting/volume control, smart home devices, toy, and robotic systems. We are looking forward to seeing more inspirational gesture applications growing out from our powerful Gesture Recognition sensors products.

PixArt also has a series of AEC-Q100 qualified automotive gesture products that can expand the gesture application to the automotive field. Click here to find out more.

Looking for other technologies to realize your HMI application? Click here to find out more about PixArt’s Optical Track Sensor series.

Source link