Gesture control system capable of interacting with 3D images

• US 9,405,378 B2
• Filed: 10/07/2014
• Issued: 08/02/2016
• Est. Priority Date: 09/03/2014
• Status: Expired due to Fees

• Current Assignee: Liquid3D Solutions Ltd
• Original Assignee: Liquid3D Solutions Ltd
• Inventors: WANG, JOHNNY PAUL ZHENG-HAO
• Primary Examiner(s): Sitta, Grant
• Application Number:
  US14/509,045
• Publication Number:
  US 20160062471A1
• Time in Patent Office: 665 Days
• Field of Search: 345156-173
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A gesture control system capable of interacting with three-dimensional (3D) images comprises a 3D image display device, a gesture recognition device, and a data processing unit. The 3D image display device displays a 3D image. The gesture recognition device captures a hand(s) and finger(s) image of a user and then calculates a hand(s) and finger(s) coordinate. The hand(s) and finger(s) coordinate is transmitted to the data processing unit and is matched with an image 3-dimensional spatial coordinate for being output via the 3D image display device in order to achieve advantages of interacting with 3D images.

Claims

1. A gesture control system capable of interacting with three-dimensional (3D) images, comprising:

a 3D image display device to display a 3D stereoscopic or auto-stereoscopic image;

a gesture recognition device, comprising a hand(s) and finger(s) image detection module and a hand(s) and finger(s) coordinate tracking module, the hand(s) and finger(s) image detection module being electrically connected to the hand(s) and finger(s) coordinate tracking module, the hand(s) and finger(s) image detection module capturing a hand(s) and finger(s) image of a user, and the hand(s) and finger(s) coordinate tracking module calculating a hand(s) and finger(s) coordinate based on image changes generated by user'"'"'s hand(s) and finger(s) movements; and

a data processing unit electrically connected to the 3D image display device and the gesture recognition device, and comprising a central processing unit and a graphic processing unit, the central processing unit being electrically connected to the graphics processing unit, the central processing unit comprising a built-in software algorithm program, and one of the central process unit and the graphic processing unit comprising a built-in image coordinate software program for establishing an image 3-dimensional spatial coordinate based on an image data to be displayed on the 3D image display device, a sensing functional area being defined by positions and orientations of the gesture recognition device, wherein when user'"'"'s hand(s) and finger(s) move within the sensing functional area, the gesture recognition device captures the hand(s) and finger(s) image of the user, and calculates a hand(s) and finger(s) coordinate based on sensed data of the hand(s) and finger(s) image detection module, the hand(s) and finger(s) coordinate is then transmitted to the data processing unit for calculation and is matched with the image 3-dimensional spatial coordinates by superimposition of the hand(s) and finger(s) coordinate and the image 3-dimensional spatial coordinate for being output via the 3D image display device, and by adjusting a virtual position of the 3D image via changing a depth map of the image data to be displayed according to 3D positions of finger(s) and hand(s) of the user in the real world so that a depth of the image data to be displayed by the 3D image display device in the real world matches z-coordinates of the finger(s) and hand(s) in real world;

wherein the sensing functional area is defined by positions and coordinates of the gesture recognition device, and a size of the sensing functional area varies based on at least one of sensing element types, quantity and deploying configuration.

2. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein the 3D image display device is selected from one of a liquid crystal display, plasma display panel, and organic light-emitting diode display capable of displaying stereoscopic or auto-stereoscopic images.

3. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein the gesture recognition device is selected from one of a 3D stereo camera, a single camera, an infrared sensor, and a time-of-flight sensor.

4. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein the hand(s) and finger(s) image comprises a node graph and a depth graph of the user'"'"'s hand(s) and finger(s) movements.

5. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein the data processing unit is equipped with a data storage module, and the data storage module is electrically connected to the central processing unit and the graphics processing unit.

6. The gesture control system capable of interacting with 3D images as claimed in claim 5, wherein the data storage module comprises a plurality of built-in image data which are image files or graphic files, and the files are subjected to be read by the data processing unit and be output via the 3D image display device.

7. The gesture control system capable of interacting with 3D images as claimed in claim 6, wherein the image files or graphic files are pre-programmed to be displayed in a preset order.

8. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein a subset of the matching hand(s) and finger(s) coordinate and the image 3-dimensional spatial coordinate is connected in software such that the user'"'"'s hand(s) and finger(s) movements dictate motions and orientation changes of the display image of the 3D image display device.

9. The gesture control system capable of interacting with 3D images as claimed in claim 1, wherein the image 3-dimensional spatial coordinate varies based on a size of the 3D image display device.