Gesture pre-processing of video stream using a markered region

• US 10,261,596 B2
• Filed: 07/03/2018
• Issued: 04/16/2019
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Burr, Jeremy
• Primary Examiner(s): Seth, Manav
• Application Number:
  US16/026,514
• Publication Number:
  US 20180314340A1
• Time in Patent Office: 287 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Techniques are disclosed for processing a video stream to reduce platform power by employing a stepped and distributed pipeline process, wherein CPU-intensive processing is selectively performed. The techniques are particularly well-suited for hand-based navigational gesture processing. In one example case, for instance, the techniques are implemented in a computer system wherein initial threshold detection (image disturbance) and optionally user presence (hand image) processing components are proximate to or within the system'"'"'s camera, and the camera is located in or proximate to the system'"'"'s primary display. In some cases, image processing and communication of pixel information between various processing stages which lies outside a markered region is suppressed. In some embodiments, the markered region is aligned with, a mouse pad or designated desk area or a user input device such as a keyboard. Pixels evaluated by the system can be limited to a subset of the markered region.

Claims

1. An imaging system, the comprising:

camera circuitry to generate frames of image data, the camera circuitry including sensor circuitry for capturing image data;

a threshold detection stage comprising integrated circuitry configured to determine a change amount between a first frame and a second frame of image data, the integrated circuitry of the threshold detection stage further configured to stop further processing of the second frame by a processing stage subsequent to the threshold detection stage, in response to the change amount failing to satisfy a change threshold; and

a target presence stage subsequent to the threshold detection stage and comprising integrated circuitry configured to, in response to the change amount satisfying the change threshold, determine if the second frame includes a skintone colored pixel, the integrated circuitry of the target presence stage further configured to stop further processing of the second frame by a stage subsequent to the target presence stage, in response to the second frame not including a skintone colored pixel.

2. The system of claim 1, further comprising a gesture processing stage subsequent to the target presence stage, the gesture processing stage comprising integrated circuitry and configured to determine whether or not the second frame includes a specific object.

3. The system of claim 2, wherein the specific object includes a hand.

4. The system of claim 2, wherein the gesture processing stage includes a graphics processor, or a general-purpose processor, or both a graphics processor and a general-purpose processor.

5. The system of claim 1, wherein one or both of the threshold detection stage and the target presence stage is further configured to process image frame data corresponding to a region within a given field of view of the system, the region being a subset of the field of view, thereby excluding from processing image frame data outside the region.

6. The system of claim 5, wherein the region corresponds to a particular portion of each of the first and second images.

7. The system of claim 1, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of the camera circuitry.

8. The system of claim 1, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of the sensor circuitry.

9. The system of claim 1, wherein the system is integrated within a display device.

10. A system for processing image data to identify user gestures captured in the image data, the system comprising:

a first processing stage comprising integrated circuitry to identify an indication of activity in image data, and to stop processing of that image data by a subsequent processing stage if insufficient activity is identified; and

a second processing stage subsequent to the first processing stage and comprising integrated circuitry, the second processing stage configured to identify presence of a user in image data in which sufficient activity has been identified, by determining if that image data includes a skintone colored pixel, and in response to that image data not including a skintone colored pixel, the second processing stage is further configured to stop further processing of that image data by a subsequent processing stage.

11. The system of claim 10 wherein at least one of the first processing stage and the second processing stage is further configured to ignore image data outside a region within a given field of view, the region being a subset of the field of view.

12. The system of claim 11 wherein the subset is one of a pixel-scanline, a partial-pixel-scanline, and a single pixel.

13. The system of claim 10 wherein at least one of the first processing stage and the second processing stage is included in circuitry of at least one of a camera and a display.

14. The system of claim 10 further comprising:

a third processing stage subsequent to the second processing stage and including a graphics processing unit configured to separate pixels of interest from pixels that are not of interest in image data in which presence of a user has been identified.

15. The system of claim 14 further comprising:

a fourth processing stage subsequent to the third processing stage and including a central processing unit, the fourth processing stage configured to perform at least one of spatial and temporal interpretation of image data in which presence of a user has been identified.

16. The system of claim 15 wherein the at least one of spatial and temporal interpretation is selectively performed on image data corresponding to a select region within a given overall field of view, the image data corresponding to select region being a smaller subset of image data than that corresponding to the overall field of view, such that image data outside that select region is not subjected to said at least one of spatial and temporal interpretation.

17. The system of claim 10 wherein the second processing stage is configured to identify presence of a user in image data in which sufficient activity has been identified by converting data of that image data into one or more HSV pixels, and if one or more skintone colored pixels are found, then data of that image data is passed to a subsequent segmentation stage, which is implemented in a graphics processing unit and configured to separate pixels of interest from pixels that are not of interest.

18. The system of claim 17 wherein pixels of interest are represented as blobs and pixels not of interest are represented by non-blobs.

19. A camera comprising:

sensor circuitry to capture image data within a field of view;

a first processing stage comprising integrated circuitry to identify an indication of activity in the image data, and to stop processing of the image data by a subsequent processing stage if insufficient activity is identified; and

a second processing stage subsequent to the first processing stage and comprising integrated circuitry, the second processing stage configured to identify presence of a user in image data in which sufficient activity has been identified, by determining if that image data includes a skintone colored pixel, and in response to that image data not including a skintone colored pixel, the second processing stage is further configured to stop further processing of that image data by a subsequent processing stage;

wherein at least one of the first processing stage and the second processing stage is further configured to ignore image data outside a region within the field of view, the region being a subset of the field of view.

20. The camera of claim 19 wherein at least one of the first processing stage and the second processing stage is included in the sensor circuitry.