Gesture pre-processing of video stream using a markered region

• US 10,146,322 B2
• Filed: 03/29/2017
• Issued: 12/04/2018
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Burr, Jeremy
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/473,002
• Publication Number:
  US 20170205893A1
• Time in Patent Office: 615 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Techniques are disclosed for processing a video stream to reduce platform power by employing a stepped and distributed pipeline process, wherein CPU-intensive processing is selectively performed. The techniques are particularly well-suited for hand-based navigational gesture processing. In one example case, for instance, the techniques are implemented in a computer system wherein initial threshold detection (image disturbance) and optionally user presence (hand image) processing components are proximate to or within the system'"'"'s camera, and the camera is located in or proximate to the system'"'"'s primary display. In some cases, image processing and communication of pixel information between various processing stages which lies outside a markered region is suppressed. In some embodiments, the markered region is aligned with, a mouse pad or designated desk area or a user input device such as a keyboard. Pixels evaluated by the system can be limited to a subset of the markered region.

Claims

1. A system for processing a video stream having a number of frames, comprising:

a threshold detection stage comprising integrated circuitry configured to determine a change amount between a first frame and a second frame of the video stream, the integrated circuitry of the threshold detection stage further configured to stop further processing of the second frame by a stage subsequent to the threshold detection stage, in response to the change amount failing to satisfy a change threshold; and

a target presence stage subsequent to the threshold detection stage and comprising integrated circuitry configured to determine if the second frame includes skintone colored pixels, the integrated circuitry of the target presence stage further configured to stop further processing of the second frame by a stage subsequent to the target presence stage, in response to the second frame not including skintone colored pixels.

2. The system of claim 1, further comprising a gesture processing stage subsequent to the target presence stage, the gesture processing stage comprising integrated circuitry and configured to determine whether or not the second frame includes a specific object.

3. The system of claim 2, wherein the specific object is a hand.

4. The system of claim 2, wherein the gesture processing stage includes a graphics processor, or a general purpose processor, or both a graphics processor and a general purpose processor.

5. The system of claim 1, wherein the integrated circuitry of the threshold detection stage, or the integrated circuitry of the target presence stage, or each of the integrated circuitry of the threshold detection stage and the integrated circuitry of the target presence stage is configured to analyze pixels only within a region of interest.

6. The system of claim 5, wherein the second frame comprises an image and the region of interest corresponds to a particular portion of the image.

7. The system of claim 1, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of a camera.

8. An imaging device comprising the system of claim 1 and a camera operatively coupled to the system.

9. The imaging device of claim 8, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of a camera.

10. One or more non-transitory machine-readable mediums encoded with instructions that when executed by one or more processors cause a method for processing a video stream having a number of frames, the method comprising:

determining, at a threshold detection stage, a change amount between a first frame and a second frame of the video stream, and in response to the change amount failing to satisfy a change threshold, stopping further processing of the second frame by a stage subsequent to the threshold detection stage; and

determining, at a target presence stage subsequent to the threshold detection stage, if the second frame includes skintone colored pixels, and in response to the second frame not including skintone colored pixels, stopping further processing of the second frame by a stage subsequent to the target presence stage.

11. The one or more non-transitory machine-readable mediums of claim 10, the method further comprising determining, at a gesture processing stage subsequent to the target presence stage, whether or not the second frame includes a specific object.

12. The one or more non-transitory machine-readable mediums of claim 11, wherein the specific object is a hand.

13. The one or more non-transitory machine-readable mediums of claim 10, the method further comprising analyzing, at the threshold detection stage, or at the target presence stage, or at each of the threshold detection stage and the target presence stage, pixels only within a region of interest.

14. The one or more non-transitory machine-readable mediums of claim 13, wherein the second frame comprises an image and the region of interest corresponds to a particular portion of the image.

15. The one or more non-transitory machine-readable mediums of claim 10, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of a camera.

16. An imaging system comprising the one or more machine-readable mediums of claim 10, a camera, and the one or more processors.

17. The imaging system of claim 16, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of a camera.

18. A system for processing a video stream having a number of frames, comprising:

a threshold detection stage means configured to determine a change amount between a first frame and a second frame of the video stream, the threshold detection stage means further configured to stop further processing of the second frame by a stage subsequent to the threshold detection stage means in response to the change amount failing to satisfy a change threshold; and

a target presence stage means subsequent to the threshold detection stage means and configured to determine if the second frame includes skintone colored pixels, the target presence stage means further configured to stop further processing of the second frame by a stage subsequent to the target presence stage means, in response to the second frame not including skintone colored pixels.

19. The system of claim 18, further comprising a gesture processing stage means subsequent to the target presence stage means and configured to determine whether or not the second frame includes a specific object.

20. The system of claim 19, wherein the specific object is a hand.

21. The system of claim 19, wherein the gesture processing stage means includes a graphics processor, or a general purpose processor, or both a graphics processor and a general purpose processor.

22. The system of claim 18, wherein the threshold detection stage means, or the target presence stage means, or each of the threshold detection stage means and the target presence stage means is configured to analyze pixels only within a region of interest.

23. The system of claim 22, wherein the second frame comprises an image and the region of interest corresponds to a particular portion of the image.

24. The system of claim 18, wherein the threshold detection stage means, or the target presence stage means, or each of the threshold detection stage means and the target presence stage means is part of a camera.

25. An apparatus comprising the system of claim 18 and an imaging means operatively coupled to the system.

26. The apparatus of claim 25, wherein the threshold detection stage means, or the target presence stage means, or each of the threshold detection stage means and the target presence stage means is part of a camera.

27. A method for processing a video stream having a number of frames, the method comprising:

determining, at a threshold detection stage, a change amount between a first frame and a second frame of the video stream, and in response to the change amount failing to satisfy a change threshold, stopping further processing of the second frame by a stage subsequent to the threshold detection stage; and

determining, at a target presence stage subsequent to the threshold detection stage, if the second frame includes skintone colored pixels, and in response to the second frame not including skintone colored pixels, stopping further processing of the second frame by a stage subsequent to the target presence stage.

28. The method of claim 27, the method further comprising determining, at a gesture processing stage subsequent to the target presence stage, whether or not the second frame includes a specific object, wherein the specific object is a hand.

29. The method of claim 27, the method further comprising analyzing, at the threshold detection stage, or at the target presence stage, or at each of the threshold detection stage and the target presence stage, pixels only within a region of interest, wherein the second frame comprises an image and the region of interest corresponds to a particular portion of the image.

30. The method of claim 27, wherein the threshold detection stage, or the target presence stage, or each of the threshold detection stage and the target presence stage is part of a camera.