Gesture pre-processing of video stream using skintone detection

• US 9,292,103 B2
• Filed: 03/13/2013
• Issued: 03/22/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Burr, Jeremy
• Primary Examiner(s): ALUNKAL, THOMAS D
• Application Number:
  US13/801,111
• Publication Number:
  US 20140267042A1
• Time in Patent Office: 1,105 Days
• Field of Search: 345/156, 345/168, 382/118, 382/221, 382/194, 382/165, 382/103, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Techniques are disclosed for processing a video stream to reduce platform power by employing a stepped and distributed pipeline process, wherein CPU-intensive processing is selectively performed. The techniques are particularly well-suited for hand-based navigational gesture processing. In one example case, for instance, the techniques are implemented in a computer system wherein initial threshold detection (image disturbance) and optionally user presence (hand image) processing components are proximate to or within the system'"'"'s camera, and the camera is located in or proximate to the system'"'"'s primary display. In some cases, image processing and communication of pixel information between various processing stages which lies outside a markered region is eliminated. In some embodiments, the markered region is aligned with a mouse pad, desk area, or a user input device (e.g., keyboard). Pixels evaluated by the system can be limited to a subset of markered region. A simplified skintone detection process can be employed.

Claims

1. A system for processing a raw video stream, comprising:

a target presence stage comprising integrated circuitry configured to determine if a current frame of the video stream includes skintone colored pixels by evaluating only pixels included in a thresholding-event region within a markered region that is a subset of a given imaging field of view, and in response to the current frame not including skintone colored pixels, the target presence stage is further configured to stop further processing of the current frame by a subsequent stage; and

a gesture processing stage configured to perform at least one of spatial and temporal interpretation of image data within the markered region for frames indicated by the target presence stage as including skintone;

wherein pixels outside the markered region are not processed by the target presence or gesture processing stages.

2. The system of claim 1 wherein the thresholding-event region is a single pixel scanline, a partial pixel scanline, or a pixel.

3. The system of claim 1 wherein the target presence stage is configured to evaluate only pixels included in the thresholding-event region by summing R, G, and B pixel values of the video stream within the thresholding-event region.

4. The system of claim 3 wherein the thresholding-event region is a single scan line or a partial scan line or a pixel.

5. The system of claim 3 wherein the target presence stage is further configured to compare the summed R-G-B pixel value to a given threshold to determine if the current frame includes skintone colored pixels.

6. The system of claim 3 wherein summing the R-G-B pixel values in the thresholding-event region is used to determine if a change in ratio of skintone relative to background color of the markered region has occurred.

7. The system of claim 1 wherein the target presence stage is configured to evaluate only pixels included in the thresholding-event region by summing only green pixel values of the video stream within the thresholding-event region.

8. The system of claim 7 wherein the thresholding-event region is a single scan line or a partial scan line or a pixel.

9. The system of claim 7 wherein the target presence stage is further configured to compare the summed green pixel value to a given threshold to determine if the current frame includes skintone colored pixels.

10. The system of claim 7 wherein summing only the green pixel values in the thresholding-event region is used to determine if a change in ratio of skintone relative to background color of the markered region has occurred.

11. The system of claim 1 further comprising a threshold detection stage configured to determine if the current frame of the video stream is sufficiently different from a previous frame of the video stream, and in response to the current frame not being sufficiently different than the previous frame, the threshold detection stage is further configured to stop further processing of the current frame.

12. The system of claim 1 wherein the markered region is aligned with a keyboard and is defined by a perimeter of the keyboard.

13. The system of claim 1 wherein the target presence stage is implemented within or proximate to a camera deployable at a first location and having the imaging field of view that includes a user input area, and the gesture processing stage is implemented at one or more locations different from the first location.

14. The system of claim 1 further comprising an application programming interface (API) for operatively coupling one or more of the stages to a computing platform in which the system is configured to operate.

15. The system of claim 1 wherein the target presence stage is further configured to signal a computing platform to wake from a low power consumption mode in response to detecting skintone in the thresholding-event region.

16. A display comprising the system of claim 1 wherein the target presence stage is implemented within sensor circuitry of a camera integrated within the display.

17. A computer navigation method using a raw video stream, comprising:

determining, at a first processing stage, if a current frame of the video stream includes skintone colored pixels by evaluating only pixels included in a thresholding-event region within a markered region that is a subset of a given imaging field of view, and in response to the current frame not including skintone colored pixels, stopping further processing of the current frame by a subsequent stage; and

performing, at one or more subsequent processing stages, at least one of spatial and temporal interpretation of image data within the markered region for frames indicated by the first processing stage as including skintone;

wherein pixels outside the markered region are not processed by the first processing stage or any of the one or more subsequent processing stages.

18. The method of claim 17 wherein the one or more subsequent processing stages include second, third, fourth, fifth, and sixth processing stages, the method further comprising:

determining, at a processing stage prior to the first stage, if the current frame of the video stream is sufficiently different from a previous frame of the video stream, and in response to the current frame not being sufficiently different than the previous frame, stopping further processing of the current frame;

receiving, at the second processing stage, data from the current frame and separating pixels of interest from pixels that are not of interest, wherein pixels of interest are represented as blobs and pixels not of interest are represented by non-blobs;

performing, at the third processing stage, spatial interpretation of blob data received from the second processing stage by mapping the blob data to human hand geometry;

performing, at the fourth processing stage, temporal interpretation of blob data received from the third processing stage to identify a user navigation gesture; and

directing a computing system based on the user navigation gesture.

19. The system of claim 17 wherein the first processing stage is configured to evaluate only pixels included in the thresholding-event region by summing R, G, and B pixel values of the video stream within the thresholding-event region to determine if a change in ratio of skintone relative to background color of the markered region has occurred.

20. The system of claim 17 wherein the first processing stage is configured to evaluate only pixels included in the thresholding-event region by summing only green pixel values of the video stream within the thresholding-event region.

21. A non-transient computer readable medium encoded with instructions that when executed by one or more processors causes a navigation method to be carried out using a raw video stream, the method comprising:

determining, at a first processing stage, if a current frame of the video stream includes skintone colored pixels by evaluating only pixels included in a thresholding-event region within a markered region that is a subset of a given imaging field of view, and in response to the current frame not including skintone colored pixels, stopping further processing of the current frame by a subsequent stage; and

performing, at one or more subsequent processing stages, at least one of spatial and temporal interpretation of image data within the markered region for frames indicated by the first processing stage as including skintone;

wherein pixels outside the markered region are not processed by the processing stages.

22. The computer readable medium of claim 21 wherein the markered region is a user-configurable subset of the given imaging field of view.

23. The computer readable medium of claim 21 wherein the first processing stage is configured to evaluate only pixels included in the thresholding-event region by summing R, G, and B pixel values of the video stream within the thresholding-event region.

24. The computer readable medium of claim 21 wherein the markered region is aligned with a keyboard and is defined by a perimeter of the keyboard.

25. The computer readable medium of claim 21 wherein the first processing stage is configured to evaluate only pixels included in the thresholding-event region by summing only green pixel values of the video stream within the thresholding-event region.