DETECTING VISUAL GESTURAL PATTERNS

• US 20090080526A1
• Filed: 09/24/2007
• Published: 03/26/2009
• Est. Priority Date: 09/24/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yang, Ting-yi, Hua, Gang, Vasireddy, Srinath, Chub, Sergey
• Granted Patent: US 8,144,780 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search

Abstract

A processing device and method are provided for capturing images, via an image-capturing component of a processing device, and determining a motion of the processing device. An adaptive search center technique may be employed to determine a search center with respect to multiple equal-sized regions of an image frame, based on previously estimated motion vectors. One of several fast block matching methods may be used, based on one or more conditions, to match a block of pixels of one image frame with a second block of pixels of a second image. Upon matching blocks of pixels, motion vectors of the multiple equal-sized regions may be estimated. The motion may be determined, based on the estimated motion vectors, and an associated action may be performed. Various embodiments may implement techniques to distinguish motion blur from de-focus blur and to determine a change in lighting condition.

Claims

1. A machine-implemented method performed on a portable processing device including an image-capturing component, the machine-implemented method comprising:

capturing a first image frame via the image-capturing component, the captured first image frame having a plurality of regions, each of the plurality of regions having a subset comprising a block of pixels;

capturing a second image frame via the image-capturing component, the captured second image having a plurality of regions;

adaptively searching for a matching block of pixels in each of the plurality of regions of the captured second image frame with respect to a corresponding one of the block of pixels in respective ones of the plurality of regions of the captured first image frame; and

estimating a motion vector with respect to each of the plurality of regions of the captured second image frame based on positions of the matching blocks of pixels with respect to positions of the blocks of pixels in the respective ones of the plurality of regions of the first image frame.

2. The machine-implemented method of claim 1, wherein adaptively searching for a matching block of pixels in each of the plurality of regions of the captured second image frame with respect to a corresponding one of the block of pixels in respective ones of the plurality of regions of the captured first image frame further comprises:

searching for the matching blocks of pixels in each of the plurality of regions of the captured second image frame by searching blocks of pixels based on a previously-estimated motion vector.

3. The machine-implemented method of claim 1, wherein adaptively searching for a matching block of pixels in each of the plurality of regions of the captured second image frame with respect to a corresponding one of the block of pixels in respective ones of the plurality of regions of the captured first image frame further comprises:

selecting one of a plurality of fast block matching methods based on one or more characteristics of the processing device.

4. The machine-implemented method of claim 1, further comprising:

detecting one of a plurality of predefined gestures based, at least partly, on estimated motion vectors with respect to each of a plurality of regions of at least one captured image frame, including the second image frame; and

causing the portable processing device to perform a predefined action associated with the detected one of the plurality of predefined gestures.

5. The machine-implemented method of claim 1, further comprising:

recording a user-defined gesture in a gesture recording mode; and

associating the user-defined gesture with a particular action to be performed by the portable processing device.

6. The machine-implemented method of claim 1, further comprising:

recording a user-defined gesture in a gesture recording mode;

associating the user-defined gesture with a particular action to be performed by the portable processing device;

detecting the user-defined gesture based, at least partly, on estimated motion vectors with respect to each of a plurality of regions of at least one captured image frame, including the second image frame; and

causing the portable processing device to perform the particular action associated with the detected user-defined gesture.

7. The machine-implemented method of claim 1, further comprising:

detecting one of a plurality of predefined gestures based, at least partly, on estimated motion vectors with respect to each of a plurality of regions of at least one captured image frame, including the second image frame; and

causing the portable processing device to perform a predefined action associated with the detected one of the plurality of predefined gestures, wherein;

the one of the plurality of predefined gestures is detected based on at least one motion of the portable processing device as determined by an estimated plurality of motion vectors.

8. A portable processing device comprising:

at least one processor;

an image-capturing component;

at least one storage component connected to the at least one processor and the image-capturing component, some of the at least one storage component comprising;

instructions for detecting one of a plurality of gestures based on at least one detected motion of the portable processing device, as determined by using one of a plurality of adaptive methods for detecting the at least one motion based on images captured by the image-capturing component, at least one of the plurality of gestures including at least one motion from a set of motions comprising panning, tilting, spinning, rotation, flicking, depth-in and depth-out, and

instructions for performing a predefined action associated with the detected one of the plurality of gestures.

9. The portable processing device of claim 8, wherein the instructions for detecting one of a plurality of gestures further comprise:

instructions for determining whether a blurred image is due to defocus blur or motion blur.

10. The portable processing device of claim 8, wherein some of the at least one storage component further comprises:

instructions for selecting the one of the plurality of adaptive methods for detecting the at least one motion based on at least one characteristic of the portable processing device.

11. The portable processing device of claim 8, wherein the instructions for detecting one of a plurality of gestures further comprise:

instructions for capturing a plurality of images via the image-capturing component, each of the images including a plurality of regions,

instructions for adaptively searching for a matching block of pixels in each of the plurality of regions with respect to two of the plurality of images,

instructions for estimating a respective motion vector for each of the plurality of regions based on positions of respective ones of the matching block of pixels, and

instructions for detecting the one of the plurality of gestures based, at least partly, on the estimated respective motion vectors for each of the plurality of regions.

12. The portable processing device of claim 8, wherein some of the at least one storage component further comprises:

instructions for recording a series of motion sequences during a gesture recording mode, and

instructions for associating the recorded series of motion sequences with an action to be performed by the portable processing device.

13. The portable processing device of claim 8, wherein some of the at least one storage component further comprises:

instructions for recording a plurality of motion sequences during a gesture recording mode,

instructions for associating the recorded plurality of motion sequences with an action to be performed by the portable processing device,

instructions for recognizing a second plurality of motion sequences matching the recorded plurality of motion sequences, and

instructions for performing the action when the second plurality of motion sequences is recognized as matching the recorded plurality of motion sequences.

14. The portable processing device of claim 8, wherein at least one storage component further comprises:

instructions for detecting a wiping gesture, and

instructions for performing an action associated with the wiping gesture when the wiping gesture is detected.

15. A tangible machine-readable medium having instructions recorded thereon for at least one processor of a processing device, the instructions comprising:

instructions for detecting one of a plurality of motions based on using an adaptive block matching technique on a plurality of images captured by the processing device;

instructions for recording a sequence of detected motions; and

instructions for associating the sequence of detected motions with an action to be performed by the processing device.

16. The tangible machine-readable medium of claim 15, wherein the instructions for detecting one of a plurality of motions based on using an adaptive block matching technique on a plurality of images captured by the processing device further comprise:

instructions for detecting a change in orientation of the processing device based on using the adaptive block matching technique on a plurality of images captured by the processing device.

17. The tangible machine-readable medium of claim 15, wherein the instructions for detecting one of a plurality of motions based on using an adaptive block matching technique on a plurality of images captured by the processing device further comprise:

instructions for distinguishing motion blur from de-focus blur in the plurality of images.

18. The tangible machine-readable medium of claim 15, wherein the instructions further comprise:

instructions for recognizing an occurrence of a sequence of detected motions matching the recorded sequence of detected motions, and

instructions for performing the action associated with the recorded sequence of detected motions in response to recognizing the occurrence of a sequence of detected motions matching the recorded sequence of detected motions.

19. The tangible machine-readable medium of claim 15, wherein the instructions for detecting one of a plurality of motions based on using an adaptive block matching technique on a plurality of images captured by the processing device further comprise:

instructions for selecting one of a plurality of fast block matching methods based on at least one characteristic of the processing device.

20. The tangible machine-readable medium of claim 15, wherein the instructions further comprise:

instructions for notifying a user of an error when the recorded sequence of detected motions is confusingly similar to a previously-entered recorded sequence of detected motions.