Optical motion sensing

• US 20080030586A1
• Filed: 08/07/2006
• Published: 02/07/2008
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies PTE (Broadcom, Inc.)
• Inventors: Helbing, Rene, Brosnan, Michael J., Sachs, Todd
• Granted Patent: US 8,013,895 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

Optical motion sensing systems and methods are described. In one aspect, light from subfields of a scene is focused onto respective capture areas of a focal plane. Successive sets of contemporaneous local images are captured from the focused light. Respective saliency measures are derived from respective ones of the local images. Respective local motion measures are determined from comparisons of corresponding ones of the local images in ones of the contemporaneous local image sets. A respective global motion measure is produced for each of the contemporaneous local image sets based on the respective ones of the local motion measures and the respective ones of the saliency measures.

Claims

1. An apparatus, comprising:

a lens system operable to focus light from subfields of a scene onto respective capture areas of a focal plane;

an imaging system having active regions each coincident with a respective one of the capture areas in the focal plane, the active regions being operable to cooperatively capture successive sets of contemporaneous local images from the light focused by the lens system;

a saliency measurement module operable to derive respective saliency measures from respective ones of the local images;

a local motion measurement module operable to determine respective local motion measures from comparisons of corresponding ones of the local images in ones of the contemporaneous local image sets; and

a global motion determination module operable to produce a respective global motion measure for each of the contemporaneous local image sets based on the respective ones of the local motion measures and the respective ones of the saliency measures.

2. The apparatus of claim 1, wherein the lens system comprises a planar array of lenses, each of the lenses being operable to focus the light from a respective one of the subfields onto a respective one of the capture areas.

3. The apparatus of claim 1, wherein the saliency measurement module is operable to derive the saliency measures based on measures of contrast in the respective ones of the local images.

4. The apparatus of claim 1, wherein the local motion measurement module is operable to determine the local motion measures by correlating features across corresponding ones of the local images in successive ones of the contemporaneous local image sets.

5. The apparatus of claim 1, wherein

in response to a determination that all the local motion measures determined for the local images in a given one of the contemporaneous local image sets are substantially the same, the global motion determination module produces the global motion measure for the given contemporaneous local image set from all the local motion measures.

6. The apparatus of claim 1, wherein

in response to a determination that none of the local motion measures determined for the local images in a given one of the contemporaneous local image sets satisfies a zero motion predicate defining one or more criteria for a state of zero relative motion between the imaging system and the scene, the global motion determination module identifies ones of the local motion measures corresponding to a dominant motion for the given contemporaneous local image set and produces the global motion measure for the given contemporaneous image set from the identified ones of the local motion measures.

7. The apparatus of claim 1, wherein

in response to a determination that one or more of the local motion measures determined for one or more corresponding ones the local images in a given one of the contemporaneous local image sets satisfies a zero motion predicate defining one or more criteria for a state of zero relative motion between the imaging system and the scene, the saliency measurement module computes respective ones of the saliency measures from the corresponding ones of the local images.

8. The apparatus of claim 7, wherein

in response to a determination that at least one of the computed saliency measures fails to satisfy a saliency threshold, the global motion determination module produces the global motion measure for the given contemporaneous local image set with a value indicative of a state of zero relative motion between the imaging system and the scene.

9. The apparatus of claim 1, further comprising an image capture system operable to capture images a field of view encompassing the subfields of the scene.

10. A method, comprising:

focusing light from subfields of a scene onto respective capture areas of a focal plane;

capturing successive sets of contemporaneous local images from the focused light;

deriving respective saliency measures from respective ones of the local images;

determining respective local motion measures from comparisons of corresponding ones of the local images in ones of the contemporaneous local image sets; and

producing a respective global motion measure for each of the contemporaneous local image sets based on the respective ones of the local motion measures and the respective ones of the saliency measures.

11. The method of claim 10, wherein the deriving comprises deriving the saliency measures based on measures of contrast in the respective ones of the local images.

12. The method of claim 10, wherein the determining comprises correlating features across corresponding ones of the local images in successive ones of the contemporaneous local image sets.

13. The method of claim 10, wherein

in response to a determination that all the local motion measures determined for the local images in a given one of the contemporaneous local image sets are substantially the same, the producing comprises determining the global motion measure for the given contemporaneous local image set from all the local motion measures.

14. The method of claim 10, wherein

in response to a determination that none of the local motion measures determined for the local images in a given one of the contemporaneous local image sets satisfies a zero motion predicate defining one or more criteria for a state of zero motion relative to the scene, the producing comprises identifying ones of the local motion measures corresponding to a dominant motion for the given contemporaneous local image set and producing the global motion measure for the given contemporaneous image set from the identified ones of the local motion measures.

15. The method of claim 10, wherein

in response to a determination that one or more of the local motion measures determined for one or more corresponding ones the local images in a given one of the contemporaneous local image sets satisfies a zero motion predicate defining one or more criteria for a state of zero motion relative to the scene, the deriving comprises computing respective ones of the saliency measures from the corresponding ones of the local images.

16. The method of claim 15, wherein

in response to a determination that at least one of the computed saliency measures fails to satisfy a saliency threshold, the producing comprises producing the global motion measure for the given contemporaneous local image set with a value indicative of a state of zero motion relative to the scene.

17. The method of claim 10, further comprising capturing images of a field of view encompassing the subfields of the scene.

18. A system comprising, comprising:

means for focusing light from subfields of a scene onto respective capture areas of a focal plane;

means for capturing successive sets of contemporaneous local images from the light focused by the lens system;

means for deriving respective saliency measures from respective ones of the local images;

means for determining respective local motion measures from comparisons of corresponding ones of the local images in ones of the contemporaneous local image sets; and

means for producing a respective global motion measure for each of the contemporaneous local image sets based on the respective ones of the local motion measures and the respective ones of the saliency measures.

19. The method of claim 18, wherein

in response to a determination that one or more of the local motion measures determined for one or more corresponding ones the local images in a given one of the contemporaneous local image sets satisfies a zero motion predicate defining one or more criteria for a state of zero relative motion between the imaging system and the scene, the deriving means computes respective ones of the saliency measures from the corresponding ones of the local images.

20. The method of claim 19, wherein

in response to a determination that at least one of the computed saliency measures fails to satisfy a saliency threshold, the determining means produces the global motion measure for the given contemporaneous local image set with a value indicative of a state of zero relative motion between the imaging system and the scene.