MOTION AND IMAGE QUALITY MONITOR

• US 20080267521A1
• Filed: 06/28/2007
• Published: 10/30/2008
• Est. Priority Date: 04/24/2007
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Jacob, Matthias, Gao, Jiang, Pulli, Kari, Schloter, C. Phillip
• Application Number:
  US11/770,452
• Publication Number:
  US 20080267521A1
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal

Abstract

A method, apparatus and computer program product for monitoring the motion and/or image quality associated with a captured image are provided. Various actions may be taken in response to the detected motion and corresponding image quality associated with a captured image. In particular, the motion and image quality monitor may, for example, only allow captured images to be used in conjunction with an image matching application, such as a mobile visual search application, when the image features of the captured image have stabilized (i.e., little to no motion is detected between consecutive frames of the captured image). The detected changes in motion and/or image quality may further be used for energy saving purposes, for example, by switching on or off various applications and/or components operating on the mobile device depending upon the amount of motion detected and/or the quality of the image captured.

Claims

1. A method of monitoring motion and image quality of a captured image, said method comprising:

detecting motion in a captured image; and

taking an action in response to the motion detected, wherein the action comprises stabilizing the captured image prior to using the captured image in an image matching application or conserving power in response to the motion detected exceeding a predetermined threshold.

2. The method of claim 1, wherein detecting motion in a captured image comprises comparing one or more features of two or more consecutive frames of the captured image.

3. The method of claim 2, wherein the features compared are selected from a set of features used for the image matching application.

4. The method of claim 2, wherein comparing one or more features further comprises:

sampling two or more frames of the captured image;

filtering the two or more sampled frames to remove noise;

extracting the one or more features from the sampled frames; and

computing a difference between the extracted features of the sampled frames.

5. The method of claim 4, wherein comparing one or more features further comprises:

dividing respective sampled frames into two or more sub-regions, wherein filtering the two or more sampled frames comprises filtering respective sub-regions of the sampled frames, extracting the one or more features from the sampled frames comprises extracting the one or more features from respective sub-regions of the sampled frames, and computing a difference between the extracted features comprises computing the difference between extracted features for respective sub-regions of the sampled frames;

accumulating the computed difference between extracted features for respective sub-regions; and

integrating the accumulated differences of the two or more sub-regions.

6. The method of claim 1, wherein detecting motion in a captured image comprises:

detecting an amount of acceleration associated with a captured image; and

comparing the detected acceleration to a predetermined threshold.

7. The method of claim 1, wherein stabilizing the captured image prior to using the captured image in an image matching application further comprises:

determining whether the detected motion exceeds a predetermined threshold; and

preventing the captured image from being used by the image matching application where the detected motion exceeds the predetermined threshold.

8. The method of claim 7, wherein the image matching application comprises a mobile visual search application, and wherein preventing the captured image from being used by the image matching application comprises instructing the mobile visual search application not to generate new search results based on the captured image.

9. The method of claim 1, wherein conserving power further comprises turning off one or more components of an electronic device used to capture the image when the motion detected exceeds the predetermined threshold.

10. The method of claim 1, wherein the action is selected from a group consisting of:

when the motion detected exceeds a predetermined threshold, preventing the image matching application from using the captured image, instructing a mobile visual search application not to generate new search results based on the captured image, turning off one or more components of an electronic device used to capture the image, switching from a first application operating on the electronic device to a second application, changing a mode of operation of an application operating on the electronic device, and changing an input method of an application operating on the electronic device.

11. An apparatus for monitoring motion and image quality of a captured image, said apparatus comprising:

a processor; and

a memory in communication with the processor and storing an application executable by the processor, wherein the application is configured, upon execution, to detect motion in a captured image, and cause an action to be taken in response to the motion detected, wherein the action comprises stabilizing the captured image prior to using the captured image in an image matching application or conserving power in response to the motion detected exceeding a predetermined threshold.

12. The apparatus of claim 11, wherein in order to detect motion in a captured image, the application is further configured, upon execution, to compare one or more features of two or more consecutive frames of the captured image.

13. The apparatus of claim 12, wherein in order to compare one or more features of the two or more consecutive frames, the application is further configured, upon execution, to:

sample two or more frames of the captured image;

filter the two or more sampled frames to remove noise;

extract the one or more features from the sampled frames; and

compute a difference between the extracted features of the sampled frames.

14. The apparatus of claim 13, wherein in order to compare one or more features of the two or more consecutive frames, the application is further configured, upon execution, to:

divide respective sampled frames into two or more sub-regions, wherein filtering the two or more sampled frames comprises filtering respective sub-regions of the sampled frames, extracting the one or more features from the sampled frames comprises extracting the one or more features from respective sub-regions of the sampled frames, and computing a difference between the extracted features comprises computing the difference between extracted features for respective sub-regions of the sampled frames;

accumulate the computed difference between extracted features for respective sub-regions; and

integrate the accumulated differences of the two or more sub-regions.

15. The apparatus of claim 11, wherein in order to detect motion in a captured image, the application is further configured, upon execution, to:

detect an amount of acceleration associated with a captured image; and

compare the detected acceleration to a predetermined threshold.

16. The apparatus of claim 11, wherein in order to cause the captured image to be stabilized prior to using the captured image in an image matching application, the application is further configured, upon execution, to:

determine whether the detected motion exceeds a predetermined threshold; and

prevent the captured image from being used by the image matching application where the detected motion exceeds the predetermined threshold.

17. The apparatus of claim 16, wherein the image matching application comprises a mobile visual search application, and wherein in order to prevent the captured image from being used by the image matching application, the application is further configured, upon execution, to instruct the mobile visual search application not to generate new search results based on the captured image.

18. The apparatus of claim 11, wherein the image matching application is stored on the memory and executable by the processor.

19. The apparatus of claim 11, wherein in order to cause power to be conserved, the application is further configured, upon execution, to turn off one or more components of the apparatus when the motion detected exceeds the predetermined threshold.

20. The apparatus of claim 11 further comprising:

a camera module operable by the processor and configured to capture one or more images, said memory configured to store the one or more captured images.

21. A computer program product for monitoring motion and image quality of a captured image, wherein the computer program product comprises at least one computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for detecting motion in a captured image; and

a second executable portion for causing an action to be taken in response to the motion detected, wherein the action comprises stabilizing the captured image prior to using the captured image in an image matching application or conserving power in response to the motion detected exceeding a predetermined threshold.

22. The computer program product of claim 21, wherein the first executable portion is configured to compare one or more features of two or more consecutive frames of the captured image.

23. The computer program product of claim 22, wherein in order to compare one or more features of two or more consecutive frames, the first executable portion is further configured to:

sample two or more frames of the captured image;

filter the two or more sampled frames to remove noise;

extract the one or more features from the sampled frames; and

compute a difference between the extracted features of the sampled frames.

24. The computer program product of claim 23, wherein in order to compare one or more features of two or more consecutive frames, the first executable portion is further configured to:

divide respective sampled frames into two or more sub-regions, wherein filtering the two or more sampled frames comprises filtering respective sub-regions of the sampled frames, extracting the one or more features from the sampled frames comprises extracting the one or more features from respective sub-regions of the sampled frames, and computing a difference between the extracted features comprises computing the difference between extracted features for respective sub-regions of the sampled frames;

accumulate the computed difference between extracted features for respective sub-regions; and

integrate the accumulated differences of the two or more sub-regions.

25. The computer program product of claim 21, wherein the first executable portion is configured to:

detect an amount of acceleration associated with a captured image; and

compare the detected acceleration to a predetermined threshold.

26. The computer program product of claim 21, wherein in order to cause the captured image to be stabilized prior to using the captured image in an image matching application, the second executable portion is further configured to:

determine whether the detected motion exceeds a predetermined threshold; and

prevent the captured image from being used by the image matching application where the detected motion exceeds the predetermined threshold.

27. The computer program product of claim 26, wherein the image matching application comprises a mobile visual search application, and wherein in order to prevent the captured image from being used by the image matching application, the second executable portion is further configured to instruct the mobile visual search application not to generate new search results based on the captured image.

28. The computer program product of claim 21, wherein in order to cause the power to be conserved, the second executable portion is further configured to turn off one or more components of an electronic device used to capture the image when the motion detected exceeds the predetermined threshold.

29. An apparatus for monitoring motion and image quality of a captured image, said apparatus comprising:

means for detecting motion in a captured image; and

means for taking an action in response to the motion detected, wherein the action comprises stabilizing the captured image prior to using the captured image in an image matching application or conserving power in response to the motion detected exceeding a predetermined threshold.