Image processing scheme

• US 20050104971A1
• Filed: 10/07/2004
• Published: 05/19/2005
• Est. Priority Date: 10/18/2003
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Pilu, Maurizio
• Granted Patent: US 7,548,256 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method for processing an image sequence captured using an image capture device, wherein a captured sequence to be stabilized by estimating motion of the device, abstracting data from the image sequence, stabilizing the image sequence in response to the estimated motion of the device, and forming an output image sequence from the stabilized image sequence and the abstracted data.

Claims

1. A method of processing an image sequence captured by an image capture device, the method comprising:

estimating motion of the device;

abstracting data from the image sequence;

stabilizing the image sequence in response to the estimated motion of the device; and

forming an output image sequence from the stabilized image sequence and the abstracted data.

2. The method as claimed in claim 1, further comprising:

providing at least one model associated with the image sequence; and

determining, on the basis of the at least one model and the image sequence, the data to be abstracted.

3. The method as claimed in claim 2, wherein the at least one model is adapted to model an aspect of the image sequence.

4. The method as claimed in claim 3, wherein determining further comprises performing a probability computation in order to resolve, on the basis of the image sequence and the at least one model, the likelihood that the aspect has occurred in the image sequence.

5. The method as claimed in claim 2, wherein the model is a hidden Markov model.

6. The method as claimed in claim 1, wherein forming the output image sequence further comprises blending the image sequence and the abstracted data using a blending function.

7. The method as claimed in claim 6, wherein the blending function is a Gaussian function.

8. The method as claimed in claim 1, wherein abstracting the data further comprises determining intentional movement of the image capture device.

9. The method as claimed in claim 8, wherein stabilizing the image sequence further comprises extracting the determined intentional movement from the image sequence.

10. The method as claimed in claim 9, wherein forming the output image sequence further comprises introducing the extracted intentional movement back into the output image sequence.

11. The method as claimed in claim 8, wherein determining the intentional movement further comprises comparing the image sequence against a plurality of intentional movement models.

12. The method as claimed in claim 11, wherein determining the plurality of intentional movement models are a plurality of Hidden Markov Models (HMMs).

13. The method as claimed in claim 11, further comprising determining the plurality of intentional movement models from training image data.

14. The method as claimed in claim 13, wherein the training image data is a pre-captured video sequence which includes a number of intentional movements associated with a scene or object of interest.

15. A method of processing an image sequence comprising a plurality of image frames, the method comprising:

estimating relative motion between the image frames;

abstracting data from the image sequence;

stabilizing the image sequence in response to the estimated motion; and

forming an output image sequence from the stabilized image sequence and the abstracted data.

16. The method as claimed in claim 15, further comprising:

providing at least one model associated with the image sequence; and

determining, on the basis of the at least one model and the image sequence, the data to be abstracted.

17. The method as claimed in claim 16, wherein the model is adapted to model an aspect of the image sequence.

18. The method as claimed in claim 17, wherein determining further comprises performing a probability computation in order to resolve, on the basis of the image sequence and the at least one model, the likelihood that the aspect has occurred in the image sequence.

19. The method as claimed in claim 16, wherein the model is a hidden Markov model.

20. The method as claimed in claim 15, wherein forming the output image sequence further comprises blending the image sequence and the abstracted data using a blending function.

21. The method as claimed in claim 20, wherein the blending function is a Gaussian function.

22. A method of processing an image sequence captured by an image capture device, the method comprising:

estimating motion of the image capture device;

abstracting data from the image sequence;

stabilizing the image sequence in response to the estimated motion of the image capture device; and

simultaneously to the stabilizing, forming an output image sequence from the stabilized image sequence and the abstracted data.

23. A computer program product for use with a computer, the computer program product comprising:

a computer useable medium having computer executable program code embodied thereon, wherein the product is operable, in association with the computer, to process an image sequence captured by an image capture device by;

estimating motion of the device;

abstracting data from the image sequence;

stabilizing the image sequence in response to the estimated motion of the device; and

forming an output image sequence from the stabilized image sequence and the abstracted data.

24. An image capture device, comprising:

an image sensor that captures image sequence data;

a motion estimation module that senses motion of the image capture device;

a central processing device (CPU); and

a memory having computer executable program code embodied thereon, wherein upon execution of the code by the CPU, the image sequence data is processed by;

estimating motion of the image capture device based upon information from the motion estimation module;

abstracting data from the image sequence data;

stabilizing the image sequence data in response to the estimated motion of the device; and

forming an output image sequence from the stabilized image sequence data and the abstracted data.

25. The image capture device as claimed in claim 24, wherein the motion estimation module further comprises at least one internal sensor that senses the motion of the image capture device.

26. The image capture device as claimed in claim 24, wherein the motion of the image capture device is inferred from the image sequence data using inter-frame displacements.

27. The image capture device as claimed in claim 24, further comprising at least one Hidden Markov Model (HMM) that determines intentional movement data that is extracted from the image sequence data, and wherein the intentional movement data is introduced back into the output image sequence when the output image sequence is formed.

28. The image capture device as claimed in claim 27, further comprising training image data, wherein the training image data is used to determine the HMM.

29. The image capture device as claimed in claim 28, wherein the training image data is a pre-captured video sequence which includes a number of intentional movements associated with a scene or object of interest.

30. The image capture device as claimed in claim 24, further comprising a look-up table having data corresponding to a predetermined set of intentional movements, wherein a rule based scheme determines intentional movement data that is extracted from the image sequence data, and wherein the intentional movement data is introduced back into the output image sequence when the output image sequence is formed.