Method and apparatus for processing image data from a primary sensor and a secondary sensor

• US 8,625,926 B1
• Filed: 03/11/2013
• Issued: 01/07/2014
• Est. Priority Date: 04/25/2007
• Status: Active Grant

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Inventors: Aldrich, Bradley C.
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US13/792,858
• Time in Patent Office: 302 Days
• Field of Search: 382/107, 382/254, 382/275, 348/497
• US Class Current: 382/275
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

Systems and methods are provided for processing image data captured by a device. In one implementation, a system is provided to process primary image data based on motion of the device during generation of the image data, as detected by a secondary image sensor. In another implementation, a method is provided for processing image data by generating primary image data, generating secondary image data, calculating, using the secondary image data, at least one motion vector based on motion of the device during generation of the primary image data, processing the primary image data based on the at least one motion vector, and outputting the processed primary image data.

Claims

1. A system for stabilizing an image displayed on a device by processing image data captured by the device, the system comprising:

a primary image sensor configured to generate primary image data, wherein the primary image data comprises one or more first data frames;

a secondary image sensor configured to generate secondary image data, wherein the secondary image data comprises a plurality of second data frames (i) corresponding to the one or more first data frames of the primary image data and (ii) synchronized with the one or more first data frames of the primary image data;

a correlation circuit configured to calculate, using (i) second data frames of the plurality of second data frames of the secondary image data and (ii) the second data frames'"'"' corresponding first data frames of the one or more first data frames of the primary image data, at least one motion vector, wherein the at least one motion vector is based on a motion of the device during generation of the primary image data; and

a data processor configured to process the primary image data based on the at least one motion vector in order to stabilize the image.

2. The system of claim 1, further comprising:

a primary interface circuit to receive the primary image data from the primary image sensor; and

a secondary interface circuit to receive the secondary image data from the secondary image sensor;

wherein the primary interface circuit and the secondary interface circuit are configured to synchronize the plurality of second data frames of the secondary image data with the one or more first data frames based on (i) a first count value embedded in each of the one or more first data frames of the primary image data and (ii) a second count value embedded in each of the plurality of second data frames of the secondary image data.

3. The system of claim 1, further comprising a memory device to store the processed primary image data.

4. The system of claim 1, further comprising a liquid crystal display panel configured to at least one of (i) preview a scene and (ii) display the processed primary image data.

5. The system of claim 4, further comprising a keypad panel to generate a trigger event to terminate the previewing of the scene, wherein the trigger event initiates generation of primary image data.

6. The system of claim 1, further comprising a transmitter to transmit the processed primary image data.

7. The system of claim 1, wherein the primary image data represents one of (i) a still image or (ii) a video.

8. The system of claim 1, further comprising a kernel estimation circuit to convert the at least one motion vector to a spatially varying function.

9. The system of claim 1, wherein the data processor comprises at least one of (i) a deblurring circuit or (ii) a third interface circuit.

10. The system of claim 1, further comprising a data compressor to compress the processed primary image data.

11. The system of claim 1, wherein the primary image sensor operates in a rolling shutter mode.

12. The system of claim 1, wherein the device is one of (i) a cell phone or (ii) a personal digital assistant.

13. The system of claim 1, wherein:

the primary image sensor is pointed in a first direction; and

the secondary image sensor is pointed in a second direction.

14. A method for stabilizing an image displayed on a device by processing image data captured by the device, the method comprising:

generating primary image data, wherein the primary image data comprises one or more first data frames;

generating secondary image data, wherein the secondary image data comprises a plurality of second data frames (i) corresponding to the one or more first data frames of the primary image data and (ii) synchronized with the one or more first data frames of the primary image data;

calculating, using (i) second data frames of the plurality of second data frames of the secondary image data and (ii) the second data frames'"'"' corresponding first data frames of the primary image data, at least one motion vector, wherein the at least one motion vector is based on a motion of the device during generation of the primary image data;

based on the at least one motion vector, processing the primary image data in order to stabilize the image; and

outputting the processed primary image data.

15. The method of claim 14, further comprising synchronizing the plurality of second data frames of the secondary image data with the one or more first data frames of the primary image data, wherein the synchronizing is based on (i) a first count value embedded in each of the one or more first data frames of the primary image data and (ii) a second count value embedded in each of the plurality of second data frames of the secondary image data.

16. The method of claim 14, further comprising calculating motion of the device based on the plurality of second data frames.

17. The method of claim 14, further comprising converting the at least one motion vector to a spatially varying function.

18. The method of claim 14, further comprising:

compressing the processed primary image data; and

displaying the compressed primary image data.

19. The method of claim 14, wherein the primary image data represents one of (i) a still image or (ii) a video.

20. The method of claim 14, wherein the device is one of (i) a cell phone or (ii) a personal digital assistant.