Method and apparatus for processing image data from a primary sensor and a secondary sensor

• US 8,396,321 B1
• Filed: 04/24/2008
• Issued: 03/12/2013
• Est. Priority Date: 04/25/2007
• Status: Expired due to Fees

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Inventors: Aldrich, Bradley C.
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US12/109,094
• Time in Patent Office: 1,783 Days
• Field of Search: 382/107, 382/254, 382/275, 348/497
• US Class Current: 382/275
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

Systems and methods are provided for processing image data captured by a device. In one implementation, a system is provided to process primary image data based on motion of the device during generation of the image data, as detected by a secondary image sensor. In another implementation, a method is provided for processing image data by generating primary image data, generating secondary image data, calculating, using the secondary image data, at least one motion vector based on motion of the device during generation of the primary image data, processing the primary image data based on the at least one motion vector, and outputting the processed primary image data.

Claims

1. A method for processing image data captured by a device, the method comprising:

generating primary image data, wherein the primary image data comprises one or more first data frames;

generating secondary image data, wherein the secondary image data comprises a plurality of second data frames corresponding to the one or more first data frames of the primary image data;

embedding a first count value in each of the one or more first data frames, wherein each first count value represents a time stamp;

embedding a second count value in each of the plurality of second data frames, wherein each second count value represents a time stamp;

synchronizing the plurality of second data frames with the one or more first data frames based on (i) the first count value embedded in each of the one or more first data frames and (ii) the second count value embedded in each of the plurality of second data frames;

calculating, using second data frames of the secondary image data and their corresponding first data frames, at least one motion vector based on motion of the device during generation of the primary image data;

processing the primary image data based on the at least one motion vector; and

outputting the processed primary image data.

2. The method of claim 1, further comprising calculating motion of the device based on the plurality of second data frames.

3. The method of claim 1, further comprising converting the at least one motion vector to a spatially varying function.

4. The method of claim 1, further comprising:

compressing the processed primary image data; and

displaying the compressed primary image data.

5. A system for processing image data captured by a device, the system comprising:

a primary image sensor to generate primary image data, wherein the primary image data comprises one or more first data frames, and wherein the primary image sensor is configured to embed a first count value in each of the one or more first data frames, wherein each first count value represents a time stamp;

a secondary image sensor to generate secondary image data, wherein the secondary image data comprises a plurality of second data frames corresponding to the one or more first data frames of the primary image data, and wherein the secondary image sensor is configured to embed a second count value in each of the plurality of second data frames, wherein each second count value represents a time stamp;

a correlation circuit to calculate, using second data frames of the secondary image data and their corresponding first data frames, at least one motion vector based on motion of the device during generation of the primary image data;

a primary interface circuit to receive the primary image data from the primary image sensor;

a secondary interface circuit to receive the secondary image data from the secondary image sensor;

a data processor to process the primary image data based on the at least one motion vector; and

a buffer to provide the plurality of second data frames of the secondary image data from the secondary interface circuit to the correlation circuit,

wherein the primary interface circuit and the secondary interface circuit are configured to synchronize the plurality of second data frames with the one or more first data frames based on (i) the first count value embedded in each of the one or more first data frames and (ii) the second count value embedded in each of the plurality of second data frames.

6. The system of claim 5, further comprising a memory device to store the processed primary image data.

7. The system of claim 5, further comprising a liquid crystal display panel to at least one of (i) preview a scene and (ii) display the processed primary image data.

8. The system of claim 7, further comprising a keypad panel to generate a trigger event to terminate the previewing of the scene, wherein the trigger event initiates generation of primary image data.

9. The system of claim 5, further comprising a transmitter to transmit the processed primary image data.

10. The system of claim 5, wherein the primary image data represents one of (i) a still image or (ii) a video.

11. The system of claim 5, further comprising a kernel estimation circuit to convert the at least one motion vector to a spatially varying function.

12. The system of claim 5, wherein the data processor comprises at least one of (i) a deblurring circuit or (ii) a third interface circuit.

13. The system of claim 5, further comprising a data compressor to compress the processed primary image data.

14. The system of claim 5, wherein the primary image sensor operates in a rolling shutter mode.

15. The system of claim 5, wherein the device is one of (i) a cell phone or (ii) a personal digital assistant.

16. The system of claim 5, wherein:

the primary image sensor is pointed in a first direction; and

the secondary image sensor is pointed in a second direction.