Systems and methods for utilizing activity detection information in relation to image processing

• US 7,791,641 B2
• Filed: 01/30/2003
• Issued: 09/07/2010
• Est. Priority Date: 09/12/2001
• Status: Active Grant

• Current Assignee: Transchip Israel Limited (Samsung Electronics Co. Ltd.)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Ariel, Viktor, Voronov, German
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US10/356,319
• Publication Number:
  US 20030174772A1
• Time in Patent Office: 2,777 Days
• Field of Search: 348/208.1, 348/208.4
• US Class Current: 348/208.1
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/684 performed by controlling the image sensor readout, e.g. by controlling the integration time
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/42 by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
• CPC Class Codes: H04N 25/707 Pixels for event detection
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

Imaging systems and methods for processing images. Various of the imaging systems include a motion detection and/or estimation engine. Information from such a motion engine can be used by one or more of a scene definition engine, a blur reduction engine, an anti-shake engine, and a video compression engine. Various of the methods include processes for accepting motion information from a motion detection and/or estimation engine and performing one or more of the following functions: anti-shake, blur reduction, scene definition, video compression, and power management. In some cases, the various systems and methods can be implemented on a single chip.

Claims

1. An imaging system, the imaging system comprising:

an energy detection element, operable to detect incident energy;

a motion detection element operable to receive a first signal derived from the energy detection element, to detect a motion evident in a scene portrayed by the incident energy, and to generate a motion signal based on the detected motion; and

a processing element operable to receive the motion signal from the motion detection element and to adjust an image output signal based at least in part on the motion signal by operating on a first subset of information comprised by the first signal and not operating on a second subset of the information comprised by the first signal, the first subset of information corresponding to a portion of the scene where motion is detected, the second subset of information corresponding to a portion of the scene where no motion is detected.

2. The imaging system of claim 1, wherein the processing element includes a readout control system.

3. The imaging system of claim 2 further comprising:

a converter, wherein the converter receives a third signal from the energy detection element, and converts the third signal to the first signal.

4. The imaging system of claim 3, wherein the converter includes an analog to digital converter.

5. The imaging system of claim 3, wherein the motion detection element is operable to identify a portion of the scene exhibiting a greatest amount of motion, and generate a motion vector based on the identified scene,

and the readout control system is operable to adjust a gain parameter of the converter based on the generated motion vector.

6. The imaging system of claim 1, wherein the energy detection element includes a CMOS pixel array.

7. The imaging system of claim 5, wherein the readout control system is operable to modify exposure duration associated with the energy detection element based at least on part on the generated motion vector.

8. An imaging system, the imaging system comprising:

an energy detection element, operable to detect incident energy;

a motion detection element operable to receive a first signal derived from the energy detection element, to detect a motion evident in a scene portrayed by the incident energy, and to generate a motion signal based on the detected motion; and

a processing element operable to receive the motion signal from the motion detection element and to adjust an image output signal based at least in part on the motion signal by operating on a first portion of the first signal and not operating on a second portion of the first signal, the first portion of the first signal corresponding to a portion of the scene where motion is detected, the second portion of the first signal corresponding to a portion of the scene where no motion is detected,

wherein the processing element includes an anti-shake processor, the energy detection element is a CMOS pixel array, and the anti-shake processor selects a subset of the CMOS pixel array within the CMOS pixel array based in part on the motion signal.

9. An imaging system, the imaging system comprising:

an energy detection element,. operable to detect incident energy;

a motion detection element operable to receive a first signal derived from the energy detection element, to detect a motion evident in a scene portrayed by the incident energy, and to generate a motion signal based on the detected motion; and

a processing element operable to receive the motion signal from the motion detection element and to adjust an image output signal based at least in part on the motion signal by operating on a first portion of the first signal and not operating on a second portion of the first signal, the first portion of the first signal corresponding to a portion of the scene where motion is detected, the second portion of the first signal corresponding to a portion of the scene where no motion is detected,

wherein the processing element comprises a scene definition processor, and the scene definition processor is operable to initiate an image output based on the motion signal.

10. The imaging system of claim 9, wherein the energy detection element is a CMOS pixel array, and wherein the scene definition processor determines an active portion of the CMOS pixel array based on the motion signal.

11. An imaging system, the imaging system comprising:

an image acquisition system, wherein the image acquisition system generates an image signal;

a motion estimation system, wherein the motion estimation system is operable to estimate motion based on the image signal, and wherein the motion estimation system generates a motion signal; and

a processing system, wherein the processing system generates an output image by operating on a first subset of information comprised by the image signal and not operating on a second subset of the information comprised by the image signal, the first subset of information corresponding to a portion of the image signal where motion is detected, the second subset of information corresponding to a portion of the image signal where no motion is detected,

wherein the processing system includes at least one of an anti-blur processor, an anti-shake processor, and a scene definition processor.

12. The imaging system of claim 11, wherein the image acquisition system includes an image detection array and a converter, the processing system includes the anti-blur processor, the motion estimation system is operable to identify a portion of the image signal exhibiting a greatest amount of motion, and to generate a motion vector based on the identified portion, and the anti-blur processor is operable to modify a gain parameter associated with the converter based on generated motion vector.

13. The imaging system of claim 12, wherein the anti-blur processor is operable to modify exposure duration in the image detection array based on the generated motion vector.

14. The imaging system of claim 11, wherein the imaging system is operable to adjust a frame-rate based on the motion signal.

15. An imaging system, the imaging system comprising:

an image acquisition system, wherein the image acquisition system generates an image signal;

a motion estimation system, wherein the motion estimation system is operable to estimate motion based on the image signal, and to generate a motion signal; and

a processing system operable to receive the motion signal, and to generate an output image by operating on a first subset of information comprised by the image signal and not operating on a second subset of the information comprised by the image signal, the first subset of information corresponding to a portion of the image signal where motion is detected, the second subset of information corresponding to a portion of the image signal where no motion is detected,

wherein the processing system includes at least one of an anti-blur processor, an anti-shake processor, and a scene definition processor, and

wherein the image acquisition system comprises a pixel array, and the scene definition processor is operable to select a portion of the pixel array for processing based on the motion signal.

16. An imaging system, the imaging system comprising:

an image acquisition system, wherein the image acquisition system generates an image signal;

a motion estimation system, wherein the motion estimation system is operable to estimate motion based on the image signal, and wherein the motion estimation system generates a motion signal; and

a processing system, wherein the processing system generates an output image by operating on a first subset of information comprised by the image signal and not operating on a second subset of the information comprised by the image signal, the first subset of information corresponding to a portion of the image signal where motion is detected, the second subset of information corresponding to a portion of the image signal where no motion is detected,

wherein the processing system includes at least one of an anti-blur processor, an anti-shake processor, and a scene definition processor, and

wherein the image acquisition system comprises a pixel array, and the anti-shake processor is operable to move a subset of the pixel array within the pixel array based on the motion signal.

17. A method for processing images, the method comprising:

receiving an image signal;

estimating motion evident in the image signal; and

generating an output signal by processing a first subset of information comprised by the image signal and not processing a second subset of the information comprised by the image signal, the first subset of information corresponding to a portion of the image signal where motion is detected, the second subset of information corresponding to a portion of the image signal where no motion is detected.

19. The method of claim 17, wherein estimating motion includes identifying a portion of the image signal exhibiting a greatest amount of motion, and generating a motion vector based on the identified portion, and

generating the output signal includes modifying a gain parameter associated with the image signal based on the generated motion vector.

20. The method of claim 17, wherein generating the output signal comprises adjusting a frame-rate based on the motion signal.

18. A method for processing images, the method comprising:

receiving an image signal;

estimating motion evident in the image signal; and

generating an output signal by processing a first subset of information comprised by the image signal and not processing a second subset of the information comprised by the image signal, the first subset of information corresponding to a portion of the image signal where motion is detected, the second subset of information corresponding to a portion of the image signal where no motion is detected;

wherein generating the output signal comprises moving a sub-area of image information within a larger area of image information.

21. A method for processing images, the method comprising:

receiving an image signal;

estimating motion evident in the image signal; and

generating an output signal by processing a first portion of the image signal and not processing a second portion of the image signal, the first portion of the image signal corresponding to a portion of the image signal where motion is detected, the second portion of the image signal corresponding to a portion of the image signal where no motion is detected,

wherein processing the first portion of the image signal includes defining a scene that includes an active portion of an image area associated with the image signal.