Method and system of immersive sensation enhancement for video sequence displaying

• US 8,295,539 B2
• Filed: 04/03/2008
• Issued: 10/23/2012
• Est. Priority Date: 12/07/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kaida, Xiao, Lee, Ho Young, Hong, Ji Young
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US12/061,921
• Publication Number:
  US 20090147986A1
• Time in Patent Office: 1,664 Days
• Field of Search: 382/100, 382/107, 382/173, 382/270, 382/274
• US Class Current: 382/100
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 5/142 Edging; Contouring
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

An immersive sensation enhancement method and system for video sequence displaying is provided. An immersive sensation enhancement system, the system including: a foreground image identification unit to identify a foreground image for each frame of an input image; a foreground image characteristic extraction unit to extract a characteristic of the foreground image; a contrast threshold identification unit to identify a contrast threshold based on the characteristic of the foreground image; and an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold.

Claims

1. An immersive sensation enhancement system, the system comprising:

a foreground image identification unit to identify a foreground image for each frame of an input image;

a foreground image characteristic extraction unit to extract a characteristic of the foreground image, among a plurality of characteristics;

a contrast threshold identification unit to identify a contrast threshold, based on the extracted characteristic of the foreground image; and

an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image, such that a contrast of the foreground image is enhanced using the identified contrast threshold and the extracted characteristic of the foreground image,

wherein the contrast threshold varies differently with respect to each of the plurality of characteristics of the foreground image.

2. The system of claim 1, wherein the foreground image identification unit comprises:

an information detection unit to detect color information and smoothness information of a background image of the input image;

a boundary detection unit to detect a boundary of the background image based on the color information; and

a boundary smoothing unit to smooth the boundary based on the smoothness information.

3. The system of claim 1, wherein the foreground image characteristic extraction unit comprises:

a position identification unit to identify a position of the foreground image in an entire area of the frame;

an area identification unit to identify an area corresponding to the foreground image in the entire area; and

a lightness and saturation calculation unit to calculate an average lightness and an average saturation of the foreground image.

4. The system of claim 3, wherein the position and the area of the foreground image are calculated by a unit of a degree corresponding to human retinal eccentricity.

6. The system of claim 1, wherein the immersive sensation enhancement processing unit comprises:

a contrast enhancement unit to enhance a contrast of the foreground image based on the contrast threshold and the characteristic of the foreground image.

7. The system of claim 6, wherein the contrast enhancement unit enhances the contrast to an average lightness of the foreground image based on a sigmoidal function.

5. An immersive sensation enhancement system, the system comprising:

a foreground image identification unit to identify a foreground image for each frame of an input image;

a foreground image characteristic extraction unit to extract a characteristic of the foreground image;

a contrast threshold identification unit to identify a contrast threshold based on the characteristic of the foreground image; and

an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold,

wherein the contrast threshold identification unit identifies the contrast threshold based on a position and an area of the foreground image from the characteristic of the foreground image, the position and the area affecting human retinal eccentricity and a reflected retina size.

8. An immersive sensation enhancement system, the system comprising:

a foreground image identification unit to identify a foreground image for each frame of an input image;

a foreground image characteristic extraction unit to extract a characteristic of the foreground image;

a contrast threshold identification unit to identify a contrast threshold based on the characteristic of the foreground image; and

an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold,

wherein the immersive sensation enhancement processing unit comprises;

a saturation enhancement scale identification unit to identify a saturation enhancement scale based on the characteristic of the foreground image; and

a saturation enhancement unit to enhance an average saturation of the inputted foreground image based on the saturation enhancement scale.

9. The system of claim 8, wherein the saturation enhancement scale identification unit identifies the saturation enhancement scale based on a position and an area of the foreground image from the characteristic of the foreground image.

10. The system of claim 9, further comprising:

a color space conversion unit to convert a color space of at least one of the input image and an output image.

11. The system of claim 10, wherein the color space conversion unit processes a conversion between the color space based on Red Green Blue (RGB) and the color space based on a human perceptual attribute, and

the perceptual attribute includes at least one of a lightness, a saturation, and a hue.

12. An immersive sensation enhancement system, the system comprising:

a movement information identification unit to identify movement information of a foreground image between frames of an input image; and

an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image, such that a contrast of the foreground image is enhanced using an identified contrast threshold and the movement information,

wherein the contrast threshold varies differently with respect to the movement information of the foreground image.

13. The system of claim 12, wherein the movement information identification unit comprises:

a position identification unit to identify a position of the foreground image for each frame based on a position of a pixel included in the foreground image; and

an area identification unit to identify an area of the foreground image for each frame based on a ratio of a number of all pixels to a number of pixels of the foreground image,

wherein the position and the area are expressed by a unit of a degree.

15. The system of claim 12, wherein the movement information includes information about a position and an area of the foreground image, and

the immersive sensation enhancement processing unit comprises:

a saturation enhancement scale identification unit to identify a saturation enhancement scale based on the position and the area; and

a saturation enhancement unit to enhance a saturation of the foreground image based on the saturation enhancement scale.

14. An immersive sensation enhancement system, the system comprising:

a movement information identification unit to identify movement information of a foreground image between frames of an input image; and

an immersive sensation enhancement processing unit to process an immersive sensation enhancement of the input image based on the movement information,

wherein the movement information includes information about a position and an area of the foreground image, and

the immersive sensation enhancement processing unit comprises;

a contrast threshold calculation unit to calculate a contrast threshold based on the position and the area; and

a contrast enhancement unit to enhance a contrast to a lightness of the foreground image based on the position, the area, and the contrast threshold.

16. An immersive sensation enhancement method, the method comprising:

identifying, by a processor, a foreground image for each frame of an input image;

extracting a characteristic of the foreground image, among a plurality of characteristics;

identifying a contrast threshold based on the extracted characteristic of the foreground image; and

processing an immersive sensation enhancement of the input image, such that a contrast of the foreground image is enhanced using the identified contrast threshold and the extracted characteristic of the foreground image,

wherein the contrast threshold varies differently with respect to each of the plurality of characteristics of the foreground image.

19. The method of claim 16, wherein the processing comprises enhancing a contrast of the foreground image based on the contrast threshold and the characteristic of the foreground image.

20. The method of claim 19, wherein the enhancing comprises enhancing the contrast to an average lightness of the foreground image based on a sigmoidal function.

22. The method of claim 16, further comprising:

converting a color space of at least one of the input image and an output image,

wherein the converting comprises processing a conversion between the color space based on RGB and the color space based on a human perceptual attribute.

17. An immersive sensation enhancement method, the method comprising:

identifying, by a processor, a foreground image for each frame of an input image;

extracting a characteristic of the foreground image;

identifying a contrast threshold based on the characteristic of the foreground image; and

processing an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold,

wherein the extracting comprises;

identifying a position of the foreground image in an entire area of the frame;

identifying an area corresponding to the foreground image in the entire area; and

calculating an average lightness and an average saturation of the foreground image.

18. An immersive sensation enhancement method, the method comprising:

identifying, by a processor, a foreground image for each frame of an input image;

extracting a characteristic of the foreground image;

identifying a contrast threshold based on the characteristic of the foreground image; and

processing an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold,

wherein the identifying of the contrast threshold comprises identifying the contrast threshold based on a position and an area of the foreground image from the characteristic of the foreground image, the position and the area affecting human retinal eccentricity and a reflected retina size.

21. An immersive sensation enhancement method, the method comprising:

identifying, by a processor, a foreground image for each frame of an input image;

extracting a characteristic of the foreground image;

identifying a contrast threshold based on the characteristic of the foreground image; and

processing an immersive sensation enhancement of the input image based on at least one of the characteristic of the foreground image and the contrast threshold,

wherein the processing comprises;

identifying a saturation enhancement scale based on the characteristic of the foreground image; and

enhancing an average saturation of the inputted foreground image based on the saturation enhancement scale.