IMAGE PROCESSING METHOD AND APPARATUS THEREFOR

• US 20100165081A1
• Filed: 09/18/2009
• Published: 07/01/2010
• Est. Priority Date: 12/26/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chung, Hyun-kwon, JUNG, Kil-soo, Jun, Hye-young, Lee, Dae-jong
• Granted Patent: US 8,767,048 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

An image processing method, including extracting shot information from metadata, wherein the shot information is used to classify a series of two-dimensional images into predetermined units; if it is determined, by using shot type information included in the shot information, that frames classified as a predetermined shot can be reproduced as a three-dimensional image, extracting background depth information from the metadata, wherein the background depth information is about a background of the frames classified as the predetermined shot; generating a depth map about the background of the frames by using the background depth information; if the frames classified as the predetermined shot comprise object, extracting object depth information about the object from the metadata; and generating a depth map about the object by using the object depth information.

Claims

1. An image processing method, comprising:

extracting shot information from metadata, wherein the shot information classifies a series of two-dimensional images into predetermined units;

determining whether frames classified as a predetermined shot can be reproduced as a three-dimensional image based on shot type information included in the shot information;

extracting background depth information from the metadata if the frames classified as the predetermined shot can be reproduced as a three-dimensional image, wherein the background depth information relates to a background of the frames classified as the predetermined shot;

generating a depth map about the background of the frames based on the background depth information;

if the frames classified as the predetermined shot comprise an object, extracting object depth information about the object from the metadata; and

generating a depth map about the object based on the object depth information.

2. The image processing method of claim 1, further comprising calculating a time at which the frames classified as the predetermined shot are required to be output based on the shot information.

3. The image processing method of claim 1, wherein the generating of the depth map about the background comprises extracting information from the background depth information, the information comprising background type information indicating a basic composition of the background, surface information for applying a depth value to a surface comprised in the background, coordinate points of the background, a depth value at each of the coordinate points of the background, and a panel position value indicating a depth value of a screen.

4. The image processing method of claim 3, wherein the generating of the depth map about the background comprises generating the depth map about the background based on a depth value of the basic composition of the background according to the background type information, and a depth value of the surface according to the surface information.

5. The image processing method of claim 3, further comprising:

filtering the depth map about the background if the background of the frames have the same depth values;

wherein the filtering of the depth map about the background comprises applying a hemisphere-shaped filter, a hemicylinder-shaped filter, or a triangular prism-shaped filter to the depth map about the background so as to modify the depth map about the background.

6. The image processing method of claim 1, wherein the object depth information comprises object type information identifying a type of the object from among a two-dimensional object, a normal object, and a highlight object.

7. The image processing method of claim 1, wherein:

if a frame comprises a plurality of objects, the object depth information comprises priority order information indicating a depth map generation order for the plurality of objects; and

the generating of the depth map about the object comprises sequentially generating depth maps about the plurality of objects according to the depth map generation order.

8. The image processing method of claim 1, wherein the generating of the depth map about the object comprises obtaining a time at which the frames comprising the object are required to be output based on the object depth information.

9. The image processing method of claim 1, further comprising:

extracting area identification information identifying an area of the object from the object depth information;

wherein the area identification information corresponds to information in which the area of the object is denoted as a coordinate point, or corresponds to a mask in which a shape of the object is marked.

10. The image processing method of claim 9, wherein, if the object is a two-dimensional object, the generating of the depth map about the object comprises generating a depth map about the two-dimensional object based on a depth value of the area of the object that is determined based on the panel position value comprised in the background depth information.

11. The image processing method of claim 9, wherein:

if the object is a normal object and the area identification information corresponds to the mask in which the shape of the object is marked, the object depth information further comprises information about one or more background coordinate points from among coordinate points of the background included in the background depth information and which are equal to one or more coordinate points indicating an area of the normal object; and

the generating of the depth map about the object comprises generating a depth map about the normal object based on depth values of the area of the normal object that are determined based on depth values of the one or more background coordinate points.

12. The image processing method of claim 9, wherein, if the object is a normal object, the generating of the depth map about the object comprises:

obtaining one or more background coordinate points from among coordinate points of the background included in the background depth information and which are equal to coordinate points indicating an area of the normal object; and

generating the depth map about the normal object based on depth values of the area of the normal object that are determined based on depth values of the background coordinate points.

13. The image processing method of claim 9, wherein, if the object is a highlight object, the generating of the depth map about the object comprises:

calculating a depth value of an area of the highlight object based on the panel position value included in the background depth information and an offset value included in the object depth information; and

generating a depth map about the highlight object based on the depth value of the area of the highlight object.

14. The image processing method of claim 13, further comprising:

filtering the depth map about the object;

wherein the filtering of the depth map about the object comprises applying a hemisphere-shaped filter, a hemicylinder-shaped filter, or a triangular prism-shaped filter to the depth map about the object so as to modify the depth map about the object.

15. The image processing method of claim 1, further comprising:

extracting title identification information from the metadata;

wherein the title identification information identifies a title comprising the frames, and thus determines which title is related to the metadata.

16. The image processing method of claim 1, further comprising:

generating a left-eye image and a right-eye image with respect to the two-dimensional image based on the depth map about the background and the depth map about the object;

compensating for holes of the left-eye image and the right-eye image; and

generating a three-dimensional format image based on the compensated left-eye image and the compensated right-eye image.

17. An image processing apparatus, comprising:

a metadata analyzing unit to extract shot information from metadata based on shot type information included in the shot information, the shot information classifying a series of two-dimensional images into predetermined units, to extract background depth information from the metadata when frames classified as a predetermined shot can be reproduced as a three-dimensional image, the background depth information relating to a background of the frames classified as the predetermined shot, and to extract and analyze object depth information about an object from the metadata when the frames classified as the predetermined shot include the object; and

a depth map generating unit to generate a depth map about the background of the frames based on the background depth information, and to generate a depth map about the object based on the object depth information.

18. The image processing apparatus of claim 17, wherein the metadata analyzing unit calculates a time at which the frames classified as the predetermined shot are required to be output, based on the shot information.

19. The image processing apparatus of claim 17, wherein:

the depth map generating unit extracts information from the background depth information to generate the depth map about the background; and

the information extracted from the background depth information comprises background type information indicating a basic composition of the background, surface information for applying a depth value to a surface comprised in the background, coordinate points of the background, a depth value at each of the coordinate points of the background, and a panel position value indicating a depth value of a screen.

20. The image processing apparatus of claim 19, wherein the depth map generating unit generates the depth map about the background based on a depth value of the basic composition of the background according to the background type information, and a depth value of the surface according to the surface information.

21. The image processing apparatus of claim 19, further comprising:

a filtering unit to filter the depth map;

wherein, when the background of the frames have the same depth values, the filtering unit applies a hemisphere-shaped filter, a hemicylinder-shaped filter, or a triangular prism-shaped filter to the depth map about the background so as to filter the depth map about the background.

22. The image processing apparatus of claim 17, wherein the object depth information comprises object type information for identifying a type of the object from among a two-dimensional object, a normal object, and a highlight object.

23. The image processing apparatus of claim 17, wherein:

when a frame comprises a plurality of objects, the object depth information comprises priority order information indicating a depth map generation order for the plurality of objects, and

the depth map generating unit sequentially generates depth maps about the plurality of objects according to the depth map generation order.

24. The image processing apparatus of claim 17, wherein the depth map generating unit obtains a time at which the frames comprising the object are required to be output based on the object depth information.

25. The image processing apparatus of claim 17, wherein:

the depth map generating unit extracts area identification information identifying an area of the object from the object depth information, and

the area identification information corresponds to information in which the area of the object is denoted as a coordinate point, or corresponds to a mask in which a shape of the object is marked.

26. The image processing apparatus of claim 25, wherein, when the object is a two-dimensional object, the depth map generating unit generates a depth map about the two-dimensional object based on a depth value of the area of the object that is determined based on a panel position value included in the background depth information t.

27. The image processing apparatus of claim 25, wherein:

when the object is a normal object and when the area identification information corresponds to the mask in which the shape of the object is marked, the object depth information further comprises information about one or more background coordinate points from among coordinate points of the background included in the background depth information and which are equal to coordinate points indicating an area of the normal object; and

the depth map generating unit generates a depth map about the normal object based on depth values of the area of the normal object that are determined based on depth values of the one or more background coordinate points.

28. The image processing apparatus of claim 25, wherein, when the object is a normal object, the depth map generating unit obtains one or more background coordinate points from among coordinate points of the background included in the background depth information and which are equal to coordinate points indicating an area of the normal object, and generates a depth map about the normal object based on depth values of the area of the normal object that are determined based on depth values of the background coordinate points.

29. The image processing apparatus of claim 25, wherein, when the object is a highlight object, the depth map generating unit generates a depth map about the highlight object based on a depth value of an area of the highlight object, wherein the depth value is calculated based on a panel position value comprised in the background depth information and an offset value included in the object depth information.

30. The image processing apparatus of claim 29, further comprising:

a filtering unit to filter the depth map about the object, and

wherein the filtering unit applies a hemisphere-shaped filter, a hemicylinder-shaped filter, or a triangular prism-shaped filter to the depth map about the object to filter the depth map about the object.

31. The image processing apparatus of claim 17, wherein:

the metadata analyzing unit extracts title identification information from the metadata; and

the title identification information identifies a title comprising the frames, and determines which title is related to the metadata based on the title identification information.

32. The image processing apparatus of claim 17, further comprising:

a stereo image generating unit to generate a left-eye image and a right-eye image with respect to the two-dimensional image based on the depth map about the background and the depth map about the object;

a hole compensating unit to compensate for holes of the left-eye image and the right-eye image; and

a three-dimensional format image generating unit to generate a three-dimensional format image by using the left-eye image and the right-eye image of which holes are compensated for.

34. The image processing apparatus of claim 17, wherein the depth map generating unit further comprises:

a background depth map generating unit to generate the depth map about the background of the frames based on the background depth information;

an object depth map generating unit to generate the depth map about the object based on the object depth information;

a filtering unit to apply a filter to the depth map about the background and/or the depth map about the object; and

a depth map buffer unit to temporarily store the depth map about the background and to generate a depth map for the frame based on the filtered depth map about the background and the filtered depth map about the object.

33. A recording medium storing a computer readable program to execute an image processing method, the image processing method comprising:

extracting shot information from metadata, wherein the shot information classifies a series of two-dimensional images into predetermined units;

determining whether frames classified as a predetermined shot can be reproduced as a three-dimensional image;

extracting background depth information from the metadata if the frames classified as the predetermined shot can be reproduced as a three-dimensional image, wherein the background depth information relates to a background of the frames classified as the predetermined shot;

generating a depth map about the background of the frames based on the background depth information;

when the frames classified as the predetermined shot comprise an object, extracting object depth information about the object from the metadata; and

generating a depth map about the object based on the object depth information.