Apparatus and method for generating depth information

• US 8,988,429 B2
• Filed: 09/10/2012
• Issued: 03/24/2015
• Est. Priority Date: 09/08/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Kwon, Oh-jae, Moon, Ji-bum, Kim, Han-soo, Sohn, Young-wook, Kim, Seo-young
• Primary Examiner(s): Thompson, James A
• Application Number:
  US13/608,630
• Publication Number:
  US 20130063430A1
• Time in Patent Office: 925 Days
• Field of Search: 345418-422
• US Class Current: 345/420
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

An apparatus for generating depth information includes: a receiver which receives a two-dimensional (2D) image signal including a plurality of frames; a user input unit; a user interface (UI) generator which generates a tool UI to input guide information for generating depth information; a display unit which displays a frame for which depth information is generated among the plurality of frames, and the generated tool UI; and a depth information generator which generates depth information corresponding to the guide information input by the user input unit through the tool UI.

Claims

1. An apparatus for generating depth information comprising:

a receiver which receives a two-dimensional (2D) image signal comprising a plurality of frames;

a user interface (UI) generator which generates a tool UI to input guide information for generating depth information;

a display unit which displays a frame, for which depth information is generated, among the plurality of frames, and the generated tool UI; and

a depth information generator which generates depth information corresponding to the guide information input by a user input through the tool UI,

wherein the guide information, which is input through the tool UI, is input via at least one of a dot, a line, and a diagram on the frame displayed on the display unit, and

wherein the depth information generator extracts location and color information of the tool UI on the frame and partitions the frame as a plurality of areas based on the location and color information.

2. The apparatus according to claim 1, wherein the guide information is input a number of times through the tool UI, and

the depth information generator generates depth information based on a plurality of pieces of the input guide information.

3. The apparatus according to claim 1, wherein the tool UI comprises a first tool UI which is shaped like a dot, and

the depth information generator extracts location and color information of the first tool UI on the frame and partitions a first area having color information within a predetermined range based on the extracted color information in a location of the frame designated through the first tool UI.

4. The apparatus according to claim 3, wherein the depth information generator automatically generates preset first depth information corresponding to the first tool UI for the partitioned first area.

5. The apparatus according to claim 4, wherein the first tool UI comprises a first sub tool UI to adjust the preset first depth information, and

the depth information generator generates second depth information based on a user'"'"'s input that is input through the first sub tool UI for the partitioned first area.

6. The apparatus according to claim 3, wherein the tool UI further comprises a second tool UI which is shaped like a line or diagram, and the depth information generator extracts location information of the second tool UI on the frame, and partitions the frame into a plurality of second areas based on the extracted location information.

7. The apparatus according to claim 6, wherein the depth information generator automatically generates preset third depth information corresponding to the second tool UI for the plurality of second areas.

8. The apparatus according to claim 7, wherein the second tool UI further comprises a second sub tool UI to adjust the preset third depth information, and

the depth information generator generates fourth depth information based on a user'"'"'s input that is input through the second sub tool UI for at least one of the plurality of second areas.

9. The apparatus according to claim 6, wherein a plurality of user'"'"'s inputs is input through the first and second tool UI, and

the depth information generator partitions the frame into a plurality of third areas based on a user'"'"'s input through the second tool UI, and automatically generates preset fifth depth information having a first depth value to create an effect that at least one of the plurality of the third areas which is disposed close to a location corresponding to a user'"'"'s input through the first tool UI appears to be far from a user, and having a second depth value to create an effect that another one of the plurality of the third areas which is disposed further away from the location corresponding to the user'"'"'s input through the first tool UI appears to be closer to the user.

10. The apparatus according to claim 9, wherein, when partitioning the frame into the third areas through the first and second tool UI is omitted,

the depth information generator automatically generates the preset fifth depth information, for the entire frame, having the first depth value to create the effect that an area disposed closer to the location corresponding to the user'"'"'s input through the first tool UI appears to be further away from the user and having the second depth value to create the effect that an area disposed further away from the location corresponding to the user'"'"'s input through the first tool UI appears to be closer to the user.

11. The apparatus according to claim 9, wherein the partitioned first, second and third areas overlap each other, and

the depth information generator generates single depth information by combining the depth information generated for each of overlapping areas.

12. A method for generating depth information comprising:

receiving a two-dimensional (2D) image signal comprising a plurality of frames;

generating a tool user interface (UI) which is used to input guide information for generating depth information;

displaying a frame for which depth information is generated among the plurality of frames, and the generated tool UI;

receiving input for guide information from a user through the displayed tool UI; and

generating depth information corresponding to the input guide information,

wherein the guide information, which is input through the tool UI, is input via at least one of a dot, a line, and a diagram on the frame displayed on the display unit, and

wherein the generating the depth information includes extracting location and color information of the tool UI on the frame and partitioning the frame as a plurality of areas based on the location and color information.

13. The method according to claim 12, wherein the guide information is input a number of times through the tool UI, and

the generating the depth information comprises generating depth information based on a plurality of pieces of the input guide information.

14. The method according to claim 12, wherein the tool UI comprises a first tool UI which is shaped like a dot, and the method further comprises:

extracting location and color information of the first tool UI on the frame; and

partitioning a first area having color information within a predetermined range based on the extracted color information in a location of the frame designated through the first tool UI.

15. The method according to claim 14, wherein the generating the depth information comprises automatically generating preset first depth information corresponding to the first tool UI for the partitioned first area.

16. The method according to claim 15, wherein the first tool UI comprises a first sub tool UI to adjust the preset first depth information, and

the generating the depth information comprises generating second depth information based on a user'"'"'s input that is input through the first sub tool UI for the partitioned first area.

17. The method according to claim 14, wherein the tool UI further comprises a second tool UI which is shaped like a line or diagram, and the method further comprises:

extracting location information of the second tool UI on the frame; and

partitioning the frame into a plurality of second areas based on the extracted location information.

18. The method according to claim 17, wherein the generating the depth information comprises automatically generating preset third depth information corresponding to the second tool UI for the plurality of second areas.

19. The method according to claim 18, wherein the second tool UI comprises a second sub tool UI to adjust the preset third depth information, and

the generating the depth information comprises generating fourth depth information based on a user'"'"'s input that is input through the second sub tool UI for at least one of the plurality of second areas.

20. The method according to claim 17, wherein the generating the depth information further comprises:

partitioning the frame into a plurality of third areas based on a user'"'"'s input through the second tool UI, and

automatically generating preset fifth depth information having a first depth value to create an effect that at least one of the plurality of third areas that is disposed close to a location corresponding to a user'"'"'s input through the first tool UI appears to be far from a user, and having a second depth value to create an effect that at least another one of the plurality of the third areas that is disposed further away from the location corresponding to the user'"'"'s input through the first tool UI appears to be closer to the user, when a plurality of user'"'"'s inputs is input through the first and second tool UIs.

21. The method according to claim 20, wherein, when the partitioning the frame into the plurality of third areas through the first and second tool UI is omitted, the generating the depth information further comprises automatically generating the preset fifth depth information, for the entire area of the frame, having the first depth value to create the effect that an area that is disposed closer to the location corresponding to the user'"'"'s input through the first tool UI appear to be further away from the user and having the second depth value to create the effect that an area that is disposed further away from the location corresponding to the user'"'"'s input through the first tool UI appears to be closer to the user.

22. The method according to claim 20, wherein the partitioned first, second and third areas overlap one another, and

the generating the depth information comprises generating single depth information by combining the depth information generated for each of overlapping areas.

23. A non-transitory computer-readable storage medium storing a program which, when executed by a computer, causes the computer to execute the method of claim 12.

24. An apparatus for generating depth information comprising:

a receiver which receives a two-dimensional (2D) image signal which comprises a plurality of frames;

a user interface (UI) generator which generates a tool UI to input guide information for generating depth information;

a display unit which displays a frame, for which the depth information is generated, among the plurality of frames, and the generated tool UI; and

a depth information generator which partitions an object on the displayed frame, and generates depth information corresponding to the guide information for the partitioned object based on the guide information input through the tool UI in a location within the partitioned object,

wherein the guide information, which is input through the tool UI, is input via at least one of a dot, a line, and a diagram on the frame displayed on the display unit, and

wherein the depth information generator extracts location and color information of the tool UI on the frame and partitions the frame as a plurality of areas based on the location and color information.

25. The apparatus according to claim 24, wherein the guide information is input a number of times through the tool UI in a plurality of locations within the object, and

the depth information generator generates the depth information based on a plurality of pieces of the input guide information for the object.

26. The apparatus according to claim 24, wherein the tool UI comprises a first tool UI which occupies a first area and has a predetermined shape comprising a vertex and the first tool UI is used to adjust a location of the vertex through a user input unit to adjust at least one of a size and a location of the first area.

27. The apparatus according to claim 26, wherein the first tool UI comprises a first sub tool UI to display and adjust preset depth information, and

the depth information generator generates first depth information based on a user'"'"'s input that is input through the first sub tool UI for the first area designated through the first tool UI.

28. The apparatus according to claim 27, wherein the first sub tool UI is located and displayed on the vertex of the first tool UI.

29. The apparatus according to claim 24, wherein the tool UI comprises a second tool UI which is shaped like a dot,

the second tool UI comprises a second sub tool UI to display and adjust preset depth information, and

the depth information generator generates second depth information based on a user'"'"'s input that is input through the second sub UI for a second area of a predetermined size centering on an area where the second tool UI is located.

30. The apparatus according to claim 29, wherein the second tool UI further comprises a third sub tool UI to adjust a size of the second area.

31. The apparatus according to claim 29, wherein the second tool UI further comprises:

a fourth sub tool UI that is located in an edge of the second area and is used to input third depth information; and

a fifth sub tool UI to adjust a degree of a curve of a depth value that one of gradually increases and gradually decreases between the second and third depth information.

32. The apparatus according to claim 29, wherein the second tool UI is adjusted in a location within the second area.

33. The apparatus according to claim 29, wherein the second tool UI further comprises a sixth sub tool UI to adjust at least one of a rotation and an inclination of the second area.

34. The apparatus according to claim 29, wherein the tool UI comprises a third tool UI to occupy a third area having a width of the second area and a length of a plurality of second tool UIs connected to one another, and

the depth information generator generates the second depth information with respect to the third area.

35. The apparatus according to claim 29, wherein the tool UI further comprises a fourth tool UI which is shaped like a line that connects a plurality of second tool UIs, and

the depth information generator extracts location information of the fourth tool UI within the object and partitions the object into a plurality of fourth areas based on the extracted location information and generates the second depth information with respect to the fourth area.

36. The apparatus according to claim 29, wherein the tool UI connects the plurality of second tool UIs and comprises a fifth tool UI to occupy a fifth area which is formed by connecting the plurality of second tool UIs, and

the depth information generator generates the second depth information with respect to the fifth area.

37. A method for generating depth information comprising:

receiving a two-dimensional (2D) image signal which comprises a plurality of frames;

generating a tool user interface (UI) to input guide information for generating depth information;

displaying a frame, for which the depth information is generated, among the plurality of frames, and the generated tool UI;

partitioning an object on the displayed frame;

inputting guide information through the tool UI in a location within the partitioned object; and

generating the depth information corresponding to the input guide information for the partitioned object,

wherein the guide information, which is input through the tool UI, is input via at least one of a dot, a line, and a diagram on the frame displayed on the display unit, and

wherein the generating the depth information includes extracting location and color information of the tool UI on the frame and partitioning the frame as a plurality of areas based on the location and color information.

38. The method according to claim 37, wherein the generating the depth information further comprises:

generating the depth information based on a plurality of pieces of the guide information for the object, the plurality of pieces of the guide information being input a number of times, through the tool UI in a plurality of locations within the object.

39. The method according to claim 37, wherein the tool UI comprises a first tool UI which occupies a first area and has a predetermined shape comprising a vertex, and the first tool UI is used to adjust a location of the vertex through a user input unit to adjust at least one of a size and a location of the first area.

40. The method according to claim 39, wherein the first tool UI further comprises a first sub tool UI to display and adjust preset depth information, and the generating the depth information comprises generating first depth information based on a user'"'"'s input that is input through the first sub tool UI for the first area designated through the first tool UI.

41. The method according to claim 40, wherein the first sub tool UI is located and displayed on the vertex of the first tool UI.

42. The method according to claim 37, wherein the tool UI comprises a second tool UI which is shaped like a dot, and the second tool UI further comprises a second sub tool UI to display and adjust preset depth information, and the generating the depth information comprises generating second depth information based on a user'"'"'s input that is input through the second sub tool UI for a preset second area in a predetermined size centering on an area where the second tool UI is located.

43. The method according to claim 42, wherein the second tool UI further comprises a third sub tool UI to adjust a size of the second area.

44. The method according to claim 42, wherein the second tool UI further comprises:

a fourth sub tool UI that is located in an edge of the second area and is used to input third depth information; and

a fifth sub tool UI to adjust a degree of a curve of a depth value that one of gradually increases and gradually decreases between the second and third depth information.

45. The method according to claim 42, wherein the second tool UI is adjusted in the location within the second area.

46. The method according to claim 42, wherein the second tool UI further comprises a sixth sub tool UI to adjust at least one of a rotation and an inclination of the second area.

47. The method according to claim 42, wherein the tool UI comprises a third tool UI to occupy a third area having a width of the second area and a length of a plurality of second tool UIs connected to one another, and

the generating the depth information comprises generating the second depth information with respect to the third area.

48. The method according to claim 42, wherein the tool UI further comprises a fourth tool UI which is shaped as a line that connects a plurality of second tool UIs, and

the generating the depth information comprises extracting location information of the fourth tool UI and partitioning the object into a plurality of fourth areas based on the extracted location information, and generating the second depth information with respect to the fourth area.

49. The method according to claim 42, wherein the tool UI connects a plurality of second tool UIs and comprises a fifth tool UI to occupy a fifth area which is formed by connecting the plurality of second tool UIs, and

the generating the depth information comprises generating the second depth information with respect to the fifth area.

50. A non-transitory computer-readable storage medium storing a program which, when executed by a computer, causes the computer to execute the method of claim 37.