Method, electronic device and medium for adjusting depth values

• US 9,363,499 B2
• Filed: 06/27/2014
• Issued: 06/07/2016
• Est. Priority Date: 11/15/2013
• Status: Active Grant

• Current Assignee: HTC Corporation
• Original Assignee: HTC Corporation
• Inventors: Wu, Jing-Lung, Tai, Pol-Lin, Chueh, Hsin-Ti, Feng, Yi-Hsuan
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US14/316,797
• Publication Number:
  US 20150139533A1
• Time in Patent Office: 711 Days
• Field of Search: 382/106, 382/154, 382/162, 382/199, 382/266
• US Class Current: 1/1
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces

Abstract

A depth processing method, an electronic device and a medium are provided. The depth processing method includes: obtaining a color image and a depth map corresponding to the color image; extracting a plurality of regions from at least one of the depth map and the color image; obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information; and adjusting a plurality of depth values in the depth map according to the region information.

Claims

1. A depth processing method for an electronic device, the depth processing method comprising:

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information; and

adjusting a plurality of depth values in the depth map according to the region information,

wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, and a first color region of the color regions corresponds to a first depth region of the depth regions, wherein the step of adjusting the depth values in the depth map according to the region information comprises;

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

2. The depth processing method as recited in claim 1, wherein the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

3. The depth processing method as recited in claim 2, further comprising:

executing an object detection procedure on the color image to obtain an object; and

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

4. The depth processing method as recited in claim 3, further comprising:

adjusting a discontinuous depth value in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

5. The depth processing method as recited in claim 1, wherein the area information is obtained by performing a clustering algorithm on the color image.

6. The depth processing method as recited in claim 1, wherein the edge information is obtained by performing an edge detection algorithm on the color image.

7. The depth processing method as recited in claim 1, wherein the electronic device comprises a touch unit, and the depth processing method comprises:

detecting a touch operation on the touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

8. An electronic device, comprising:

a memory, storing a plurality of instructions; and

a processor, coupled to the memory, and configured to execute the instructions to perform a plurality of steps;

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information; and

adjusting a plurality of depth values in the depth map according to the region information,

wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, a first color region of the color regions corresponds to a first depth region of the depth regions, and the step of adjusting the depth values in the depth ma according to the region information comprises;

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

9. The electronic device as recited in claim 8, wherein the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

10. The electronic device as recited in claim 9, wherein the steps further comprises:

executing an object detection procedure on the color image to obtain an object; and

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

11. The electronic device as recited in claim 8, further comprising a touch unit, and the steps further comprising:

detecting a touch operation on the touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

12. A non-transitory computer readable medium, for storing a plurality of instructions, and the instructions are configured to perform a plurality of steps:

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information; and

adjusting a plurality of depth values in the depth map according to the region information,

wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, a first color region of the color regions corresponds to a first depth region of the depth regions, and the step of adjusting the depth values in the depth map according to the region information comprises;

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation, procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

13. The non-transitory computer readable medium as recited in claim 12, wherein the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

14. The non-transitory computer readable medium as recited in claim 13, wherein the steps further comprises:

executing an object detection procedure on the color image to obtain an object; and

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

15. The non-transitory computer readable medium as recited in claim 12, wherein the steps further comprises:

detecting a touch operation on a touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

16. A depth processing method for an electronic device, wherein the electronic device comprises a touch unit, and the depth processing method comprising:

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information;

adjusting a plurality of depth values in the depth map according to the region information;

detecting a touch operation on the touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

17. The depth processing method as recited in claim 16, wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, and the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

18. The depth processing method as recited in claim 17, wherein a first color region of the color regions corresponds to a first depth region of the depth regions, and the step of adjusting the depth values in the depth map according to the region information comprises:

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

19. The depth processing method as recited in claim 17, further comprising:

executing an object detection procedure on the color image to obtain an object; and

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

20. The depth processing method as recited in claim 19, further comprising:

adjusting a discontinuous depth value in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

21. The depth processing method as recited in claim 16, wherein the area information is obtained by performing a clustering algorithm on the color image.

22. The depth processing method as recited in claim 16, wherein the edge information is obtained by performing an edge detection algorithm on the color image.

23. An electronic device, comprising:

a memory, storing a plurality of instructions;

a processor, coupled to the memory; and

a touch unit,

wherein the processor is configured to execute the instructions to perform a plurality of steps;

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information;

adjusting a plurality of depth values in the depth map according to the region information;

detecting a touch operation on the touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

24. The electronic device as recited in claim 23, wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, and the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

25. The electronic device as recited in claim 24, wherein a first color region of the color regions corresponds to a first depth region of the depth regions, and the step of adjusting the depth values in the depth map according to the region information comprises:

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

26. The electronic device as recited in claim 24, wherein the steps further comprises:

executing an object detection procedure on the color image to obtain an object;

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.

27. A non-transitory computer readable medium, for storing a plurality of instructions, and the instructions are configured to perform a plurality of steps:

obtaining a color image and a depth map corresponding to the color image;

extracting a plurality of regions from at least one of the depth map and the color image;

obtaining region information of the regions, and classifying the regions into at least one of a region-of-interest and a non-region-of-interest according to the region information, wherein the region information comprises area information and edge information;

adjusting a plurality of depth values in the depth map according to the region information;

detecting a touch operation on a touch unit;

obtaining a touch range according to the touch operation;

calculating a variance of the depth values in the touch range, wherein the depth values smaller than a first predetermined depth value and greater than a second predetermined depth value are selected for calculating the variance; and

if the variance is greater than a threshold, obtaining a plurality of neighboring range next to the touch range, and setting a focal length according to the depth values in one of the neighboring ranges which has smallest variance.

28. The non-transitory computer readable medium as recited in claim 27, wherein the regions comprises a plurality of color regions and a plurality of depth regions, the depth regions are obtained from the depth map, and the step of classifying the regions into at least one of the region-of-interest and the non-region-of-interest according to the region information comprises:

classifying the depth regions into the region-of-interest or the non-region-of-interest according to at least one of a size, a depth, a location of each of the depth regions.

29. The non-transitory computer readable medium as recited in claim 28, wherein a first color region of the color regions corresponds to a first depth region of the depth regions, and the step of adjusting the depth values in the depth map according to the region information comprises:

performing an erosion procedure on the first depth region to obtain a first boundary;

performing a dilation procedure on the first depth region to obtain a second boundary; and

mapping the first boundary and the second boundary into the color image, and performing a segmentation algorithm in an area between the first boundary and the second boundary in the color image for adjusting a contour of the first depth region.

30. The non-transitory computer readable medium as recited in claim 28, wherein the steps further comprises:

executing an object detection procedure on the color image to obtain an object; and

mapping an object area of the object into the depth map, and filling a depth hole in the object area of the object in the depth map according to the depth values in the object area of the object in the depth map.