Using pattern recognition to reduce noise in a 3D map

• US 10,181,089 B2
• Filed: 12/19/2016
• Issued: 01/15/2019
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Shintani, Peter, Gonzalez Hernandez, Jose Omar, Clay, William, Espinosa, Pablo Antonio, Ansfield, Fred, Mohapatra, Bibhudendu, Resch, Keith, Usami, Morio
• Primary Examiner(s): Bezuayehu, Solomon G.
• Application Number:
  US15/383,392
• Publication Number:
  US 20180173990A1
• Time in Patent Office: 757 Days
• Field of Search: 379 47, 379103, 379217, 379156, 379199, 382 47, 382103, 382217, 382156, 382199
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

To reduce the random noise in a depth map that is rendered using colors to convey the various depths, pattern recognition may be used to selectively apply noise reduction or to modulate the strength of the noise reduction. In this way, the potential adverse effect on the detail/sharpness of the image can be ameliorated. For example, in an image of a person, the skin does not have any sharp edges so noise reduction can be applied to such an image with little adverse consequence, whereas noise reduction applied to the image of a person'"'"'s eye can cause loss of the detail of the iris, eyelashes, etc. Using pattern recognition on objects in the image, the appropriate level of noise reduction can be applied across an image while minimizing blurring/loss of detail.

Claims

1. A device comprising:

at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to;

receive a digital depth map;

execute image recognition on the depth map;

based at least in part on the image recognition, identify a first object at least in part by identifying at least a segment of a continuous sequence of pixels in an image that have characteristics comparable to each other in the sequence but that are different by from characteristics of pixels nearby the sequence;

based at least in part on the image recognition, identify a second object;

identify a first noise reduction level associated with the first object;

identify a second noise reduction level associated with the second object, the first noise reduction level being different from the second level noise reduction level; and

apply the first and second noise reduction levels to respective entireties of the first and second objects.

2. The device of claim 1, wherein the digital depth map is based on laser emissions.

3. The device of claim 1, wherein the instructions are executable to:

access at least one data structure; and

using the data structure, correlate the first object with the first noise reduction level.

4. The device of claim 3, wherein the instructions are executable to:

using the data structure, correlate the second object with the second noise reduction level.

5. The device of claim 1, comprising the at least one processor.

6. The device of claim 1, where in the second noise reduction level is less than the first noise reduction level, and the second object comprises an ocular iris.

7. The device of claim 1, wherein the second noise reduction level is less than the first noise reduction level, and the second object comprises at least an eyelash.

8. The device of claim 1, wherein the second noise reduction level is zero such that no noise reduction is applied to the second object.

9. An assembly comprising:

plural devices;

each device comprising at least one range finder configured to output signals useful for generating images of other devices on the surface;

at least one processor configured with instructions for selectively applying noise reduction to at least a first one of the images based on pattern recognition executed on the first image, wherein the instructions are executable to;

receive a digital depth map generated by signals from at least one of the range finders;

execute image recognition on the depth map;

based at least in part on the image recognition, identify a first object at least in part by identifying at least a segment of a continuous sequence of pixels in an image that have characteristics similar to each other in the sequence but that are different by from characteristics of pixels nearby the sequence;

based at least in part on the image recognition, identify a second object;

identify a first noise reduction level associated with the first object;

identify a second noise reduction level associated with the second object, the first noise reduction level being different from the second level noise reduction level; and

apply the first and second noise reduction levels to entireties of the respective first and second objects.

10. The assembly of claim 9, wherein at least some of the devices are motorized for movement on at least one surface, the devices being disposed on the surface.

11. The assembly of claim 9, wherein at least some of the devices comprise transceivers for receiving remote control signals from respective controllers.

12. The assembly of claim 9, wherein the at least one range finder comprises at least one laser range finder.

13. The assembly of claim 9, wherein the instructions are executable to:

access at least one data structure, and

using the data structure, correlate the first object with the first noise reduction level.

14. The assembly of claim 13, wherein the instructions are executable to:

using the data structure, correlate the second object with the second noise reduction level.

15. The assembly of claim 9, wherein the second noise reduction level is less than the first noise reduction level.

16. The assembly of claim 9, wherein the second noise reduction level is zero such that no noise reduction is applied to the second object.