USING PATTERN RECOGNITION TO REDUCE NOISE IN A 3D MAP

• US 20180173990A1
• Filed: 12/19/2016
• Published: 06/21/2018
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Shintani, Peter, Gonzalez Hernandez, Jose Omar, Clay, William, Espinosa, Pablo Antonio, Ansfield, Fred, Mohapatra, Bibhudendu, Resch, Keith, Usami, Morio
• Granted Patent: US 10,181,089 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

To reduce the random noise in a depth map that is rendered using colors to convey the various depths, pattern recognition may be used to selectively apply noise reduction or to modulate the strength of the noise reduction. In this way, the potential adverse effect on the detail/sharpness of the image can be ameliorated. For example, in an image of a person, the skin does not have any sharp edges so noise reduction can be applied to such an image with little adverse consequence, whereas noise reduction applied to the image of a person'"'"'s eye can cause loss of the detail of the iris, eyelashes, etc. Using pattern recognition on objects in the image, the appropriate level of noise reduction can be applied across an image while minimizing blurring/loss of detail.

Claims

1. A device comprising:

at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to;

receive a digital depth map;

execute image recognition on the depth map;

based at least in part on the image recognition, identify a first object;

based at least in part on the image recognition, identify a second object;

identify a first noise reduction level associated with the first object;

identify a second noise reduction level associated with the second object, the first noise reduction level being different from the second level noise reduction level; and

apply the first and second noise reduction levels to the respective first and second objects.

2. The device of claim 1, wherein the digital depth map is based on laser emissions.

3. The device of claim 1, wherein the instructions are executable to:

access at least one data structure; and

using the data structure, correlate the first object with the first noise reduction level.

4. The device of claim 3, wherein the instructions are executable to:

using the data structure, correlate the second object with the second noise reduction level.

5. The device of claim 1, comprising the at least one processor.

6. The device of claim 1, where in the second noise reduction level is less than the first noise reduction level, and the second object comprises an object which has much fine detail such as an ocular iris.

7. The device of claim 1, wherein the second noise reduction level is less than the first noise reduction level, and the second object comprises at another object with a different type of fine detail such as an eyelash.

8. The device of claim 1, wherein the second noise reduction level is zero such that no noise reduction is applied to the second object.

9. A method comprising:

applying noise reduction to reduce noise in a depth map that is rendered using colors to convey various depths of objects in an image;

executing pattern recognition on the image;

selectively adjusting the noise reduction and/or modulating a strength of the noise reduction based at least in part on the pattern recognition.

10. The method of claim 9, wherein adverse effects on detail/sharpness of the image are ameliorated.

11. The method of claim 9, comprising:

based on pattern recognition indicating an object with no sharp edges, applying the noise reduction to the object.

12. The method of claim 11, comprising:

based on pattern recognition indicating an object with sharp edges, not applying the noise reduction to the object, or applying the noise reduction to the object at a lesser level than full noise reduction.

13. An assembly comprising:

plural devices;

each device comprising at least one laser range finder configured to output signals useful for generating images of other devices on the surface;

at least one processor configured with instructions for selectively applying noise reduction to at least a first one of the images based on pattern recognition executed on the first image.

14. The assembly of claim 13, wherein at least some of the devices are motorized for movement on at least one surface, the devices being disposed on the surface.

15. The assembly of claim 13, wherein at least some of the devices comprise transceivers for receiving remote control signals from respective controllers.

16. The assembly of claim 13, comprising at least one computer medium that comprises instructions executable by the at least one processor to:

receive a digital depth map generated by signals from at least one of the laser range finders;

execute image recognition on the depth map;

based at least in part on the image recognition, identify a first object;

based at least in part on the image recognition, identify a second object;

identify a first noise reduction level associated with the first object;

identify a second noise reduction level associated with the second object, the first noise reduction level being different from the second level noise reduction level; and

apply the first and second noise reduction levels to the respective first and second objects.

17. The assembly of claim 16, wherein the instructions are executable to:

access at least one data structure; and

using the data structure, correlate the first object with the first noise reduction level.

18. The assembly of claim 17, wherein the instructions are executable to:

using the data structure, correlate the second object with the second noise reduction level.

19. The assembly of claim 16, wherein the second noise reduction level is less than the first noise reduction level.

20. The assembly of claim 16, wherein the second noise reduction level is zero such that no noise reduction is applied to the second object.