Automatic identification of projection surfaces

• US 9,336,607 B1
• Filed: 11/28/2012
• Issued: 05/10/2016
• Est. Priority Date: 11/28/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chang, Samuel Henry, Yao, Ning, Gopalan, Sowmya
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US13/687,952
• Time in Patent Office: 1,259 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Techniques are described for identifying suitable surfaces within a room upon which visual content may be displayed. One or more images of the room are obtained and used to identify planar surfaces. The images may also be analyzed to determine the visual textures of the surfaces within the room. Suitable projection surfaces may then be selected from the identified planar surfaces, based on the visual textures of the planar surfaces.

Claims

1. A system comprising:

one or more processors;

an image sensor;

a depth sensor;

a projector;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

obtaining a depth image of a room using the depth sensor, wherein the depth image indicates distances of surface points of the room from the depth sensor;

calculating a gradient map based at least in part on the depth image, wherein the gradient map indicates relative distances of the surface points from neighboring surface points;

calculating an orientation map based at least in part on the depth image, wherein the orientation map indicates surface orientations at the surface points;

identifying one or more planar surfaces within the room based at least in part on the gradient map and the orientation map;

obtaining a visual image of the room using the image sensor;

calculating a visual texture map based at least in part on the visual image, wherein the visual texture map indicates visible textures of surfaces within the room;

selecting a projection surface from the one or more identified planar surfaces based at least in part on the visual textures of the identified planar surfaces indicated by the visual texture map; and

projecting content onto the projection surface using the projector.

2. The system of claim 1, wherein selecting the projection surface comprises identifying one of the identified planar surfaces having relatively low visible texture.

3. The system of claim 1, wherein selecting the projection surface comprises selecting one of the identified planar surfaces having visible texture that is below a predetermined threshold.

4. The system of claim 1, further comprising adjusting the projected content to compensate for a color or shade of the projection surface.

5. The system of claim 1, wherein selecting the projection surface is based at least in part on sizes of the identified planar surfaces.

6. The system of claim 1, wherein selecting the projection surface is based at least in part on sizes of rectangular areas that can be fitted within the identified planar surfaces.

7. A method, comprising:

obtaining one or more images of a user environment;

generating a gradient map that indicates a distance of one or more surface points of the user environment and one or more neighboring surface points of the user environment;

identifying one or more planar surfaces of the user environment based at least in part on the gradient map;

evaluating visual characteristics of one or more surfaces of the user environment based at least in part on the one or more images;

selecting a projection surface from the one or more identified planar surfaces based at least in part on the evaluated visual characteristics; and

projecting content onto the projection surface, the projection surface comprising a physical surface in the user environment.

8. The method of claim 7, wherein the visual characteristics comprise one or more of:

color;

hue;

shade;

texture;

reflectivity;

whiteness;

or

brightness.

9. The method of claim 7, further comprising:

adjusting the projected content to compensate for a color or shade of the projection surface.

10. The method of claim 7, wherein evaluating the visual characteristics comprises calculating a texture map based at least in part on the one or more images.

11. The method of claim 7, wherein selecting the projection surface is further based at least in part on one or more of:

sizes of the identified planar surfaces;

locations of the identified planar surfaces;

or

heights of the identified planar surfaces.

12. The method of claim 7, wherein the one or more images comprise a visual image and a depth image.

13. The method of claim 12, wherein evaluating the visual characteristics comprises analyzing the visual image.

14. The method of claim 12, wherein identifying the one or more planar surfaces comprises analyzing the depth image.

15. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

obtaining a depth image of a user environment;

generating a gradient map that indicates a distance of one or more surface points of the user environment and one or more neighboring surface points of the user environment;

identifying one or more planar surfaces of the user environment based at least in part on the gradient map;

obtaining a visual image of the user environment;

selecting a projection surface from the one or more identified planar surfaces based at least in part on the visual image; and

projecting content onto the projection surface, the projection surface comprising a physical surface in the user environment.

16. The one or more computer-readable media of claim 15, wherein selecting the projection surface is based at least in part on one or more visual characteristics of the planar surfaces, the visual characteristics comprising one or more of:

color;

hue;

shade;

texture;

reflectivity;

whiteness;

or

brightness.

17. The one or more computer-readable media of claim 15, wherein the depth image indicates distances of individual surface points of the room from an observation location.

18. The one or more computer-readable media of claim 15, wherein selecting the projection surface is based at least in part on one or more of:

sizes of the identified planar surfaces;

locations of the identified planar surfaces;

or

heights of the identified planar surfaces.

19. The one or more computer-readable media of claim 15, the acts further comprising evaluating visual textures of the identified planar surfaces based at least in part on the visual image, wherein selecting the projection surface is based at least in part on the evaluated visual textures of the identified planar surfaces.