Three-dimensional model generation

• US 9,892,666 B1
• Filed: 06/20/2012
• Issued: 02/13/2018
• Est. Priority Date: 06/20/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Liu, Qiang, Chang, Samuel Henry, Marason, Eric G.
• Primary Examiner(s): Truong, Nguyen
• Application Number:
  US13/528,759
• Time in Patent Office: 2,064 Days
• Field of Search: 348 49, 353 98
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 21/28 Reflectors in projection beam in illumination beam G03B21/2066
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: G09F 19/18 involving the use of optical projection means, e.g. projection of images on clouds projection apparatus per se G03B

Abstract

Within a particular environment, a projection that may include light or one or more patterns may be directed at one or more mirrors. The projection may be reflected by a first one of the mirrors onto the surroundings of the environment, which may include a floor, one or more walls, and/or objects in the environment having a surface. A second mirror may represent an entire view of the surroundings of the environment. As a result, by taking an image directed at the second mirror, a single image may be captured that represents a view of the entire environment without having to rotate the device that captures the image. The single image may then be utilized to generate a three-dimensional model of the environment.

Claims

1. A system comprising:

a projection device configured to transmit one or more light patterns;

one or more convex mirrors configured to reflect the one or more light patterns onto one or more objects in a room;

a stationary image capture device configured to capture an image representative of the room, wherein;

a first one of the one or more convex mirrors is associated with the projection device and reflects light corresponding to the one or more light patterns onto the one or more objects included in the room;

a second one of the one or more convex mirrors is associated with the stationary image capture device and is directed at the stationary image capture device; and

an optical axis of the stationary image capture device is substantially parallel to and non-coaxial with an optical axis of the projection device; and

a computing device configured to generate, based at least in part on the image, a three-dimensional model of the room and the one or more objects.

2. The system as recited in claim 1, wherein at least one of the one or more convex mirrors is a hyperboloidal or paraboloidal mirror.

3. The system as recited in claim 1, wherein the computing device is configured to utilize the three-dimensional model to cause the projection device to project content onto a surface within the room based at least in part on determining that the surface is flat.

4. The system as recited in claim 1, wherein the one or more light patterns include a grid, a set of horizontal lines, or a set of vertical lines.

5. The system as recited in claim 1, wherein the computing device is further configured to:

estimate at least one disparity between the one or more objects from a disparity map and corresponding objects from the three-dimensional model of the room; and

adjust the three-dimensional model based on the at least one estimated disparity.

6. The system as recited in claim 1, wherein the computing device is further configured to:

record a position and an orientation of one or more of the projection device, the one or more convex mirrors, or the stationary image capture device; and

generate the three-dimensional model of the room based at least partly on the recorded position and the recorded orientation.

7. A method comprising:

under control of one or more processors of a computing device,

projecting light from a projector onto a first curved mirror, the first curved mirror being associated with the projector and reflecting the light onto one or more objects in an environment;

capturing, by a stationary capture device directed at a second curved mirror, an image that represents the environment, the second curved mirror being associated with the stationary capture device, and wherein an optical axis of the stationary capture device is substantially parallel to and offset from an optical axis of the projector; and

generating, based at least in part on the image, a three-dimensional model of the environment.

8. The method as recited in claim 7, wherein the image is captured without rotating the stationary capture device and without rotating the second curved mirror.

9. The method as recited in claim 7, further comprising:

determining an orientation of the second curved mirror and the stationary capture device;

determining spatial relationships between the second curved mirror, the stationary capture device, and the one or more objects included in the environment; and

generating, based at least in part on the orientation and the spatial relationships, the three-dimensional model of the environment.

10. The method as recited in claim 7, wherein:

the environment is a room that includes a floor, one or more walls, and one or more objects that have at least one flat surface;

the second curved mirror is located in a first position in the room; and

the stationary capture device is located in a second position in the room that is below a reflective surface of the second curved mirror.

11. The method as recited in claim 7, wherein the environment that is represented by the image includes one or more light patterns that are projected by the projector and that are reflected by the first curved mirror onto the environment.

12. The method as recited in claim 7, wherein an optical axis of the first curved mirror is non-coaxial with an optical axis of the second curved mirror.

13. One or more non-transitory computer-readable media having computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

causing, by a projection device having an optical axis, light to be projected onto a first curved mirror, the first curved mirror being associated with the projection device and reflecting the light onto one or more objects in an environment;

receiving, at a capture device having an optical axis, one or more images that represent an environment, the capture device being directed at a second curved mirror associated with the capture device, and wherein the optical axis of the capture device is non-coaxial with the optical axis of the projection device;

generating a three-dimensional model of the environment based at least partly on the one or more images, the one or more images being formed at least partly by;

identifying a point that represents a three-dimensional object in the environment, the point being reflected by the second curved mirror; and

transforming the point to a corresponding pixel in the one or more images such that the corresponding pixel represents an actual location of the point in the environment; and

adjusting the three-dimensional model based at least in part on a difference in position between a first coordinate associated with a location of the projection device and a second coordinate associated with a location of the capture device.

14. The one or more computer-readable media as recited in claim 13, wherein the point is transformed to the corresponding pixel in the one or more images using a transformation matrix.

15. The one or more computer-readable media as recited in claim 14, wherein the transformation matrix includes a rotation matrix that rotates a direction from the point to the corresponding pixel and a translation vector that translates a position of the point to the corresponding pixel.

16. The one or more computer-readable media as recited in claim 13, wherein the operations further comprise identifying a surface in the three-dimensional model on which to project at least one of text, one or more graphics, or one or more images.

17. The one or more computer-readable media as recited in claim 13, wherein the second curved mirror reflects the entire environment.

18. A system comprising:

a projection device that is configured to transmit light corresponding to a projection onto a first curved mirror, wherein the first curved mirror is associated with the projection device and reflects the light corresponding to the projection onto one or more objects included in an environment; and

a stationary capture device directed at a second curved mirror and configured to capture an image of the environment, the second curved mirror being associated with the stationary capture device, wherein the stationary capture device and the projection device are oriented in the same direction.

19. The system as recited in claim 18, further comprising a computing device that is configured to generate a three-dimensional model of the environment based at least partly on the image.

20. The system as recited in claim 18, wherein one or more pixels associated with the image correspond to the one or more objects included in the environment.

21. The system as recited in claim 18, further comprising a computing device that is configured to:

determine an orientation and a location of the projection device, the first curved mirror, the stationary capture device, and the one or more objects included in the environment; and

generate a three-dimensional model of the environment based at least partly on the orientation and the location.

22. The system as recited in claim 21, wherein the three-dimensional model of the environment represents spatial relationships between the projection device, the first curved mirror, the stationary capture device, and the one or more objects included in the environment.

23. The system as recited in claim 18, wherein the stationary capture device is configured to capture the image without being rotated and without being moved.