Model generation and database

• US 9,094,670 B1
• Filed: 09/25/2012
• Issued: 07/28/2015
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Furio, Paul J., Tanner, Kesler Williams
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Huang, Frank
• Application Number:
  US13/626,620
• Time in Patent Office: 1,036 Days
• Field of Search: 348/47
• US Class Current: 1/1
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 2213/006 Pseudo-stereoscopic systems, i.e. systems wherein a stereoscopic effect is obtained without sending different images to the viewer's eyes

Abstract

Various embodiments enable a user to capture image information with one or more cameras of a computing device for generating a three-dimensional model of an object and further to enable subsequent users to refine the model by submitting additional image information. For example, an ecommerce service could receive image information from a user for a product offered for sale that was captured with a computing device. With the image information, the service can generate a three-dimensional model of the product and provide the model for display through a website for customers interested in the product to view.

Claims

1. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

presenting a display by a first portable computing device to capture a plurality of images of an item at different points of view relative to the item;

receiving the plurality of images captured in response to the prompting;

generating a three-dimensional model of the item using information obtained from the plurality of images;

storing the three-dimensional model in a database containing a plurality of three-dimensional models;

determining that the three-dimensional model does not meet at least one model criterion;

presenting a second display by a second portable computing device to capture at least one image of a view of a same type of item; and

generating an updated three-dimensional model with information obtained from the at least one image captured by the second portable computing device;

wherein at least one of the presenting the display by the first portable computing device and presenting the second display by the second portable computing device comprises;

presenting instructions on a display of the first portable computing device or on a display of the second portable computing device to position the first portable computing device or the second portable computing device relative to the item when the updated model is incomplete, the instructions based, at least in part, on orientation information obtained from the first portable computing device or the second portable computing device.

2. The computer-implemented method of claim 1, wherein at least one of the first portable computing device or the second computing device includes a first camera having a first field of view and a second camera having a second field of view, the second field of view at least partially overlapping the first field of view.

3. The computer-implemented method of claim 1, further comprising:

presenting a third display by the first portable computing device, by the second portable computing device, or by a third personal computing device to capture additional images of the same type of item until a quality factor of the updated three-dimensional model exceeds a quality factor threshold.

4. The computer-implemented method of claim 1, further comprising:

identifying a portion of the updated three-dimensional model for which a quality factor is below a quality factor threshold; and

presenting instructions on the display of the first portable computing device or on the display of the second portable computing device to capture at least one image that includes a view of the item corresponding to the identified portion.

5. The computer-implemented method of claim 1, wherein the at least one model criterion includes a number of images used to generate the three-dimensional model, an image resolution of the model, or a number of data points of the model.

24. The computer-implemented method of claim 1, wherein the second portable computing device is the first portable computing device or a third portable computing device.

6. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

receiving, from a portable computing device, information associated with an item;

identifying a three-dimensional model of the item based on the received information associated with the item;

receiving at least one image of the item captured with the portable computing device;

generating an updated three-dimensional model with information obtained from the at least one image; and

presenting instructions on a display of the portable computing device to position the portable computing device relative to the item where the updated three-dimensional model is incomplete, the instructions based, at least in part, on orientation information obtained from the portable computing device.

7. The computer-implemented method of claim 6, further comprising:

presenting instructions on the display of the portable computing device to capture the at least one image of the item with the portable computing device.

8. The computer-implemented method of claim 7, further comprising:

receiving orientation information indicating an orientation of the computing device at approximately the time the at least one image is captured;

determining a type of view associated with the orientation; and

presenting instructions on the display of the portable computing device to capture at least one additional image of a view of the item at an orientation different from the orientation associated with the determined type of view.

9. The computer-implemented method of claim 7, further comprising:

presenting an additional display by an additional portable communications device to capture additional image views of the item until the updated three-dimensional model reaches a determined quality threshold.

10. The computer-implemented method of claim 6, wherein receiving information for the item includes receiving at least one of a barcode, a QR code, product identifier, an optical character recognizer, or by image information associated with the item.

11. The computer-implemented method of claim 6, wherein the item is a product available for consumption through an ecommerce website.

12. The computer-implemented method of claim 6, wherein the portable computing device includes a first camera having a first field of view and a second camera having a second field of view, the second field of view at least partially overlapping the first field of view.

13. A computing device, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the computing device to;

receive image information for an object captured with a computing device;

generate a three-dimensional model of the object using the image information;

presenting instructions to position the computing device relative to the object when the three-dimensional model is incomplete, the instructions based, at least in part, on orientation information obtained from the computing device;

from each of a set of additional computing devices, obtain additional image information for the three-dimensional model; and

refine the three-dimensional model using the additional image information until a quality factor reaches a quality factor threshold.

14. The computing device of claim 13, wherein the image information includes information from a first camera and a second camera of the computing device for generating a stereoscopic image of the model using depth information.

15. The computing device of claim 13, wherein the set of additional computing devices display instructions to capture the additional image information of the item until the quality factor of the three-dimensional model reaches the quality factor threshold.

16. The computing device of claim 13, wherein refining the three-dimensional model using the additional image information includes at least one of filling in gaps in the model, adding data points to the model, or improving quality of the model.

17. The computing device of claim 13, wherein the object is a product available for consumption through an ecommerce website.

18. The computing device of claim 13, wherein the quality factor includes at least one of a number of images used to generate the three-dimensional model, an image resolution of the model, or number of data points of the model.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive, from a portable computing device, information associated with an item;

identify a three-dimensional model of the item based on the received information associated with the item;

receive at least one image of the item captured with the portable computing device;

update the three-dimensional model with information obtained from the at least one image; and

present instructions on a display of the computing device to position the computing device relative to the item where the three-dimensional model is incomplete, the instructions are based, at least in part, on orientation information obtained from the computing device.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions that, when executed by the at least one processor of the computing device, further cause the computing device to:

present instructions on the display of the computing device to capture at least one image of the item with the computing device.

21. The non-transitory computer-readable storage medium of claim 19, wherein the image information includes information from a first camera and a second camera of the computing device for generating a stereoscopic image of the model using depth information.

22. The non-transitory computer-readable storage medium of claim 19, wherein receiving information for the item includes receiving at least one of a barcode, a QR code, product identifier, an optical character recognizer, or image information associated with the item.

23. The non-transitory computer-readable storage medium of claim 19, wherein the item is a product available for consumption through an ecommerce website.