Vehicle component installation preview image generation

• US 9,928,544 B1
• Filed: 03/10/2015
• Issued: 03/27/2018
• Est. Priority Date: 03/10/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hasan, Shiblee Imtiaz
• Primary Examiner(s): Williams, Kimberly A
• Application Number:
  US14/644,030
• Time in Patent Office: 1,113 Days
• Field of Search: 345419
• CPC Class Codes: G06F 30/15 Vehicle, aircraft or watercraft design
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 2219/2008 Assembling, disassembling
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Systems and processes described herein can identify interface points for an item in an image and generate a preview image that illustrates one or more additional items interfacing with at least one of the identified interface points. The system described herein can use augmented reality to present the preview image of an additional item connected to the item at the at least one identified interface point. Further, the system described herein can use a three-dimensional model created based at least in part on measurements of the additional item and a determination of the size of the item from context or reference information within the image to confirm that the additional item will fit the interface point of the item within a threshold degree of fit.

Claims

1. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

accessing an image from a continuous stream of images that depicts at least a portion of a vehicle;

analyzing the image to determine spatial characteristics of the portion of the vehicle;

identifying a vehicle interface depicted in the image, the vehicle interface comprising a connection location at which one or more items can be connected to the vehicle;

annotating the image with an identification of a location of the vehicle interface to obtain an annotated image;

providing the annotated image to a user device for presentation to a user while the user device is capturing subsequent images from the continuous stream of images;

receiving a selection of an item for potential connection with the vehicle interface;

accessing a three-dimensional (3D) model for the item;

accessing a 3D model for the vehicle, wherein the 3D model for the vehicle represents at least the portion of the vehicle;

transforming the 3D model for the item and the 3D model for the vehicle based at least in part on the spatial characteristics of the portion of the vehicle;

creating a 3D image scene by positioning the 3D model for the item with respect to the 3D model for the vehicle such that an item interface of the item is aligned with respect to the vehicle interface;

identifying a second vehicle interface, wherein creating the 3D image scene further comprises positioning a second instance of the 3D model for the item with respect to the 3D model for the vehicle based at least in part on the second vehicle interface; and

generating a display image based at least in part on a rendering of the 3D image scene.

2. The computer-implemented method of claim 1, wherein the vehicle interface is located between at least two components of the vehicle.

3. The computer-implemented method of claim 2, wherein creating the 3D image scene further comprises removing a portion of the 3D model for the vehicle corresponding to a component from the at least two components.

4. The computer-implemented method of claim 1, wherein the vehicle interface is identified based at least in part on the selection of the item.

5. The computer-implemented method of claim 1, further comprising identifying a vehicle component included in the portion of the vehicle based at least in part on context information included in the image.

6. The computer-implemented method of claim 5, wherein the spatial characteristics of the portion of the vehicle are determined based on the identified vehicle component and without using a reference marker.

7. The computer-implemented method of claim 5, further comprising:

determining whether the vehicle component comprises an original component, wherein the original component comprises a component included by the manufacturer of the vehicle; and

in response to determining that the vehicle component is not an original component;

identifying the vehicle component; and

accessing a 3D model for the vehicle component, wherein creating the 3D image scene further comprises positioning the 3D model for the vehicle component with respect to the 3D model for the vehicle based at least in part on a location of a depiction of the vehicle component in the image.

8. The computer-implemented method of claim 7, wherein the vehicle interface is on the vehicle component.

9. The computer-implemented method of claim 1, wherein transforming the 3D model for the item and the 3D model for the vehicle comprises rotating, translating, and/or scaling the 3D model for the item and/or the 3D model for the vehicle.

10. The computer-implemented method of claim 1, wherein vehicle interface dimensions of the vehicle interface are determined based on an identity of the vehicle, and wherein a set of items available for selection by a user are filtered based at least in part on the vehicle interface dimensions.

11. A system comprising:

an electronic data store configured to at least store specific computer-executable instructions; and

an interactive computing system comprising computer hardware in communication with the electronic data store, the interactive computing system configured to execute the specific compute-executable instructions to at least;

access an image that depicts at least a portion of a vehicle;

analyze the image to identify the vehicle;

receive a selection of an item for potential addition to the portion of the vehicle;

determine an item type of the selected item;

identify a plurality of vehicle connection interfaces based at least in part on the item type, wherein the item type is designed to attach to the vehicle at one or more of the plurality of vehicle connection interfaces;

access a three-dimensional (3D) model for the selected item;

access a 3D model for the vehicle;

scale the 3D model for the vehicle and the 3D model for the selected item based at least in part on the scale of the portion of the vehicle depicted in the image;

create an annotated image corresponding to the image, wherein a location of two or more of the plurality of vehicle connection interfaces are annotated in the annotated image;

provide the annotated image to a user device for presentation to a user;

receive a selection of at least one vehicle connection interface from the annotated vehicle connection interfaces;

create a 3D image scene based at least in part on a placement of the 3D model for the selected item with respect to the 3D model for the vehicle, wherein creating the 3D image scene further comprises placing one or more instances of the 3D model of the selected item with respect to the 3D model for the vehicle based at least in part on the received selection of the at least one vehicle connection interface; and

render a display image based at least in part on the 3D image scene.

12. The system of claim 11, wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least determine whether the selected item fits the vehicle by determining whether the 3D model for the selected item mates with the 3D model for the vehicle at the at least one vehicle connection interface within a threshold degree of error.

13. The system of claim 12, wherein the threshold degree of error is based at least in part on the item type and/or the material properties of the selected item.

14. The system of claim 12, wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least annotate the at least one vehicle connection interface within the display image to indicate whether the selected item fits the vehicle.

15. The system of claim 12, wherein, in response to determining that the selected item does not fit the vehicle, the interactive computing system is further configured to execute the specific computer-executable instructions to at least:

identify an alternative item that does fit the vehicle; and

output a recommendation of the alternative item.

16. The system of claim 11, wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least modify a view of the display image corresponding to movement of the user device that captures a continuous stream of images of one or more portions of the vehicle, wherein movement of the user device is detected based at least in part on a comparison of the image with a subsequent image from the continuous stream of images.

17. The system of claim 11, wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least:

determine, based at least in part on the image, whether the vehicle includes an aftermarket component, wherein the aftermarket component comprises a component that is not provided by the manufacturer of the vehicle and that modifies a manufacturer specification for the vehicle; and

in response to determining that the vehicle includes an aftermarket component the interactive computing system is further configured to execute the specific computer-executable instructions to at least;

identify the aftermarket vehicle component; and

access a 3D model for the aftermarket vehicle component, wherein creating the 3D image scene further comprises positioning the 3D model for the aftermarket vehicle component with respect to the 3D model for the vehicle based at least in part on a location of a depiction of the aftermarket vehicle component in the image.

18. The system of claim 17, wherein the interactive computing system is further configured to execute the specific computer-executable instructions to at least generate the 3D model for the aftermarket vehicle component based at least in part on a portion of the image that depicts the aftermarket vehicle component.

19. A computer-readable, non-transitory storage medium storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:

accessing a continuous stream of images that depict at least a portion of a vehicle;

analyzing a frame of the continuous stream of images to identify the vehicle;

identifying one or more vehicle connection nodes viewable in the frame;

annotating the frame with a location of the one or more vehicle connection nodes to obtain an annotated frame;

outputting, for presentation on a user device, the annotated frame;

receiving a selection of a vehicle connection node from the one or more vehicle connection nodes;

outputting, for presentation on the user device, a representation of one or more vehicle components capable of being attached to the selected vehicle connection node;

receiving a selection of a vehicle component from the one or more vehicle components;

receiving a selection of a second vehicle connection node;

accessing a three-dimensional (3D) model for the vehicle component;

accessing a 3D model for the vehicle;

scaling the 3D model for the vehicle and the 3D model for the selected vehicle component based at least in part on the scale of the portion of the vehicle depicted in the frame; and

rendering a 3D image scene based at least in part on a placement of the 3D model for the selected vehicle component with respect to the 3D model for the vehicle, wherein the placement of the 3D model for the selected vehicle component is determined based at least in part on the selected vehicle connection node, and wherein rendering the 3D image scene further comprises positioning a second instance of the 3D model for the selected vehicle component based at least in part on the second selected vehicle connection node.

20. The computer-readable, non-transitory storage medium of claim 19, wherein the operations further comprise performing a transformation on the rendered 3D image scene to obtain a two dimensional (2D) image.

21. The computer-readable, non-transitory storage medium of claim 19, wherein the operations further comprise updating the 3D image scene based at least in part on movement of the user device.