Object customization and accessorization in video content

• US 9,747,727 B2
• Filed: 06/27/2014
• Issued: 08/29/2017
• Est. Priority Date: 03/11/2014
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Heinz, II, Gerard Joseph, Pesce, Michael Schleif, Davis, Collin Charles, Frazzini, Michael Anthony, Alkarmi, Ashraf, George, Michael Martin, Limp, David A., Carr, William Dugald
• Primary Examiner(s): Mazumder, Saptarshi
• Application Number:
  US14/318,002
• Publication Number:
  US 20150262402A1
• Time in Patent Office: 1,159 Days
• Field of Search: None
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

A real-time video exploration (RVE) system that allows users to pause, step into, move through, and explore 2D or 3D modeled worlds of scenes in a video. The RVE system may allow users to select and manipulate objects within a scene, and to modify an object by adding or removing accessories from the object or otherwise customizing the object according to the user'"'"'s preferences or desires. The RVE system may also provide an interface or interfaces via which the user can obtain additional information for the object, customize the object, be given a price or price(s) for the object as customized, and purchase the object as specified if desired.

Claims

1. A system, comprising:

one or more computing devices comprising one or more processors and one or more memories storing program instructions executable by the one or more processors to implement a real-time video exploration (RVE) system comprising;

a playback module implemented by at least one of the one or more computing devices and configured to begin playback of at least a portion of a pre-rendered video to a client device for display to a viewer associated with the client device; and

a graphics processing and rendering module implemented by at least one of the one or more computing devices and configured to, during playback of the pre-rendered video to the client device;

pause playback of the pre-rendered video to the client device in response to input from the client device;

receiving viewpoint input from the client device indicating a change of viewing angle, a viewpoint movement, or both, based on interactions of the viewer exploring one or more scenes in the pre-rendered video;

render and stream, to the client device, new video of the one or more scenes in the pre-rendered video in response to the viewpoint input, wherein the new video includes modified views of the one or more scenes in the pre-rendered video rendered from viewpoints that are not included in the pre-rendered video;

receive selection input from the client device selecting an object in the one or more scenes of the pre-rendered video;

receive modification input from the client device specifying one or more modifications to be applied to the selected object in the one or more scenes of the pre-rendered video;

modify a model of the object according to the one or more modifications to generate a modified model of the object;

render and stream, to the client device, the new video of the one or more scenes to include the modified model of the object in response to the modification input, wherein the selected object in the pre-rendered video is replaced with the modified model of the object; and

resume playback of the pre-rendered video to the client device in response to resume input from the client device specifying that the pre-rendered video is to be resumed, wherein at least one portion of the pre-rendered video is replaced with the new video of the one or more scenes.

2. The system as recited in claim 1, wherein the input specifying one or more modifications to be applied to the selected object specifies one or more customizations to be applied to the selected object, wherein a customization changes at least one feature of the selected object, and wherein the new video shows the selected object as customized.

3. The system as recited in claim 1, wherein the selected object comprises a three-dimensional (3D) object in the one or more scenes of the pre-rendered video, wherein the model of the object comprises a 3D model, and wherein, to modify the model of the object according to the one or more modifications to generate the modified model of the object, the graphics processing and rendering module is configured to:

obtain graphics data for the object from one or more sources, wherein the graphics data comprises 3D graphics data; and

generate the modified model of the object according to the obtained graphics data for the object.

4. The system as recited in claim 1, wherein the RVE system further comprises an interface component implemented by at least one of the one or more computing devices and configured to:

receive viewer input from the client device requesting information about the selected object;

obtain information about the selected object from one or more sources; and

send at least a portion of the obtained information about the selected object to the client device for display to the viewer associated with the client device.

5. The system as recited in claim 4, wherein the information sent to the client device includes information for ordering a version of the selected object, wherein the version of the selected object comprises a virtual representation of the selected object or a physical version of the selected object, and wherein the interface component is further configured to:

receive additional input from the client device ordering a version of the selected object as modified; and

facilitate generation of an order for the version of the selected object as modified according to the additional input.

6. The system as recited in claim 1, wherein the playback module is further configured to generate a user interface to be displayed at the client device, wherein the user interface is configured to:

receive the selection input identifying the object in the one or more scenes;

display a second user interface corresponding to the object;

receiving object input manipulating the second user interface; and

send the object input to the graphics processing and rendering module; and

wherein the graphics processing and rendering module is further configured to;

generate the second user interface; and

manipulate the second user interface in response to the object input.

7. The system as recited in claim 1, wherein the one or more computing devices that implement the RVE system are on a provider network, and wherein the graphics processing and rendering module is configured to leverage one or more computing resources of the provider network to perform said rendering in real time in response to the input modifying the selected object.

8. The system as recited in claim 1, wherein the input selecting the object and the input specifying one or more modifications to be applied to the selected object is received from the client device according to an application programming interface (API) of the RVE system.

9. The system as recited in claim 1, wherein the playback module is further configured to begin playback of at least a portion of the pre-rendered video to multiple other client devices, and wherein the graphics processing and rendering module is further configured to:

render and stream new video to the multiple other client devices based on the viewpoint input from the client device, the modified model of the object based on modification input from the client device, or both.

10. A method, comprising:

performing, by a real-time video exploration (RVE) system implemented on one or more computing devices;

sending at least a portion of a pre-recorded video to a client device for display to a viewer associated with the client device; and

while sending the pre-recorded video to the client device;

receiving viewpoint input from the client device indicating a change of viewing angle, a viewpoint movement, or both, based on interactions of the viewer exploring the one or more scenes in the pre-recorded video;

pausing the pre-recorded video in response to the viewpoint input;

rendering and sending new video of one or more scenes in the pre-recorded video to the client device for display to the viewer in response to the viewpoint input, wherein the new video includes modified views of the one or more scenes in the pre-recorded video rendered from viewpoints that are not included in the pre-recorded video;

receiving selection input from the client device selecting an object in a particular scene of the pre-recorded video;

receiving modification input from the client device indicating one or more modifications to be applied to the selected object in the particular scene of the pre-recorded video;

modifying a model of the object according to the one or more modifications to generate a modified model of the object;

rendering and sending new video of the particular scene including the modified model of the object to the client device;

rendering and sending new video of at least one other scene in the pre-recorded video including the modified model of the object to the client device for display to the viewer associated with the client device, wherein the selected object in the pre-recorded video is replaced with the modified model of the object; and

resuming playback of the pre-recorded video to the client device in response to resume input from the client device specifying that the pre-recorded video is to be resumed, wherein at least one portion of the pre-recorded video is replaced with the new video of the one or more scenes, the new video of the particular scene, the new video of the at least one other scene, or any combination thereof.

11. The method as recited in claim 10, wherein sending comprises streaming.

12. The method as recited in claim 10, wherein the input specifying one or more modifications to be applied to the selected object comprises an indication of a preference of the viewer associated with the client device.

13. The method as recited in claim 12, wherein the input selecting the object and the input specifying one or more modifications to be applied to the selected object is received from a seller of a physical version of the rendered object.

14. The method as recited in claim 10, wherein modifying a model of the object according to the one or more modifications to generate a modified model of the object comprises:

obtaining graphics data for the object from one or more sources; and

generating the modified model of the object according to the obtained graphics data for the object.

15. The method as recited in claim 10, further comprising:

receiving input from the client device requesting information about the selected object;

obtaining information about the selected object from one or more sources; and

sending at least a portion of the obtained information about the selected object to the client device for display.

16. The method as recited in claim 15, wherein the information sent to the client device includes information for ordering a version of the selected object, wherein the version of the selected object comprises a virtual representation of the selected object or a physical version of the selected object.

17. The method as recited in claim 16, further comprising:

receiving additional input from the client device ordering a version of the selected object as modified; and

generating an order for the version of the selected object as modified according to the additional input.

18. The method as recited in claim 10, further comprising receiving additional viewpoint input and additional modification input from one or more other client devices, wherein the additional viewpoint input is added to the viewpoint input received from the client device, and wherein the additional modification input is added to the modification input received from the client device.

19. A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement a real-time video exploration (RVE) system configured to:

send at least a portion of a pre-recorded video to a client device for display to a viewer associated with the client device; and

while sending the pre-recorded video to the client device;

receive viewpoint input from the client device indicating a change of viewing angle, a viewpoint movement, or both, based on interactions of the viewer exploring the one or more scenes in the pre-recorded video

render and send new video of one or more scenes in the pre-recorded video to the client device for display to the viewer in response to the viewpoint input, wherein the new video includes views of the one or more scenes in the pre-recorded video rendered from viewpoints that are not included in the pre-recorded video;

pause the pre-recorded video in response to input from the client device selecting an object in a particular scene of the pre-recorded video;

receive input from the client device indicating one or more modifications to be applied to the selected object in the particular scene;

modify a model of the object according to the one or more modifications to generate a modified model of the object;

render and send new video of the particular scene including the modified model of the object to the client device;

render and send new video of at least one other scene in the pre-recorded video including the modified model of the object to the client device for display to the viewer associated with the client device, wherein the selected object in the pre-recorded video is replaced with the modified model of the object; and

resume playback of the pre-recorded video to the client device in response to resume input from the client device specifying that the pre-recorded video is to be resumed, wherein at least one portion of the pre-recorded video is replaced with the new video of the one or more scenes, the new video of the particular scene, the new video of the at least one other scene, or any combination thereof.

20. The non-transitory computer-readable storage medium as recited in claim 19, wherein the RVE system is further configured to:

send additional information about the selected object to the client device for display, wherein the additional information includes information for ordering a physical version of the selected object; and

receive input from the client device ordering the selected object as modified.

21. The non-transitory computer-readable storage medium as recited in claim 19, wherein, to modify a model of the object according to the one or more modifications to generate a modified model of the object, the RVE system is configured to:

obtain graphics data for the object from one or more sources; and

generate the modified model of the object according to the obtained graphics data for the object;

wherein the selected object is a three-dimensional (3D) object, wherein the graphics data is 3D graphics data, and wherein the model of the object is a 3D model.