Real-time content filtering and replacement

• US 10,057,637 B2
• Filed: 07/31/2017
• Issued: 08/21/2018
• Est. Priority Date: 06/19/2015
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Kellner, Hans
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US15/664,870
• Publication Number:
  US 20170332137A1
• Time in Patent Office: 386 Days
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/45452 applied to an object-based stream, e.g. MPEG-4 streams
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for augmenting video streams, includes processing a video stream prior to presentation to identify objects to be selectively augmented in the video stream. The method includes: receiving and storing a first user selection of a preference for one or more aesthetic attributes to be associated with the first user and to be projected to a second user; receiving, at a second different device not associated with the first user, the preference for the one or more aesthetic attributes; identifying, at the second different device, image data associated with the first user and selectively augmenting the identified image data using the one or more aesthetic attributes; and presenting, on the second different device, the identified image data that is augmented by the one or more aesthetic attributes.

Claims

1. A computer-implemented method comprising:

at a first device, receiving a first user selection of a preference for one or more aesthetic attributes to be associated with the first user and to be projected to a second user capable of viewing either the first user or an object associated with the first user, and storing the preference for the one or more aesthetic attributes;

capturing an image of the first user;

evaluating the preference;

selectively augmenting the image of the first user using the one or more aesthetic attributes and presenting, on the first device, image data that is augmented by the one or more aesthetic attributes;

receiving, at a second different device not associated with the first user, the preference for the one or more aesthetic attributes;

identifying, at the second different device, image data associated with the first user and selectively augmenting the identified image data using the one or more aesthetic attributes; and

presenting, on the second different device, the identified image data that is augmented by the one or more aesthetic attributes.

2. The computer-implemented method of claim 1 wherein the one or more aesthetic attributes are attributes associated with clothes the first user is wearing, wherein the identified image data is an image of the first user as viewed by the second different device, and wherein augmenting the identified image data includes superimposing over or embellishing on clothes worn by the first user such that the one or more aesthetic attributes appear to be worn by the first user.

3. The computer-implemented method of claim 2 wherein the one or more aesthetic attributes constitute an outfit, appearance, or look that the first user has opted to project, and wherein augmenting the identified image data includes augmenting the identified image data associated with the first user to apply the outfit, appearance or look to the first user when viewed through the second different device.

4. The computer-implemented method of claim 1 wherein identifying image data includes capturing image data associated with the first user using a camera included in the second different device, and wherein augmenting the identified image data includes augmenting a captured image.

5. The computer-implemented method of claim 1 further comprising capturing an image of a scene that includes the first user, recognizing the first user in the captured scene, and augmenting a portion of the image of the captured scene that includes the first user to apply an outfit to the first user.

6. The computer-implemented method of claim 5 further comprising receiving at the second different device one or more additional aesthetic attributes associated with other users, recognizing one of the other users in the captured scene, and augmenting a portion of the image of the captured scene that includes the one other user based on the one or more additional aesthetic attributes for the one other user.

7. The computer-implemented method of claim 1 wherein selectively augmenting includes selectively controlling when to augment the identified image data including augmenting the identified image data when a control has been selected by the second user associated with the second different device to enable augmentation and further disabling augmentation when the control has been disabled.

8. The computer-implemented method of claim 1, wherein the preference for the one or more aesthetic attributes includes an object identifier, and augmenting the identified image data includes augmenting the identified image data using the object identifier to access a fully encapsulated representation to use in the augmenting.

9. The computer-implemented method of claim 8, wherein using the object identifier to access a fully encapsulated representation to use in the augmenting comprises:

transmitting the object identifier from device to device or through a central service; and

receiving one of at least two versions of the fully encapsulated representation based on a capability of the second different device.

10. A system comprising:

one or more processors; and

one or more memory devices including instructions that, when executed, cause the one or more processors to perform operations comprising

receiving and storing, at a first device, a first user selection of a preference for one or more aesthetic attributes to be associated with the first user and to be projected to a second user capable of viewing either the first user or an object associated with the first user,

capturing an image of the first user,

evaluating the preference,

selectively augmenting the image of the first user using the one or more aesthetic attributes and presenting, on the first device, image data that is augmented by the one or more aesthetic attributes,

receiving, at a second different device not associated with the first user, the preference for the one or more aesthetic attributes,

identifying, at the second different device, image data associated with the first user and selectively augmenting the identified image data using the one or more aesthetic attributes, and

presenting, on the second different device, the identified image data that is augmented by the one or more aesthetic attributes.

11. The system of claim 10 wherein the preference for the one or more aesthetic attributes includes an object identifier, and

augmenting the identified image data includes augmenting the identified image data using the object identifier to access a fully encapsulated representation to use in the augmenting.

12. The system of claim 11 wherein using the object identifier to access a fully encapsulated representation to use in the augmenting comprises:

transmitting the object identifier from device to device or through a central service; and

receiving one of at least two versions of the fully encapsulated representation based on a capability of the second different device.

13. A non-transitory computer-readable medium storing instructions, that when executed, cause one or more processors to perform operations comprising:

receiving and storing, at a first device, a first user selection of a preference for one or more aesthetic attributes to be associated with the first user and to be projected to a second user capable of viewing either the first user or an object associated with the first user;

capturing an image of the first user;

evaluating the preference;

selectively augmenting the image of the first user using the one or more aesthetic attributes and presenting, on the first device, image data that is augmented by the one or more aesthetic attributes;

receiving, at a second different device not associated with the first user, the preference for the one or more aesthetic attributes;

identifying, at the second different device, image data associated with the first user and selectively augmenting the identified image data using the one or more aesthetic attributes; and

presenting, on the second different device, the identified image data that is augmented by the one or more aesthetic attributes.

14. The non-transitory computer-readable medium of claim 13 wherein the one or more aesthetic attributes are attributes associated with clothes the first user is wearing, wherein the identified image data is an image of the first user as viewed by the second different device, and wherein augmenting the identified image data includes superimposing over or embellishing on clothes worn by the first user such that the one or more aesthetic attributes appear to be worn by the first user.

15. The non-transitory computer-readable medium of claim 14 wherein the one or more aesthetic attributes constitute an outfit, appearance, or look that the first user has opted to project, and wherein augmenting the identified image data includes augmenting the identified image data associated with the first user to apply the outfit, appearance or look to the first user when viewed through the second different device.

16. The non-transitory computer-readable medium of claim 13 wherein identifying image data includes capturing image data associated with the first user using a camera included in the second different device, and wherein augmenting the identified image data includes augmenting the captured image data using outfits data obtained by the second different device from a digital catalog for outfits for an online purchase system.

17. The non-transitory computer-readable medium of claim 16, wherein the preference for the one or more aesthetic attributes includes an object identifier for an outfit, and augmenting the identified image data includes augmenting the captured image data using the object identifier to access a fully encapsulated representation to use in the augmenting.

18. The non-transitory computer-readable medium of claim 17, wherein using the object identifier to access a fully encapsulated representation to use in the augmenting comprises:

transmitting the object identifier from device to device or through a central service; and

receiving one of at least two versions of the fully encapsulated representation based on a capability of the first device.

19. The non-transitory computer-readable medium of claim 13 wherein the operations further comprise capturing an image of a scene that includes the first user, recognizing the first user in the captured image of the scene, and augmenting a portion of the captured image of the scene that includes the first user to apply an outfit to the first user.

20. The non-transitory computer-readable medium of claim 19 wherein the operations further comprise receiving, at the second different device, one or more additional aesthetic attributes associated with other users, recognizing one of the other users in the captured image of the scene, and augmenting a portion of the captured image of the scene that includes the one other user based on the one or more additional aesthetic attributes for the one other user.