Tagging virtualized content

• US 10,679,061 B2
• Filed: 06/07/2019
• Issued: 06/09/2020
• Est. Priority Date: 07/09/2013
• Status: Active Grant

• Current Assignee: Outward, Inc. (Williams-Sonoma Inc.)
• Original Assignee: Outward, Inc. (Williams-Sonoma Inc.)
• Inventors: Chui, Clarence
• Primary Examiner(s): Rush, Eric
• Application Number:
  US16/435,376
• Publication Number:
  US 20190362152A1
• Time in Patent Office: 368 Days
• Field of Search: 382100, 382154, 382159, 382160, 382180, 382181, 382209, 382224, 382228, 382229, 382285, 382291, 345419, 345420, 345581, 715230, 715231
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Techniques for tagging virtualized content are disclosed. In some embodiments, a modeled three-dimensional scene of objects representing abstracted source content is generated and analyzed to determine a contextual characteristic of the scene that is based on a plurality of objects comprising the scene. The modeled scene is tagged with a tag specifying the determined contextual characteristic.

Claims

1. A method, comprising:

generating a modeled scene to represent source content by mapping elements identified in the source content to existing three-dimensional database objects, wherein the modeled scene comprises a modified and virtualized three-dimensional version of the source content;

determining a metric associated with the modeled scene based on at least a subset of the three-dimensional objects comprising the modeled scene and their positions in the modeled scene;

inferring at least a partial semantic understanding of the modeled scene based on interpreting the determined metric; and

tagging the modeled scene with a tag specifying the at least partial semantic understanding of the modeled scene inferred from the determined metric.

2. The method of claim 1, wherein the method is completely automated.

3. The method of claim 1, further comprising providing a specification of the modeled scene instead of the source content in response to a request for the source content.

4. The method of claim 1, wherein determining the metric associated with the modeled scene comprises statistically analyzing data defining the modeled scene.

5. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene comprises a contextual characteristic associated with the modeled scene.

6. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene comprises an action or behavior associated with the modeled scene.

7. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on a number or density of objects comprising the modeled scene.

8. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on types or classes of objects comprising the modeled scene.

9. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on spatial relationships of objects comprising the modeled scene.

10. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on correlations of objects comprising the modeled scene.

11. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on motions of objects comprising the modeled scene.

12. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on time evolutions of objects comprising the modeled scene.

13. The method of claim 1, wherein the at least partial semantic understanding of the modeled scene is based on inter-class or intra-class spatial correlations of objects comprising the modeled scene.

14. The method of claim 1, wherein the tag is associated with an object included in the modeled scene or a frame comprising the modeled scene.

15. The method of claim 1, wherein the tag is inherited by a frame derived from the modeled scene.

16. The method of claim 1, further comprising determining an advertisement to be provided with respect to the modeled scene based on the at least partial semantic understanding of the modeled scene.

17. The method of claim 1, wherein the tagged modeled scene is searchable via the tag.

18. The method of claim 1, wherein the source content comprises two-dimensional content.

19. A system, comprising:

a processor configured to;

generate a modeled scene to represent source content by mapping elements identified in the source content to existing three-dimensional database objects, wherein the modeled scene comprises a modified and virtualized three-dimensional version of the source content;

determine a metric associated with the modeled scene based on at least a subset of the three-dimensional objects comprising the modeled scene and their positions in the modeled scene;

infer at least a partial semantic understanding of the modeled scene based on interpreting the determined metric; and

tag the modeled scene with a tag specifying the at least partial semantic understanding of the modeled scene inferred from the determined metric; and

a memory coupled to the processor and configured to provide the processor with instructions.

20. The system of claim 19, wherein the processor is further configured to provide a specification of the modeled scene instead of the source content in response to a request for the source content.

21. The system of claim 19, wherein to determine the metric associated with the modeled scene comprises to statistically analyze data defining the modeled scene.

22. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene comprises a contextual characteristic associated with the modeled scene.

23. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene comprises an action or behavior associated with the modeled scene.

24. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on a number or density of objects comprising the modeled scene.

25. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on types or classes of objects comprising the modeled scene.

26. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on spatial relationships of objects comprising the modeled scene.

27. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on correlations of objects comprising the modeled scene.

28. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on motions of objects comprising the modeled scene.

29. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on time evolutions of objects comprising the modeled scene.

30. The system of claim 19, wherein the at least partial semantic understanding of the modeled scene is based on inter-class or intra-class spatial correlations of objects comprising the modeled scene.

31. The system of claim 19, wherein the tag is associated with an object included in the modeled scene or a frame comprising the modeled scene.

32. The system of claim 19, wherein the tag is inherited by a frame derived from the modeled scene.

33. The system of claim 19, wherein the processor is further configured to determine an advertisement to be provided with respect to the modeled scene based on the at least partial semantic understanding of the modeled scene.

34. The system of claim 19, wherein the tagged modeled scene is searchable via the tag.

35. The system of claim 19, wherein the source content comprises two-dimensional content.

37. The computer program product of claim 20, further comprising computer instructions which when executed cause a computer to provide a specification of the modeled scene instead of the source content in response to a request for the source content.

38. The computer program product of claim 20, wherein to determine the metric associated with the modeled scene comprises to statistically analyze data defining the modeled scene.

39. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene comprises a contextual characteristic associated with the modeled scene.

40. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene comprises an action or behavior associated with the modeled scene.

41. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on a number or density of objects comprising the modeled scene.

42. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on types or classes of objects comprising the modeled scene.

43. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on spatial relationships of objects comprising the modeled scene.

44. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on correlations of objects comprising the modeled scene.

45. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on motions of objects comprising the modeled scene.

46. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on time evolutions of objects comprising the modeled scene.

47. The computer program product of claim 20, wherein the at least partial semantic understanding of the modeled scene is based on inter-class or intra-class spatial correlations of objects comprising the modeled scene.

48. The computer program product of claim 20, wherein the tag is associated with an object included in the modeled scene or a frame comprising the modeled scene.

49. The computer program product of claim 20, wherein the tag is inherited by a frame derived from the modeled scene.

50. The computer program product of claim 20, further comprising computer instructions which when executed cause a computer to determine an advertisement to be provided with respect to the modeled scene based on the at least partial semantic understanding of the modeled scene.

51. The computer program product of claim 20, wherein the tagged modeled scene is searchable via the tag.

52. The computer program product of claim 20, wherein the source content comprises two-dimensional content.

36. A computer program product stored on a non-transitory computer readable storage medium and comprising computer instructions which when executed cause a computer to:

generate a modeled scene to represent source content by mapping elements identified in the source content to existing three-dimensional database objects, wherein the modeled scene comprises a modified and virtualized three-dimensional version of the source content;

determine a metric associated with the modeled scene based on at least a subset of the three-dimensional objects comprising the modeled scene and their positions in the modeled scene;

infer at least a partial semantic understanding of the modeled scene based on interpreting the determined metric; and

tag the modeled scene with a tag specifying the at least partial semantic understanding of the modeled scene inferred from the determined metric.