Interaction analysis systems and methods

• US 10,699,487 B2
• Filed: 08/31/2018
• Issued: 06/30/2020
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Soon-Shiong, Patrick
• Primary Examiner(s): Bashore, William L
• Assistant Examiner(s): Shrewsbury, Nathan K
• Application Number:
  US16/119,857
• Publication Number:
  US 20180374272A1
• Time in Patent Office: 669 Days
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/00 Image analysis

Abstract

Interaction-based ecosystems are presented. Interaction analysis engine analyze media content to derive a set of media features. The engine can then identify one or more interaction objects (e.g., transactions, searches, game play, etc.) based on the set of media features. Relevant interaction objects can then be instantiated as persistent available or active points of interaction readily accessed by a consumer. The consumer need only capture a digital representation of the content via a user device, a smart phone for example. A second set of media features can be derived from the digital representation and the second set of media features can then be used to find the instantiated interactions.

Claims

1. An interaction analysis system comprising:

one or more processors;

an interaction database storing interaction objects, each interaction object comprising criteria defined based on media features and a location of an electronic device; and

at least one memory storing software instructions that, when executed by the one or more processors, cause the one or more processors to;

obtain media content via a media input interface;

identify a first set of digital media attributes obtained from the media content;

determine at least one interaction object having criteria satisfied based on the set of digital media attributes and the location of the electronic device;

instantiate an interaction instance in memory according to the at least one interaction object;

configure the electronic device to support an interaction with a user according to the interaction instance, wherein the interaction instance is based at least in part on user data;

receive a captured digital representation of the media content;

obtain a second set of digital media attributes from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

2. The system of claim 1, wherein the interaction object comprises a template interaction object.

3. The system of claim 1, wherein the media content comprises at least one of the following:

a stream, a file, a record, a transmission, a broadcast, a poster, a billboard, and a magazine.

4. The system of claim 1, wherein the media content comprises at least one of the following data modalities:

audio data, image data, video data, kinesthetic data, medical data, and metadata.

5. The system of claim 1, wherein the media content comprises at least one of a stream, a file, a record, a transmission, or a broadcast and the media content lacks interaction annotation metadata.

6. The system of claim 1, wherein the media content comprises content received substantially in real-time from an external source.

7. The system of claim 1, wherein at least one member of the set of digital media attributes depends on time.

8. The system of claim 7, wherein the at least one member of the set of digital media attributes depends on a specified time.

9. The system of claim 8, wherein the specified time comprises a relative time.

10. The system of claim 1, wherein the interaction object criteria comprise optional conditions.

11. The system of claim 1, wherein the interaction instance is instantiated as a function of a device attribute associated with the electronic device.

12. The system of claim 11, wherein the device attributes include at least one of the following:

a user identifier, a device identifier, a device location, a device time, a device movement, a user preference, device sensor data, a device state, and a device capability.

13. The system of claim 1, wherein the interaction instance is hosted by the interaction analysis engine.

14. The system of claim 1, wherein the interaction instance is hosted by a third party device.

15. The system of claim 1, wherein the interaction instance is hosted by the electronic device.

16. The system of claim 1, wherein the interaction instance represents a transient interaction as the interaction.

17. The system of claim 16, wherein the transient interaction is active for a limited time.

18. The system of claim 1, further comprising an interaction interface coupled with the interaction database and configured to allow a user to define interaction objects.

19. The system of claim 1, wherein the interaction analysis engine is further configured to offer access to the interaction in exchange for a fee.

20. The system of claim 19, wherein the fee comprises at least one of the following:

an auction bid, a user paid fee, an advertiser fee, a subscription fee, a sponsor fee, and a per-interaction fee.

21. The system of claim 1, wherein the interaction comprises at least one of the following:

a transaction, a product purchase, a search, a comparison of items, playing a game, and issuing a command to a device.

22. The system of claim 1, wherein the electronic device comprises at least one of the following:

a server, a mobile phone, a kiosk, an appliance, a set top box, a vehicle, a point of sales device, a tablet, and a computer.

23. The system of claim 1, wherein the interaction instance comprises interaction activation criteria defined based on the set of digital media attributes.

25. The system of claim 23, wherein the electronic device is the recognition platform.

28. The system of claim 1, the interaction object criteria comprising a set of requirements that must be satisfied to indicate that the at least one interaction object can be associated with the media content.

29. The system of claim 1, the interaction object criteria comprising one or more media attribute values, the one or more media values being at least one of a threshold and a range.

30. The system of claim 1, the interaction object criteria comprising a pointer to a second interaction object, the second interaction object stored in the interaction database and being a subset of the interaction objects.

31. The system of claim 1, the selecting at least one interaction object having criteria satisfied by the set of digital media attributes comprising including the set of digital media attributes in a query of the interaction database.

32. The system of claim 1, wherein the interaction object criteria comprise at least one of logical connectors AND, OR, XOR, NOT, or NAND.

33. The system of claim 1, wherein the interaction object criteria comprise position data.

34. The system of claim 1, wherein a namespace of the media features represents image features.

35. The system of claim 1, wherein the user data comprises at least one of:

historical user behavior, user attribute information, gaming interaction information, interaction information, and game player account information.

24. An interaction analysis system comprising:

one or more processors;

an interaction database storing interaction objects, each interaction object comprising criteria defined based on media features and a location of an electronic device; and

at least one memory storing software instructions that, when executed by the one or more processors, cause the one or more processors to;

obtain media content via a media input interface;

identify a first set of digital media attributes obtained from the media content;

determine at least one interaction object having criteria satisfied based on the set of digital media attributes and the location of the electronic device;

instantiate an interaction instance in memory according to the at least one interaction object, wherein the interaction instance comprises interaction activation criteria defined based on the first set of digital media attributes; and

configure the electronic device to support an interaction with a user according to the interaction instance; and

a recognition platform coupled with the electronic device and configured to;

receive a captured digital representation of the media content;

obtain a second set of digital media attributes from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

26. An interaction analysis system comprising:

one or more processors;

an interaction database storing interaction objects, each interaction object comprising criteria defined based on media features and a location of an electronic device; and

at least one memory storing software instructions that, when executed by the one or more processors, cause the one or more processors to;

obtain media content via a media input interface;

identify a first set of digital media attributes obtained from the input media content;

determine at least one interaction object having criteria satisfied based on the set of digital media attributes and the location of the electronic device;

instantiate an interaction instance in memory according to the at least one interaction object; and

configure the electronic device to support an interaction with a user according to the interaction instance, wherein the interaction instance is based at least in part on user data, without modifying the media content, wherein the electronic device is configured to;

receive a captured digital representation of the media content;

obtain a second set of digital media attributes from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

36. The system of claim 26, wherein the user data comprises at least one of:

historical user behavior, user attribute information, gaming interaction information, interaction information, and game player account information.

27. An interaction analysis system comprising:

one or more processors;

an interaction database storing interaction objects, each interaction object comprising criteria defined based on media features and a location of at least one of a first electronic device and a second electronic device; and

at least one memory storing software instructions that, when executed by the one or more processors, cause the one or more processors to;

receive input media content via a media input interface;

identify a first set of digital media features derived from the input media content;

determine at least one interaction object having relevancy criteria satisfied based on the set of digital media features and the location of the electronic device;

instantiate an interaction instance in memory according to the at least one interaction object;

configure the first electronic device to support a first interaction with a first user and the second electronic device to support a second interaction with a second user according to the interaction instance, wherein the first interaction is based at least in part on data relating to the first user and the second interaction is based at least in part on data relating to the second user;

receive a captured digital representation of the media content;

obtain a second set of digital media attributes from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

37. The system of claim 27, wherein the data relating to the first and second users comprises at least one of:

historical user behavior, user attribute information, gaming interaction information, interaction information, and game player account information.

38. An interaction analysis method comprising:

obtaining media content via a media input interface;

identifying a first set of digital media attributes obtained from the media content;

determining at least one interaction object having criteria satisfied based on the set of digital media attributes and the location of an electronic device;

instantiating an interaction instance in a memory according to the at least one interaction object; and

configuring the electronic device to support an interaction with a user according to the interaction instance, wherein the interaction instance is based at least in part on user data;

receiving a captured digital representation of the media content;

obtaining a second set of digital media attributes from the digital representation;

recognizing the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activating the interaction corresponding to the interaction instance.

39. The method of claim 38, wherein the user data comprises at least one of:

historical user behavior, user attribute information, gaming interaction information, interaction information, and game player account information.

40. A non-transitory computer-readable medium having computer instructions stored thereon, which, when executed by a one or more processors, cause the one or more processors to perform one or more operations comprising:

obtaining media content via a media input interface;

identifying a first set of digital media attributes obtained from the media content;

determining at least one interaction object having criteria satisfied based on the set of digital media attributes and the location of an electronic device;

instantiating an interaction instance in a memory according to the at least one interaction object;

configuring the electronic device to support an interaction with a user according to the interaction instance, wherein the interaction instance is based at least in part on user data, and the electronic device is configured to

receive a captured digital representation of the media content;

obtain a second set of digital media attributes from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media attributes satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

41. The non-transitory computer-readable medium of claim 40, wherein the user data comprises at least one of:

historical user behavior, user attribute information, gaming interaction information, interaction information, and game player account information.