Interaction analysis systems and methods

• US 10,068,384 B2
• Filed: 07/12/2017
• Issued: 09/04/2018
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Soon-Shiong, Patrick
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Shrewsbury, Nathan
• Application Number:
  US15/648,411
• Publication Number:
  US 20170316615A1
• Time in Patent Office: 419 Days
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/00 Image analysis

Abstract

Interaction-based ecosystems are presented. Interaction analysis engine analyze media content to derive a set of media features. The engine can then identify one or more interaction objects (e.g., transactions, searches, game play, etc.) based on the set of media features. Relevant interaction objects can then be instantiated as persistent available or active points of interaction readily accessed by a consumer. The consumer need only capture a digital representation of the content via a user device, a smart phone for example. A second set of media features can be derived from the digital representation and the second set of media features can then be used to find the instantiated interactions.

Claims

1. An interaction analysis system comprising:

a processor;

an interaction database storing interaction objects, each interaction object comprising relevancy criteria defined based on media features, each of the interaction objects stored in the interaction database comprising one or more fields populated when each of the interaction objects is instantiated, the one or more fields determined as a function of a namespace of the media features; and

an interaction analysis engine coupled with the interaction database and configured to;

receive input media content via a media input interface;

identify a set of digital media features derived from the input media content;

select at least one interaction object having relevancy criteria satisfied by the set of digital media features;

instantiate an interaction instance in memory according to the at least one interaction object; and

configure an electronic device to support an interaction with a user according to the interaction instance.

2. The system of claim 1, wherein the interaction object comprises a template interaction object.

3. The system of claim 1, wherein the input media content comprises at least one of the following:

a stream, a file, a record, a transmission, a broadcast, a poster, a billboard, and a magazine.

4. The system of claim 1, wherein the input media content comprises at least one of the following data modalities:

audio data, image data, video data, kinesthetic data, medical data, and metadata.

5. The system of claim 1, wherein the input media content comprises at least one of a stream, a file, a record, a transmission, or a broadcast and the input media content lacks interaction annotation metadata.

6. The system of claim 1, wherein the input media content comprises content received substantially in real-time from an external source.

7. The system of claim 1, wherein at least one member of the set of digital media features depends on time.

8. The system of claim 7, wherein the at least one member of the set of digital media features depends on a specified time.

9. The system of claim 8, wherein the specified time comprises a relative time.

10. The system of claim 1, wherein the interaction relevancy criteria comprises optional conditions.

11. The system of claim 1, wherein the interaction instance is instantiated as a function of a device attribute associated with the electronic device.

12. The system of claim 11, wherein the device attributes include at least one of the following:

a user identifier, a device identifier, a device location, a device time, a device movement, a user preference, device sensor data, a device state, and a device capability.

13. The system of claim 1, wherein the interaction instance is hosted by the interaction analysis engine.

14. The system of claim 1, wherein the interaction instance is hosted by a third party device.

15. The system of claim 1, wherein the interaction instance is hosted by the electronic device.

16. The system of claim 1, wherein the interaction instance represents a transient interaction as the interaction.

17. The system of claim 16, wherein the transient interaction is active for a limited time.

18. The system of claim 1, further comprising an interaction interface coupled with the interaction database and configured to allow a user to define interaction objects.

19. The system of claim 1, wherein the interaction analysis engine is further configured to offer access to the interaction in exchange for a fee.

20. The system of claim 19, wherein the fee comprises at least one of the following:

an auction bid, a user paid fee, an advertiser fee, a subscription fee, a sponsor fee, and a per-interaction fee.

21. The system of claim 1, wherein the interaction comprises at least one of the following:

a transaction, a product purchase, a search, a comparison of items, playing a game, and issuing a command to a device.

22. The system of claim 1, wherein the electronic device comprises at least one of the following:

a server, a mobile phone, a kiosk, an appliance, a set top box, a vehicle, a point of sales device, and a computer.

23. The system of claim 1, wherein the interaction instance comprises interaction activation criteria defined based on the set of digital media features.

24. The system of claim 23, further comprising a recognition platform coupled with the electronic device and configured to:

receive a captured digital representation of the input media content;

derive a second set of digital media features from the digital representation;

recognize the interaction instance as being available to a user upon the second set of digital media features satisfying the activation criteria; and

activate the interaction corresponding to the interaction instance.

25. The system of claim 23, wherein the electronic device is the recognition platform.

28. The system of claim 1, the relevancy criteria comprising a set of requirements that must be satisfied to indicate that the at least one interaction object can be associated with the input media content.

29. The system of claim 1, the relevancy criteria comprising one or more media feature values, the one or more media feature values being at least one of a threshold and a range.

30. The system of claim 1, the relevancy criteria comprising a pointer to a second interaction object, the second interaction object stored in the interaction database and being a subset of the interaction objects.

31. The system of claim 1, the selecting at least one interaction object having relevancy criteria satisfied by the set of digital media features comprising including the set of digital media features in a query of the interaction database.

32. The system of claim 1, the relevancy criteria comprising at least one of logical connectors AND, OR, XOR, NOT, or NAND.

33. The system of claim 1, the relevancy criteria comprising position data.

34. The system of claim 1, wherein the namespace of the media features represents image features.

26. An interaction analysis system comprising:

a processor;

an interaction database storing interaction objects, each interaction object comprising relevancy criteria defined based on media features, each of the interaction objects stored in the interaction database comprising one or more fields populated when each of the interaction objects is instantiated, the one or more fields determined as a function of a namespace of the media features; and

an interaction analysis engine coupled with the interaction database and configured to;

receive input media content via a media input interface;

identify a set of digital media features derived from the input media content;

select at least one interaction object having relevancy criteria satisfied by the set of digital media features;

instantiate an interaction instance in memory according to the at least one interaction object; and

configure an electronic device to support an interaction with a user according to the interaction instance, without modifying the input media content.

27. An interaction analysis system comprising:

a processor;

an interaction database storing interaction objects, each interaction object comprising relevancy criteria defined based on media features, each of the interaction objects stored in the interaction database comprising one or more fields populated when each of the interaction objects is instantiated, the one or more fields determined as a function of a namespace of the media features; and

an interaction analysis engine coupled with the interaction database and configured to;

receive input media content via a media input interface;

identify a set of digital media features derived from the input media content;

select at least one interaction object having relevancy criteria satisfied by the set of digital media features;

instantiate an interaction instance in memory according to the at least one interaction object; and

configure a first electronic device to support a first interaction with a first user and a second electronic device to support a second interaction with a second user, according to the interaction instance.