Recognition-based content management, systems and methods

• US 11,042,607 B2
• Filed: 08/22/2014
• Issued: 06/22/2021
• Est. Priority Date: 08/23/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Wiacek, John, Soon-Shiong, Patrick
• Primary Examiner(s): Kim, Steven S
• Assistant Examiner(s): Fenstermacher, Jason B
• Application Number:
  US14/466,903
• Publication Number:
  US 20150058229A1
• Time in Patent Office: 2,496 Days
• Field of Search: None
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06Q 50/184 Intellectual property management
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

A content activation system is presented. A recognition engine is configured to obtain a descriptor set associated with an identified object represented within a digital representation. A content management module can be coupled with the recognition engine. The content management module can be configured to: obtain the descriptor set from the recognition engine, identify at least one content activation policy as a function of the descriptor set, identify a content set associated with an identified object and based on the descriptor set, and configure an output device to activate (e.g., release, render, present, interact, etc.) the content set according to the activation policy.

Claims

1. A content activation system comprising:

a database of activation policies indexed by descriptors generated from image data of different objects, the descriptors containing information about relationships between pixels in the image data;

at least one processor; and

at least one non-transitory computer readable memory for storing software instructions executable by the at least one processor to perform operations comprising;

obtaining a descriptor set associated with an object by executing an image recognition algorithm to recognize the object in a digital image containing the object, the descriptor set including at least one descriptor containing information about a relationship between pixels in the digital image;

identifying at least one content activation policy as a function of the descriptor set by traversing the activation policies stored in the database, the content activation policy comprising content activation criteria including one or more selected from the group consisting of;

payment requirements, location requirements, time requirements, and user identity;

identifying a content set associated with the object and based on the descriptor set; and

activating the content set in an output device according to content rendering rules of the activation policy.

2. The system of claim 1, wherein the descriptor set comprises image descriptors.

3. The system of claim 1, wherein the descriptor set comprises multi-modal descriptors.

4. The system of claim 1, wherein the descriptor set comprises a homogenous mix of descriptors.

5. The system of claim 1, wherein the at least one content activation policy comprises a license.

6. The system of claim 5, wherein the content rendering rules operate based on a license fee associated with the license.

7. The system of claim 1, wherein the content rendering rules operate based on recognized features in the digital image that are derived from the descriptor set.

8. The system of claim 1, wherein the content rendering rules unlock at least some of the content set as a function of the descriptor set.

9. The system of claim 1, wherein the content rendering rules operate based on an in-app purchase of content.

10. The system of claim 7, wherein the content set comprises purchased content.

11. The system of claim 1, wherein the at least one content activation policy operates as a function of at least one of tracked features and recognized features within the image.

12. The system of claim 1, wherein the activation criteria comprise a selection of content.

13. The system of claim 1, wherein the output device comprises at least one of the following:

a printer, a 3D printer, a tablet, a smart phone, a kiosk, an appliance, a television, a vehicle, a computer, a phone, a wearable headset, a wearable AR headset, a wearable camera, a wearable device, and a camera.

14. The system of claim 1, wherein the recognized object comprises a real-world object.

15. The system of claim 14, wherein the real-world object comprises at least one of the following:

a product, a toy, a vehicle, a machine, a living creature, an inanimate object, printed matter, a video, a video clip, a sound, a music, a light, a RFID, a magnet, a picture, a building, a human, an appliance, and an electronic device.

16. The system of claim 1, wherein the recognized object comprises a virtual object.

17. The system of claim 16, wherein the virtual object comprises at least one of the following:

a rendered graphic, a game object, a digital image, a digital video, an animation, an augmented reality object, a sound, and a 3D modeled object.

18. The system of claim 1, wherein the operations further comprising enforcing digital rights with respect to the content set according to a content rights rules set that operates as a function of the descriptor set.

19. The system of claim 18, wherein the content rights rules set includes rules for at least one of the following as a function of the descriptor set:

activation of the content set, deactivation of the content set, modification of the content sent, creation of the content set, localization of the content set, transactions related to the content set, and account interactions.

20. The system of claim 1, wherein the content set comprises at least one of the following:

image data, light data, RFID data, magnetic data, video data, game data, augmented reality data, audio data, kinesthetic data, tactile data, vibration data, texture data, application data, device command data, account data, shopping data, travel data, education data, sports data, news data, weather data, and medical data.

21. The system of claim 1, wherein the activation criteria define content release requirements.

22. The system of claim 1, wherein the activation criteria define content release conditions.

23. The system of claim 1, wherein the activation criteria comprise at least one of the following:

descriptor requirements, payment requirements and location requirements.

24. The system of claim 1, wherein the activation criteria include security clearance requirements.

25. The system of claim 24, wherein the security clearance requirements comprise one or more of the following:

passwords, credentials and identification information.

26. The system of claim 1 wherein the output device comprises at least one of the following:

a 3D printer, a tablet, a smart phone, a gaming console, a kiosk, an appliance, a television, a vehicle, a computer, a phone, a wearable headset, a wearable AR headset, a wearable camera, a wearable device, and a camera.

27. The system of claim 1, wherein the image recognition algorithm comprises an algorithm selected from the group consisting of SIFT (scale-invariant feature transform), FREAK (fast retina keypoint), DAISY, and FAST (features from accelerated segment test).

28. The system of claim 1, wherein the object comprises a three-dimensional object.

29. The system of claim 28, wherein the three-dimensional object comprises a card.

30. The system of claim 1, wherein the content activation policy includes a group of descriptors and a descriptor condition for identifying the content activation policy as a function of the descriptor set associated with the object, the descriptor condition requiring a specified portion of the group of descriptors to match the descriptor set according to a similarity measure.

31. The system of claim 1, wherein said identifying the at least one content activation policy includes traversing the activation policies stored in the database in real time with said obtaining the descriptor set.

32. The system of claim 1, wherein the activation policies are indexed in the database by the descriptors using a tree data structure or a k-nearest neighbors scheme.

33. A content activation system comprising:

a database of activation policies indexed by descriptors generated from image data of different objects, the descriptors containing information about relationships between pixels in the image data;

a recognition engine that obtains a descriptor set associated with an object by executing an image recognition algorithm to recognize the object in a digital image containing the object, the descriptor set including at least one descriptor containing information about a relationship between pixels in the digital image; and

a content management module that;

obtains the descriptor set from the recognition engine;

identifies at least one content activation policy as a function of the descriptor set by traversing the activation policies stored in the database, the content activation policy comprising content activation criteria including one or more selected from the group consisting of;

payment requirements, location requirements, time requirements, and user identity;

identifies a content set associated with the object and based on the descriptor set; and

activates the content set in an output device according to content rendering rules of the activation policy.

34. The system of claim 33, wherein the content rendering rules operate based on recognized features in the digital image that are derived from the descriptor set.

35. The system of claim 33, wherein the content rendering rules unlock at least some of the content set as a function of the descriptor set.

36. The system of claim 33, wherein the content rendering rules operate based on an in-app purchase of content.

37. The system of claim 34, wherein the content set comprises purchased content.

38. A non-transitory computer readable memory storing software instructions executable by a processor to perform operations comprising:

obtaining a descriptor set associated with an object by executing an image recognition algorithm to recognize the object in a digital image containing the object, the descriptor set including at least one descriptor containing information about a relationship between pixels in the digital image;

identifying at least one content activation policy as a function of the descriptor set by traversing activation policies stored in a database of activation policies indexed by descriptors generated from image data of different objects, the descriptors containing information about relationships between pixels in the image data, the content activation policy comprising content activation criteria including one or more selected from the group consisting of;

payment requirements, location requirements, time requirements, and user identity;

identifying a content set associated with the object and based on the descriptor set; and

activating the content set in an output device according to content rendering rules of the activation policy.