Visual hash tags via trending recognition activities, systems and methods

• US 9,544,655 B2
• Filed: 12/09/2014
• Issued: 01/10/2017
• Est. Priority Date: 12/13/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Soon-Shiong, Patrick
• Primary Examiner(s): Sheleheda, James R
• Application Number:
  US14/564,448
• Publication Number:
  US 20150172778A1
• Time in Patent Office: 763 Days
• Field of Search: 725/51, 725/56, 725/109, 725/112
• US Class Current: 1/1
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4823 using a channel name
• CPC Class Codes: H04N 21/4826 using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A system and method of treating image data as a visual hash tag are presented. A device is able to subscribe to a content channel, possibly a channel that provide social media information, by the act of recognizing related objects without requiring a user to annotate content with unnatural hash tags.

Claims

1. A digital content subscription system comprising:

a digital data processing module executable on a processor according to software instruction stored in a memory and configured to;

obtain a digital representation of an object;

identify a set of features in the digital representation;

derive a set of descriptors corresponding to the set of features, each descriptor of the set of descriptors existing within a descriptor space; and

determine a plurality of weights, the plurality of weights comprising a weight of each descriptor in the set of descriptors based on a determination of relevance of a feature in the set features, the feature corresponding to the each descriptor;

a channel database configured to store channel identifiers indexed with respect to related descriptors within the descriptor space, each channel identifier representative of a digital content channel; and

a subscription module executable on a processor according to software instruction stored in a memory, the subscription module coupled with the channel database and the digital processing module, and configured to;

obtain the set of descriptors and the plurality of weights from the digital data processing module; and

generate a list of channel identifiers from the channel database that have related descriptors near the set of descriptors within the descriptor space, wherein the generating utilizes the plurality of weights to search for related descriptors corresponding to only a subset of descriptors in the set of descriptors, the subset of descriptors corresponding to features in the set of features that are more relevant than other features in the set of features.

2. The system of claim 1, where the digital representation comprise at least one of the following data modalities:

image data, audio data, video data, biometric data, sensor data, and weather data.

3. The system of claim 1, wherein the set of descriptors comprises at least one the following types of descriptors:

image descriptors, invariant descriptors, metadata descriptors, facial descriptors, and audio descriptors.

4. The system of claim 1, wherein the set of descriptors comprises at least one of the following types of image descriptors:

SIFT descriptors, FREAK descriptors, FAST descriptors, DAISY descriptors, BRISK descriptors, Harris descriptors, and MSER descriptors.

5. The system of claim 1, wherein the set of descriptors within the descriptor space comprises at least one cluster in the descriptor space.

6. The system of claim 5, wherein the cluster comprises cluster sub-structure.

7. The system of claim 1, wherein the channel database comprises a remote database service relative to the device.

8. The system of claim 1, wherein the channel database comprises an in-memory data structure within a memory of the device.

9. The system of claim 8, wherein the in-memory data structure comprises a nearest neighbor channel look up module within the descriptor space.

10. The system of claim 9, wherein the nearest neighbor channel look up module is configured to utilize at least one of the following:

a spill tree and a k-d tree.

11. The system of claim 1, wherein the device comprises at least one of the following:

a mobile phone, a vehicle, a tablet, a phablet, a gaming device, a kiosk, an appliance, a set top box, a server, a computer, a virtual reality device, a projected reality device, a medical device, a point of sales device, and a vending machine.

12. The system of claim 1, wherein the interaction includes rendering content obtained from the at least one digital content channel on the device.

13. The system of claim 1, wherein the interaction includes submitting content to the at least one digital content channel from the device.

14. The system of claim 1, wherein the interaction includes initiating a transaction with respect to the at least one digital content channel.

15. The system of claim 14, wherein the transaction includes at least one of the following:

a financial transaction, an account transaction, and a protocol transaction.

16. The system of claim 1, wherein the at least one digital content channel comprises a broadcast channel.

17. The system of claim 1, wherein the at least one digital content channel comprises a multicast channel.

18. The system of claim 1, wherein the at least one digital content channel comprises a content feed.

19. The system of claim 18, wherein the content feed comprise at least one of the following:

a medical feed, a social feed, a advertising feed, a curated feed, a programmed feed, and a peer-to-peer feed.

20. The system of claim 1, wherein the list of list of channel identifiers fails to satisfy user channel selection criteria.

21. The system of claim 20, wherein the list of channel identifiers has less than two channel identifiers.

22. The system of claim 20, wherein the subscription module is further configured to:

generate a new channel identifier;

instantiate a new digital content channel referenced by the new channel identifier; and

store the new channel identifier within the channel database according to the set of descriptors.

23. The system of claim 1, wherein the subscription module is further configured to register the device with a content subscription service.