Methods for using extracted feature vectors to perform an action associated with a work identifier

• US 10,063,936 B1
• Filed: 08/29/2017
• Issued: 08/28/2018
• Est. Priority Date: 09/14/2000
• Status: Expired due to Fees

• Current Assignee: Network-1 Technologies, Inc.
• Original Assignee: Network-1 Technologies, Inc.
• Inventors: Cox, Ingemar J.
• Primary Examiner(s): Chen, Cai
• Application Number:
  US15/689,528
• Time in Patent Office: 364 Days
• Field of Search: None
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0277 : Online advertisement
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 40/12 : Accounting
• CPC Class Codes: H04H 2201/90 : characterised by the use of signatures
• CPC Class Codes: H04H 60/37 : for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/58 : of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 : of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 61/256 : NAT traversal
• CPC Class Codes: H04L 65/611 : for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/75 : Media network packet handling
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04M 1/72469 : for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04N 21/23418 : involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2407 : Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/25841 : involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/43 : Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/4394 : involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 : involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47815 : Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8586 : by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A method comprising the steps of:

• • receiving, from a user electronic device at a computer system, one or more extracted feature vectors that are compact representations of one or more video frames of a video displayed at the user device; identifying, at the computer system, one or more products related to the one or more video frames based at least in part on an approximate nearest neighbor search of an electronic database of reference feature vectors corresponding to one or more reference works that identifies any match within a threshold of at least one of the reference feature vectors to the one or more extracted feature vectors but does not guarantee to identify the closest match to the one or more extracted feature vectors; providing, by the computer system to the user electronic device, the identified one or more products along with action information associated with the identified one or more products, wherein the action information is related to an action to be performed at the user electronic device; receiving, at the computer system from the user electronic device, a selection of one of the identified one or more products; and providing, to the electronic device, machine readable instructions for performance of the action associated with the selected one of the identified one or more products.

Claims

1. A computer implemented method comprising:

(a) obtaining, by a first computer system, a first electronic media work, wherein the first electronic media work comprises at least a first audio file;

(b) assigning, by the first computer system, a work identifier to the first electronic media work;

(c) extracting, by the first computer system, from at least a portion of the at least the first audio file, using, at least in part, a frequency-based decomposition of the portion produced by a Fourier frequency decomposition, a first electronic feature vector comprising a first compact representation of the first electronic media work;

(d) sending, from the first computer system to a second computer system, the first electronic feature vector and the work identifier;

(e) receiving, by the first computer system from the second computer system, first action information and the work identifier, wherein;

(1) the second computer system is operatively connected to a database comprising;

(A) first electronic data related to identification of one or more reference electronic works; and

(B) second electronic data related to action information comprising an action to perform corresponding to each of the one or more reference electronic works; and

(2) the first action information is determined by the second computer system by using a sub-linear search of the first electronic data that identifies a match to the first electronic feature vector within a threshold but does not guarantee to identify the closest match to the first electronic feature vector, wherein the first electronic data is organized using a clustering technique to eliminate a subset of the first electronic data to be searched by performing a comparison of cluster features;

(f) associating, by the first computer system, the first action information with the work identifier associated with the first electronic media work; and

(g) performing, by the first computer system, the first action information in association with the first electronic media work.

2. The method of claim 1, wherein the first action information comprises information related to a URL.

3. The method of claim 1, wherein an action associated with the first action information further comprises redirecting a request to an associate website by a network address translation.

4. The method of claim 1, wherein an action associated with the first action information further comprises automatically dialing a telephone number.

5. The method of claim 1, wherein the first electronic media work further comprises at least a first video file.

6. The method of claim 1, wherein the first action information is determined by the second computer system further based on a temporal sequence of the extracted first electronic feature vector.

7. The method of claim 1, further comprising the steps of:

(h) generating, at the first computer system, machine-readable instructions to be executed by a first electronic media device in performing the determined first action information; and

(i) providing, from the first computer system to the first electronic media device, the machine-readable instructions in response to a query received from the first electronic media device related to the first electronic media work.

8. The method of claim 7, wherein the machine-readable instructions are further based on device information relating to the first electronic media device at which the first electronic media work is rendered.

9. The method of claim 8, wherein the device information comprises a geographic location of the first electronic media device.

10. The method of claim 1, further comprising the steps of:

(h) obtaining, by the first computer system, a second electronic media work, wherein the second electronic media work comprises at least a second audio file;

(i) assigning, by the first computer system, a second work identifier to the second electronic media work;

(j) extracting, by the first computer system, from at least a second portion of the at least the second audio file, using, at least in part, the frequency-based decomposition of the second portion produced by a Fourier frequency decomposition, a second electronic feature vector comprising a second compact representation of the second electronic media work;

(k) sending, from the first computer system to a second computer system, the second electronic feature vector and the second work identifier;

(l) receiving, by the first computer system from the second computer system, second action information, wherein;

(1) the second computer system is operatively connected to the database; and

(2) the second action information is determined by the second computer system by using a sub-linear search of the first electronic data that identifies a match to the second electronic feature vector within a threshold but does not guarantee to identify the closest match to the first electronic feature vector, wherein the first electronic data is organized using a clustering technique to eliminate a subset of the first electronic data to be searched by performing a comparison of cluster features and wherein none of the one or more reference electronic works is identified to match the second electronic feature vector;

(m) associating, by the first computer system, the second action information with the second work identifier associated with the second electronic media work; and

(n) performing, by the first computer system, the second action information in association with the second electronic media work.