Content recognition for targeting video advertisements

• US 8,689,251 B1
• Filed: 09/14/2012
• Issued: 04/01/2014
• Est. Priority Date: 04/18/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Heath, Taliver Brooks
• Primary Examiner(s): EKPO, NNENNA NGOZI
• Application Number:
  US13/620,191
• Time in Patent Office: 564 Days
• Field of Search: 725/32, 704/235
• US Class Current: 725/32
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 7/165 Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

Methods, systems, and apparatus, including computer program products, for providing advertisements. A plurality of advertisement targeting criteria is determined from a video stream or file. A first advertisement targeting criterion of the advertisement targeting criteria is associated with a first time position in the video. A second advertisement targeting criterion of the advertisement targeting criteria is associated with a second time position in the video. One or more first advertisements are provided, for presentation at or after the first time position in the video, based on the first advertisement targeting criterion. One or more second advertisements are provided, for presentation at or after the second time position in the video, based on the second advertisement targeting criterion.

Claims

1. A computer-implemented method, the method comprising:

performing object recognition on visual content of a video to identify one or more first time positions in the video where a respective recognized object appears in the visual content;

performing speech recognition on the video to identify one or more second time positions in the video where respective recognized speech occurs;

determining, by one or more processors a respective confidence level for one or more pairs of corresponding first and second time positions wherein the respective confidence level is based on a determination of whether the recognized speech of the second time position of the pair describes the respective recognized object of the first time position of the pair;

associating a respective advertising targeting criterion with one or more of the one or more first time positions based on the respective confidence level and the respective recognized object of the one or more first time position; and

providing an advertisement and the video based on, at least, the respective advertising targeting criterion.

2. The method of claim 1, further comprising:

providing the video and the respective advertisement targeting criterion to a user device;

receiving from the user device a request for advertisements, the request including the respective advertisement targeting criterion;

identifying the advertisement based on the respective advertisement targeting criterion included in the request; and

providing the advertisement to the user device.

3. The method of claim 1 wherein performing speech recognition on the video, further comprises applying speech recognition to audio content of the video to identify speech in the video.

4. The method of claim 1 wherein performing speech recognition on the video, further comprises analyzing closed captions data for the video to identify speech in the video.

5. The method of claim 1 wherein performing object recognition on visual content of a video, further comprises applying one or more of the following to the visual content of the video:

a hidden Markov model, a neural network, appearance-based object recognition, and object recognition based on local features.

6. The method of claim 1 wherein the respective advertising targeting criterion is a word or a phrase that describes the respective recognized object of the first time period.

7. A system comprising:

a non-transitory computer-readable medium having instructions stored thereon; and

data processing apparatus programmed to;

perform object recognition on visual content of a video to identify one or more first time positions in the video where a respective recognized object appears in the visual content;

perform speech recognition on the video to identify one or more second time positions in the video where respective recognized speech occurs;

determine a respective confidence level for one or more pairs of corresponding first and second time positions wherein the respective confidence level is based on a determination of whether the recognized speech of the second time position of the pair describes the respective recognized object of the first time position of the pair;

associate a respective advertising targeting criterion with one or more of the one or more first time positions based on the respective confidence level and the respective recognized object of the one or more first time position; and

provide an advertisement and the video based on, at least, the respective advertising targeting criterion.

8. The system of claim 7, wherein the data processing apparatus programmed to:

provide the video and the respective advertisement targeting criterion to a user device;

receive from the user device a request for advertisements, the request including the respective advertisement targeting criterion;

identify the advertisement based on the respective advertisement targeting criterion included in the request; and

provide the advertisement to the user device.

9. The system of claim 7, wherein the data processing apparatus programmed to perform speech recognition on the video, further comprises applying speech recognition to audio content of the video to identify speech in the video.

10. The system of claim 7 wherein the data processing apparatus programmed to perform speech recognition on the video, further comprises analyzing closed captions data for the video to identify speech in the video.

11. The system of claim 7 wherein the data processing apparatus programmed to perform object recognition on visual content of a video, further comprises applying one or more of the following to the visual content of the video:

a hidden Markov model, a neural network, appearance-based object recognition, and object recognition based on local features.

12. The system of claim 7 wherein the respective advertising targeting criterion is a word or a phrase that describes the respective recognized object of the first time period.

13. A storage device having instructions stored thereon that, when executed by data processing apparatus, are operable to cause the data processing apparatus to perform operations comprising:

performing object recognition on visual content of a video to identify one or more first time positions in the video where a respective recognized object appears in the visual content;

performing speech recognition on the video to identify one or more second time positions in the video where respective recognized speech occurs;

determining a respective confidence level for one or more pairs of corresponding first and second time positions wherein the respective confidence level is based on a determination of whether the recognized speech of the second time position of the pair describes the respective recognized object of the first time position of the pair;

associating a respective advertising targeting criterion with one or more of the one or more first time positions based on the respective confidence level and the respective recognized object of the one or more first time position; and

providing an advertisement and the video based on, at least, the respective advertising targeting criterion.

14. The storage device of claim 13, wherein the operations further comprise:

providing the video and the respective advertisement targeting criterion to a user device;

receiving from the user device a request for advertisements, the request including the respective advertisement targeting criterion;

identifying the advertisement based on the respective advertisement targeting criterion included in the request; and

providing the advertisement to the user device.

15. The storage device of claim 13 wherein performing speech recognition on the video, further comprises applying speech recognition to audio content of the video to identify speech in the video.

16. The storage device of claim 13 wherein performing speech recognition on the video, further comprises analyzing closed captions data for the video to identify speech in the video.

17. The storage device of claim 13 wherein performing object recognition on visual content of a video, further comprises applying one or more of the following to the visual content of the video:

a hidden Markov model, a neural network, appearance-based object recognition, and object recognition based on local features.

18. The storage device of claim 13 wherein the respective advertising targeting criterion is a word or a phrase that describes the respective recognized object of the first time period.