Video visual and audio query
First Claim
Patent Images
1. A method, comprising:
- receiving, by a computing device, content comprising a media clip including audio and video;
analyzing, by the computing device, at least a portion of the content, said analyzing including preparing query frames of the content by selecting representative frames in the content and selecting at least one of the audio and video in the content to generate a signature for a search query;
generating, by the computing device, the signature, the signature including a set of bits identifying the content, wherein the set of bits are determined from a plurality of regions of a video frame of the representative frames in the content, a bit of the set of bits associated with a region of the plurality of regions, the bit being assigned a first value if a predetermined half of the region is brighter than the other half of the region, and the bit being assigned a second value if the predetermined half of the region is not brighter than the other half of the region; and
receiving video search results resulting from a comparison of the generated signature in a search query against a search corpus to identify the media clip in the search corpus.
3 Assignments
0 Petitions
Accused Products
Abstract
A video visual and audio query system for quickly identifying video within a large known corpus of videos being played on any screen or display. In one embodiment, the system can record via a mobile phone camera and microphone a live video clip from the TV and transcode it into a sequence of frame-signatures. The signatures representative of the clips can then be matched against the signatures of the TV content in a corpus across a network to identify the correct TV show or movie.
-
Citations
17 Claims
-
1. A method, comprising:
-
receiving, by a computing device, content comprising a media clip including audio and video; analyzing, by the computing device, at least a portion of the content, said analyzing including preparing query frames of the content by selecting representative frames in the content and selecting at least one of the audio and video in the content to generate a signature for a search query; generating, by the computing device, the signature, the signature including a set of bits identifying the content, wherein the set of bits are determined from a plurality of regions of a video frame of the representative frames in the content, a bit of the set of bits associated with a region of the plurality of regions, the bit being assigned a first value if a predetermined half of the region is brighter than the other half of the region, and the bit being assigned a second value if the predetermined half of the region is not brighter than the other half of the region; and receiving video search results resulting from a comparison of the generated signature in a search query against a search corpus to identify the media clip in the search corpus. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-readable storage device embodied with instructions to cause, when executed, a computing device to:
-
receive content comprising a query frame including video content and audio content; analyze the query frame; generate a signature of the video content of the query frame wherein each bit of the signature is associated with a brightness of a region of the frame and each bit is assigned a first value if a predetermined half of the region is brighter than the other half of the region, and each bit is assigned a second value if the predetermined half of the region is not brighter than the other half of the region; and use the signature of the audio content and the signature of the video content in a search query including randomly selecting multiple collections of a sub-set of bits in the signature of the video content and comparing the multiple collections against a search corpus.
-
-
11. A method comprising:
-
receiving, by a first computing device from a second computing device, a signature of a portion of a media clip, the signature representing a recording of a video playing on a screen of a different device which is captured by the second computing device, the signature generated from selected query frames of either or both of audio content or video content in the media clip; using, by a first computing device, the signature in a search query by comparing the signature against a search corpus to reduce a set of result candidates for the search query the comparing including a coarse direct comparison of the signature to ground truth signatures and, if needed, a second comparison to an index of the search corpus, the comparing to identify the video playing on the screen of the different device; and
outputting results of the search query from the first computing device to the second computing device. - View Dependent Claims (12, 13, 14, 15)
-
-
16. A computing device comprising a processor configured to:
-
receive a recording of a video playing on a screen of a different device, the recording including a video portion and an audio portion; determine boundaries of the screen in video portion of the recording by determining lines that have an intensity variance indicating one or more of the lines is not dark along an inward edge relative to the screen and an intensity variance indicating the one or more lines are dark along an external edge relative to the screen; extract the video within the determined boundaries from the recording; extract the audio portion from the recording; generate a signature for the audio portion and a bit wise signature for the video portion and use the signatures in one or more search queries to identify the recording.
-
-
17. A method comprising:
-
receiving a recording of a video playing on a screen of a different device, the recording including a video portion and an audio portion; determining boundaries of the screen in the video portion of the recording, the determining of the boundaries includes looking for lines that have an intensity variance indicating one or more of the lines is not dark along an inward edge relative to the screen and an intensity variance indicating the one or more lines are dark along an external edge relative to the screen; and extracting the video within the determined boundaries from the recording; extracting the audio portion from the recording; generating a signature for the extracted audio portion and a bit wise signature for the extracted video portion and using the extracted video in a search query identify the content as part of a broadcast media content.
-
Specification