Video search system and method of use

• US 8,892,572 B2
• Filed: 12/30/2011
• Issued: 11/18/2014
• Est. Priority Date: 12/30/2011
• Status: Active Grant

• Current Assignee: Cellco Partnership, Inc. (Verizon Communications Inc.)
• Original Assignee: Cellco Partnership, Inc. (Verizon Communications Inc.)
• Inventors: Sanjeev, Kumar
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): May, Robert F
• Application Number:
  US13/341,606
• Publication Number:
  US 20130173635A1
• Time in Patent Office: 1,054 Days
• Field of Search: 707/707, 707/754, 707/755, 707/E17.004, 707/E17.28
• US Class Current: 707/748
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335

Abstract

A system and method identifies a video file in response to a video based search query. A video imaging device in a mobile device captures a video file, and sends the video file to a search engine. A database associated with the search engine stores pre-indexed metadata of pre-indexed frames of video. A video analyzer separates the received video file into individual frames, analyzes the individual frames received from the mobile device by converting the individual frames into metadata, and compares the metadata to the pre-indexed metadata of the pre-indexed frames stored in the database. The video analyzer then sends a message containing information about the identified pre-existing video back to the mobile device based on the comparison of metadata. The metadata of the file and/or the metadata in the database may include one or more of pixel information, histogram information, image recognition information and audio information for each individual frame.

Claims

1. A system, comprising:

a network interface configured to enable a mobile device to capture a video file comprising a plurality of frames, and to send said video file through a mobile communication network to a server;

a processor configured to execute computer-implemented programs;

a storage device configured to store pre-indexed metadata of pre-indexed frames of pre-existing videos, wherein a plurality of pre-indexed metadata items are stored for each pre-indexed frame; and

a program associated with the server, wherein execution of the program by the processor configures the system to perform functions, including functions to;

separate the video file received from the mobile device into individual frames,

convert the individual frames into metadata for the video file, wherein each individual frame is converted into a plurality of metadata items corresponding to the plurality of pre-indexed metadata items, and

for each of a plurality of the pre-indexed frames of each of a plurality of the pre-existing videos, compare the plurality of metadata items for one individual frame of the video file to respective pre-indexed metadata items of the pre-indexed frame of the stored pre-existing video by;

generating a plurality of numeric values each numeric value representing an amount by which each of the metadata items for the individual frame matches a respective pre-indexed metadata item for the pre-indexed frame, and

combining the plurality of numeric values into a single value,

identify one of the pre-existing videos that matches the video file based on the plurality of single values corresponding to the respective plurality of pre-indexed frames of the plurality of existing videos,

selecting ones of the plurality of metadata items in the individual frame and the pre-indexed frame of the identified pre-existing video for which the numeric values exceed a predetermined value;

comparing next video frame following the one individual frame of the video file to a next pre-indexed video frame following the pre-indexed video frame in the identified pre-indexed video by;

generating a plurality of further numeric values, each further numeric value representing an amount by which each of the selected metadata items of the next video frame matches the respective metadata item in the next pre-indexed video frame; and

combining the plurality of further numeric values into a further single value; and

when a combination of the single value and the further single value is greater than a threshold send a message containing information about the identified pre-existing video based on the results of the comparison back to the mobile device.

2. The system of claim 1, wherein the plurality of metadata items for the video file and the plurality of pre-indexed metadata items include respective metadata items for pixel information, histogram information, image recognition information and audio information for each individual frame and for each pre-indexed frame for the pre-existing videos, respectively.

3. The system of claim 2, wherein the program assigns the respective numeric values to the comparisons of the plurality of metadata items for the individual frame of the video file to the respective plurality of pre-indexed metadata items for the pre-indexed frame based on an amount to which the respective metadata items of the individual frame for the pixel information, the histogram information, the image recognition information and the audio information match the respective pre-indexed metadata items for the pixel information, the histogram information, the image recognition information and the audio information of the pre-indexed frame.

4. The system of claim 3, wherein in assigning the respective numeric values for the comparisons of the plurality of metadata items of the video file and the respective plurality of pre-indexed metadata items, the relative values of the pixel information, the histogram information, the image recognition information and the audio information are weighted.

5. The system of claim 3, wherein in the comparing of the metadata, the single value having a greatest value identifies a best match.

6. A method, comprising the steps of:

receiving a video file comprising a plurality of frames from a mobile device,

separating the video file into a plurality of individual frames,

converting each individual frame into metadata for the received video file, wherein each individual frame is converted into a plurality of metadata items,

for each of a plurality of the pre-indexed frames of each of a plurality of the pre-indexed videos, comparing the plurality of metadata items for one individual frame of the received video file to respective pre-indexed metadata items of the pre-indexed frame of the pre-existing video to;

generate a plurality of numeric values, each numeric value representing an amount by which each of the metadata items for the individual frame matches the respective pre-indexed metadata item for the pre-indexed frame, and

combine the plurality of numeric values into a single value;

identify one of the plurality of pre-existing videos as corresponding to the received video file based on the comparison between the metadata items of the individual frame of the video file to the pre-indexed metadata items of the plurality of pre-indexed frames of the plurality of pre-existing videos,

select ones of the plurality of metadata items in the individual frame and the pre-indexed frame of the identified pre-existing video for which the numeric values exceed a predetermined value;

compare a next video frame following the individual frame of the video file to a next pre-indexed video frame following the pre-indexed video frame in the identified one of the pre-indexed video by;

generating a plurality of further numeric values, each further numeric value representing an amount by which each of the selected metadata items of the next video frame matches the respective metadata item in the next pre-indexed video frame; and

combining the plurality of further numeric values into a further single value; and

when a combination of the single value and the further single value is greater than a threshold, send a message containing information about the identified pre-existing video based on the results of the comparison back to the mobile device.

7. The method of claim 6, wherein the plurality of metadata items for the received video file and the plurality of pre-indexed metadata items include respective metadata items for pixel information, histogram information, image recognition information and audio information for each individual frame and for each pre-indexed frame for the pre-existing videos, respectively.

8. The method of claim 7, wherein in the step of comparing the plurality of metadata items of the received video file to the respective plurality of pre-indexed metadata items for the one of the pre-indexed frames, the respective numeric value is assigned for each comparison based on an amount by which the respective metadata items of the individual frame for the pixel information, the histogram information, the image recognition information and the audio information match the respective pre-indexed metadata items for the pixel information, the histogram information, the image recognition information and the audio information of the pre-indexed frame.

9. The method of claim 8, wherein in assigning the respective numeric values for the comparisons of the plurality of metadata items of the individual frame of the received video file and the respective plurality of pre-indexed metadata items for the pre-indexed frame, the relative values of the pixel information, the histogram information, the image recognition information and the audio information are weighted.

10. The method of claim 8, wherein in the step of comparing the metadata, the single value having a greatest value identifies a best match.

11. An article of manufacture comprising a non-transitory machine readable medium and executable instruction embodied in the medium for configuring a computer to perform the method of claim 6.

12. A system, comprising:

a video imaging device configured to capture a video file comprising a plurality of frames;

a mobile device coupled to the video imaging device configured to send said video file through a mobile communication network to a server;

a database configured to store pre-indexed metadata of pre-indexed frames of pre-existing videos, wherein a plurality of pre-indexed metadata items are stored for each pre-indexed frame;

a video analyzer associated with the server, configured to;

separate the video file received from the wireless communication device into individual frames,

convert the individual frames into metadata for the video file, wherein each individual frame is converted into a plurality of metadata items corresponding to the plurality of pre-indexed metadata items, and

for each of a plurality of the pre-indexed frames of each of a plurality of the pre-existing videos, compare the plurality of metadata items for one individual frame of the video file to the respective pre-indexed metadata items of the pre-indexed frame of the pre-existing video stored in the database by;

generating a plurality of numeric values each numeric value representing an amount by which each of the metadata items for the individual frame matches a respective pre-indexed metadata item for the pre-indexed frame, and

combining the plurality of numeric values into a single value;

identify one of the pre-existing videos as corresponding to the received video based on the plurality of single values corresponding to the plurality of pre-indexed frames of the plurality of existing videos,

select ones of the plurality of metadata items in the individual frame and the pre-indexed frame of the identified pre-existing video for which the numeric values exceed a predetermined value;

compare a next video frame following the individual frame of the video file to a next pre-indexed video frame following the pre-indexed video frame in the identified pre-indexed video by;

generating a plurality of further numeric values, each further numeric value representing an amount by which each of the selected metadata items of the next frame matches the respective metadata item in the next pre-indexed video frame; and

combining the plurality of further numeric values into a further single value; and

when a combination of the single value and the further single value is greater than a threshold, send a message containing information about the identified pre-existing video back to the mobile device.

13. The system of claim 12, wherein the plurality of metadata items for the video file and the plurality of pre-indexed metadata items include respective metadata items for pixel information, histogram information, image recognition information and audio information for each individual frame and for each pre-indexed frame for the pre-existing videos, respectively.

14. The system of claim 13, wherein the video analyzer assigns the respective numeric values to the comparisons of the plurality of metadata items for the individual frame of the video file to the plurality of pre-indexed metadata items for the pre-indexed frame based on an amount by which the respective metadata items of the individual frame for the pixel information, the histogram information, the image recognition information and the audio information match the respective pre-indexed metadata items for the pixel information, the histogram information, the image recognition information and the audio information of the pre-indexed frame.

15. The system of claim 14, wherein in assigning the respective numeric values for the comparisons of the plurality of metadata items for the individual frame of the video file and the respective plurality of pre-indexed metadata items, the relative values of the pixel information, the histogram information, the image recognition information and the audio information are weighted.

16. The system of claim 14, wherein in the comparing of the metadata, the single value having a greatest value identifies a best match.