Large scale video search using queries that define relationships between objects
First Claim
Patent Images
1. A relationship-based video server system, programmed to provide a platform for large scale video search based on relationships between objects in video segments, the platform executing search based on probabilistically discriminative objects and relationships, wherein the server system is configured to run processes that:
- for each video segment received for storage in a video database,analyze image data in the video segment to (1) recognize objects that are represented in the video segment and (2) identify relationships between the objects,determine, based on probabilities of occurrence for the recognized objects and identified relationships, at least one of (1) the most discriminative object or (2) the most discriminative relationship in the video segment,configure, based on at least one of the most discriminative object or the most discriminative relationship, a data structure that includes the recognized objects and identified relationships of the video segment;
store the data structure in the video database;
receive a video search query defining relationships between a plurality of objects;
determine, based on probabilities of occurrence for the objects and relationships of the video search query, at least one of the most discriminative object or the most discriminative relationship of the video search query;
identify, based on at least one of the most discriminative object or the most discriminative relationship of the video search query, a subset of the data structures to search for video segments;
determine, for each video segment represented in the subset of the data structures, a score based on similarity between the objects and the relationships in the video search query and the objects and the relationships associated with the video segment; and
display representations of video segments with the highest scores.
2 Assignments
0 Petitions
Accused Products
Abstract
One embodiment of the present solution is a relationship-based video server system. The server system is programmed to provide a platform for large scale video search based on relationships between objects in video segments, and the platform executes search based on probabilistically discriminative objects and relationships.
-
Citations
22 Claims
-
1. A relationship-based video server system, programmed to provide a platform for large scale video search based on relationships between objects in video segments, the platform executing search based on probabilistically discriminative objects and relationships, wherein the server system is configured to run processes that:
-
for each video segment received for storage in a video database, analyze image data in the video segment to (1) recognize objects that are represented in the video segment and (2) identify relationships between the objects, determine, based on probabilities of occurrence for the recognized objects and identified relationships, at least one of (1) the most discriminative object or (2) the most discriminative relationship in the video segment, configure, based on at least one of the most discriminative object or the most discriminative relationship, a data structure that includes the recognized objects and identified relationships of the video segment; store the data structure in the video database; receive a video search query defining relationships between a plurality of objects; determine, based on probabilities of occurrence for the objects and relationships of the video search query, at least one of the most discriminative object or the most discriminative relationship of the video search query; identify, based on at least one of the most discriminative object or the most discriminative relationship of the video search query, a subset of the data structures to search for video segments; determine, for each video segment represented in the subset of the data structures, a score based on similarity between the objects and the relationships in the video search query and the objects and the relationships associated with the video segment; and display representations of video segments with the highest scores. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)
-
-
20. A computer program product having a non-transitory computer-readable medium with computer code thereon for searching large scale video data, the computer program product comprising:
-
program code for receiving a video search query defining relationships between a plurality of objects; program code for determining, based on probabilities of occurrence for the objects and relationships of the video search query, at least one of the most discriminative object or the most discriminative relationship of the video search query; program code for accessing a video database that stores video segments; program code for identifying, based on at least one of the most discriminative object or the most discriminative relationship of the video search query, a subset of the video segments stored in a video database to be subject to search; program code for determining, for each video segment in the subset subject to search, a score based on similarity between the objects and the relationships in the video search query and the objects and the relationships associated with the video segment; and program code for displaying representations of video segments with the highest scores. - View Dependent Claims (21, 22)
-
Specification