SYSTEM AND METHOD FOR CONTEXUALLY INTERPRETING IMAGE SEQUENCES

• US 20130007872A1
• Filed: 06/28/2011
• Published: 01/03/2013
• Est. Priority Date: 06/28/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: RAMANATHAN, Sri, TREVATHAN, Matthew B., HARIHARAN, Rajaraman, SUBBIAN, Karthik
• Granted Patent: US 8,904,517 B2
• Time in Patent Office: Days
• US Class Current: 726/17
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 8/43 Checking; Contextual analysis
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04H 20/65 Arrangements characterised by transmission systems for broadcast
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/76 Arrangements characterised by transmission systems other than for broadcast, e.g. the Internet for broadcast H04H20/65
• CPC Class Codes: H04L 41/0618 based on the physical or logical position
• CPC Class Codes: H04L 9/10 with particular housing, physical features or manual controls
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 21/4758 for providing answers, e.g. voting
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A system and method for contextually interpreting image sequences are provided. The method comprises receiving video from one or more video sources, and generating one or more questions associated with one or more portions of the video based on at least one user-defined objective. The method further comprises sending the one or more portions of the video and the one or more questions to one or more assistants, receiving one or more answers to the one or more questions from the one or more assistants, and determining a contextual interpretation of the video based on the one or more answers and the video.

Claims

1. A method, comprising:

receiving video from one or more video sources;

generating one or more questions associated with one or more portions of the video based on at least one user-defined objective;

sending the one or more portions of the video and the one or more questions to one or more assistants; and

receiving one or more answers to the one or more questions from the one or more assistants; and

determining a contextual interpretation of the video based on the one or more answers and the video.

2. The method of claim 1, further comprising analyzing the video based on the at least one user-defined objective.

3. The method of claim 2, further comprising:

determining whether the analyzing of the video requires assistance based on the at least one user-defined objective; and

identifying the one or more portions of the video that requires assistance based on the at least one user-defined objective when the analyzing of the video requires assistance.

4. The method of claim 1, further comprising sending the one or more portions of the video and the one or more questions to the one or more assistants based on at least one of geographic constraints, bandwidth constraints, and security constraints.

5. The method of claim 4, wherein the security constraints specify that the one or more assistants be at least one of authenticated by an authentication and authorization mechanism, and verified by a Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA).

6. The method of claim 1, wherein the one or more answers are at least one of categorical, quantitative, and arbitrary, in nature.

7. The method of claim 1, further comprising changing an order of the one or more portions of the video before the sending of the one or more portions of the video such that the one or more assistants receiving the one or more portions of the video avoid receiving consecutive portions of the video in a same video sequence.

8. The method of claim 1, further comprising sending the one or more portions of the video and the one or more questions to the one or more assistants which are at least one of geographically-distributed amongst one or more locations, and specifically-trained to answer the one or more questions based on the one or more portions of the video.

9. The method of claim 1, wherein the contextual interpretation of the video is at least one of dynamic based on prospective video and prospective answers associated with the prospective video, static, and deferred for the prospective video and the prospective answers.

10. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports a computer infrastructure operable to perform the steps of claim 1.

11. The method of claim 1, wherein the steps of claim 1 are provided by a service provider on a subscription, advertising, and/or fee basis.

12. A system implemented in hardware, comprising:

a computer infrastructure operable to;

receive video from one or more video sources;

determine one or more questions associated with one or more Application Video Packets (AVPs) of the video that require user assistance based on at least one of user-defined criteria and predetermined criteria;

send the one or more AVPs and the one or more questions to one or more processor stations;

collate one or more answers to the one or more questions from the one or more processor stations; and

determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video.

13. The system of claim 12, wherein the computer infrastructure is further operable to parse the video based on the user-defined criteria.

14. The system of claim 13, wherein the computer infrastructure is further operable to:

determine whether the parsing of the video requires user assistance based on the user-defined criteria; and

determine the one or more AVPs based on the user-defined criteria when the parsing of the video requires user assistance.

15. The system of claim 12, wherein the computer infrastructure is further operable to send the one or more AVPs and the one or more questions to the one or more processor stations based on at least one of geographic constraints, bandwidth constraints, and security constraints.

16. The system of claim 15, wherein the security constraints specify that the one or more processor stations be at least one of authenticated by an authentication and authorization mechanism, and verified by a Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA).

17. The system of claim 12, wherein the one or more answers are at least one of categorical, quantitative, and arbitrary, in nature.

18. The system of claim 12, wherein the computer infrastructure is further operable to change an order of the one or more AVPs before the sending of the one or more AVPs such that the one or more processor stations receiving the one or more AVPs avoid receiving consecutive AVPs in a same video sequence.

19. The system of claim 12, wherein the computer infrastructure is further operable to send the one or more AVPs and the one or more questions to the one or more processor stations which are at least one of geographically-distributed amongst one or more locations, and operated by one or more users specifically-trained to answer the one or more questions based on the one or more AVPs.

20. The system of claim 12, wherein the contextual interpretation of the video is at least one of dynamic based on prospective video and prospective answers associated with the prospective video, static, and deferred for the prospective video and the prospective answers.

21. A computer program product comprising a computer usable storage medium having readable program code embodied in the storage medium, the computer program product comprises at least one component operable to:

receive video from one or more video sources;

generate one or more questions associated with one or more portions of the video based on at least one user-defined objective;

send the one or more portions of the video and the one or more questions to one or more assistants; and

receive one or more answers to the one or more questions from the one or more assistants; and

determine a contextual interpretation of the video based on the one or more answers and the video.

22. The computer program product of claim 21, wherein the at least one component is further operable to:

analyze the video based on the at least one user-defined objective;

determine whether the analyzing of the video requires assistance based on the at least one user-defined objective; and

identify the one or more portions of the video which requires assistance based on the at least one user-defined objective when the analyzing of the video requires assistance.

23. A method of contextually interpreting image sequences, comprising:

providing a computer infrastructure, being operable to;

receive one or more Application Video Packets (AVPs) of video that requires user assistance and one or more questions associated with the one or more AVPs, the one or more questions determined based on at least one of user-defined criteria and predetermined criteria;

determine one or more answers to the one or more questions based on the one or more AVPs; and

send the one or more answers, to determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video.

24. The method of claim 23, wherein the steps of claim 23 are provided by a processor station which is at least one of geographically-distributed amongst one or more locations, and operated by at least one user specifically-trained to answer the one or more questions based on the one or more AVPs.

25. A computer system for contextually interpreting image sequences, the system comprising:

a CPU, a computer readable memory and a computer readable storage media;

first program instructions to receive video from one or more video sources;

second program instructions to parse the video based on user-defined criteria;

third program instructions to determine whether the parsing of the video requires user assistance based on the user-defined criteria; and

fourth program instructions to determine one or more Application Video Packets (AVPs) of the video that requires user assistance and one or more questions associated with the one or more AVPs based on at least one of the user-defined criteria and predetermined criteria when the parsing of the video requires user assistance;

fifth program instructions to send the one or more AVPs and the one or more questions to one or more processor stations; and

sixth program instructions to collate one or more answers to the one or more questions from the one or more processor stations, to determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video,

wherein the first, second, third, fourth, fifth, and sixth program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.