Methods and systems for interacting with viewers of video content

• US 8,676,030 B2
• Filed: 01/08/2009
• Issued: 03/18/2014
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: Novafora, Inc.
• Original Assignee: Novafora, Inc.
• Inventors: Rakib, Shlomo Selim, Devictor, Gilles Bruno Marie
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Tran, William
• Application Number:
  US12/350,883
• Publication Number:
  US 20100008643A1
• Time in Patent Office: 1,895 Days
• Field of Search: 386/248, 386/262
• US Class Current: 386/248
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06V 20/48 Matching video sequences

Abstract

A viewer interaction system and method receives video data from a video source and identifies object content in the received video data. A video signal is generated that represents the video data. That video signal is communicated to a display device which displays a video image representative of the video signal. The system and method monitor user interaction with the displayed video image and selects an interactive activity based on the user'"'"'s interaction with the displayed video image and the identified object content in the received video data.

Claims

1. A method of automatically selecting an interactive activity to be presented to viewers of a video program, said interactive activity not being transmitted in time synchronization with said video program, said video program not embedded with interactive activity synchronization information to be used in said method, said method comprising:

receiving video data from a video source, said video data being free from embedded interactive activity synchronization information to be used in said method;

automatically identifying specific video object content in the received video data by using a processor to determine a plurality of features in said video data;

at least some of said plurality of features being selected for their ability to persist when an image from said video data is rotated, presented with altered resolution, or presented with different lighting;

abstracting said video data into a three dimensional structure of features comprising two spatial dimensions and one time dimension;

determining the time order in which various feature groups appear;

assigning at least some of said feature groups to objects;

generating a video signal representing the video data;

communicating the video signal to a display device, wherein the display device displays a video image representative of the video signal;

monitoring user interaction with the displayed video image; and

selecting an interactive activity based on the user'"'"'s interaction with the displayed video image and based on the automatically identified object content in the video data.

2. The method of claim 1, further comprising offering the selected interactive activity to the user.

3. The method of claim 1, wherein monitoring user interaction with the displayed video image includes monitoring user selection of an object in the displayed video image.

4. The method of claim 1, wherein monitoring user interaction with the displayed video image includes monitoring user control of playback of the displayed video image.

5. The method of claim 4, wherein the user control of playback includes pausing playback of the displayed video image, fast forwarding playback of the displayed video image, or rewinding playback of the displayed video image.

6. The method of claim 1, wherein the selected interactive activity is a question or an interactive user survey associated with the received video data.

7. The method of claim 1, wherein the selected interactive activity is an interactive user selection among a plurality of possible advertisements to be presented in the video image.

8. The method of claim 1, wherein automatically identifying object content in the received video data includes retrieving metadata associated with the video data, wherein the metadata identifies objects and associated object positions within the video data.

9. The method of claim 1, wherein said features are selected from the group consisting of Harris corner detectors, Scale invariant feature transform (SIFT), Speed up robust features (SURF), motion vectors, direction of spatial-temporal edges, distribution of color, coefficients of decompositions of the pixels in a known dictionary of wavelets or curvelets, or specific objects known a priori.

10. A method of automatically selecting an interactive activity to be presented to viewers of a video program, said interactive activity not being transmitted in time synchronization with said video program, said video program not embedded with interactive activity synchronization information to be used in said method, said method comprising:

receiving a video program from a video source, said video data being free from embedded interactive activity synchronization information to be used in said method;

displaying said video program to a plurality of current viewers;

automatically identifying specific video objects in said video program by using a processor to determine a plurality of features in said video program;

at least some of said plurality of features being selected for their ability to persist when an image from said video program is rotated, presented with altered resolution, or presented with different lighting;

abstracting said video program into a three dimensional structure of features comprising two spatial dimensions and one time dimension;

determining the time order in which various feature groups appear;

assigning at least some of said feature groups to objects;

communicating a question to current viewers of a video program pertaining to said objects;

receiving responses to the question from a plurality of the current viewers during presentation of the video program;

analyzing the received responses substantially in real time;

displaying a summary of the received responses to current viewers; and

determining a remainder of the video program content based on the received responses.

11. The method of claim 10, wherein the video program is a live video program.

12. The method of claim 11, wherein determining a remainder of the video program content includes determining content to display during the remaining live portion of the video program.

13. The method of claim 10, wherein communicating a question to current viewers of a video program includes communicating the question to viewers who have expressed an interest in interacting with the video program.

14. The method of claim 10, wherein communicating a question to current viewers of a video program includes communicating the question to viewers who have interacted with the video program.

15. The method of claim 10, wherein the question communicated to current viewers of the video program is associated with a plurality of contestants appearing in the video program.

16. The method of claim 10, wherein the question communicated to current viewers of the video program is associated with a desired ending to the video program.

17. The method of claim 10, wherein the question communicated to current viewers of the video program is associated with a desired advertisement to be presented during the video program.

18. The method of claim 10, wherein said features are selected from the group consisting of Harris corner detectors, Scale invariant feature transform (SIFT), Speed up robust features (SURF), motion vectors, direction of spatial-temporal edges, distribution of color, coefficients of decompositions of the pixels in a known dictionary of wavelets or curvelets, or specific objects known a priori.

19. An apparatus for automatically selecting an interactive activity to be presented to viewers of a video program, said interactive activity not being transmitted in time synchronization with said video program, said video program not embedded with interactive activity synchronization information to be used by said apparatus for the automatic selection, said apparatus comprising:

a communication module configured to receive video data from a video source and configured to receive information regarding object content in the received video data;

said video data being free from embedded interactive activity synchronization information to be used by said apparatus to receive information regarding said object content for said automatic selection;

a processor or video processor configured for identifying specific video objects in said video data by determining a plurality of features in said video program;

at least some of said plurality of features being selected for their ability to persist when a video image from said video data is rotated, presented with altered resolution, or presented with different lighting;

abstracting said video data into a three dimensional structure of features comprising two spatial dimensions and one time dimension;

determining the time order in which various feature groups appear;

and assigning at least some of said feature groups to objects;

the communication module further configured to receive interactive activity data from the video source;

a user interface module configured to monitor a user'"'"'s interaction with a displayed video image associated with the received video data; and

a processor coupled to the communication module and the user interface module, the processor configured to select an interactive activity based on the user'"'"'s interaction with the displayed video image, the automatically identified objects in said video image, and based on the received information regarding object content in the received video data.

20. The apparatus of claim 19, further comprising an interactive video module configured to manage interaction between the user of the apparatus and the video source.

21. The apparatus of claim 19, wherein the processor is further configured to offer the selected interactive activity to the user.

22. The apparatus of claim 19, wherein the user interface module monitors the user'"'"'s interaction with the displayed video image by identifying selection of an automatically identified object in the displayed video image or by identifying user control of playback of the displayed video image.

23. The apparatus of claim 19, wherein said features are selected from the group consisting of Harris corner detectors, Scale invariant feature transform (SIFT), Speed up robust features (SURF), motion vectors, direction of spatial-temporal edges, distribution of color, coefficients of decompositions of the pixels in a known dictionary of wavelets or curvelets, or specific objects known a priori.