Multiple resolution video network with context based control

• US 9,467,647 B2
• Filed: 07/17/2008
• Issued: 10/11/2016
• Est. Priority Date: 07/17/2007
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Cai, Yang
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US12/669,680
• Publication Number:
  US 20100231734A1
• Time in Patent Office: 3,008 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/12 Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system. The system includes a computing device configured for communication with a plurality of multiple resolution cameras and with a display device. The computing device includes a composite image module. The composite image module is configured for receiving composite information from at least one of the multiple resolution cameras, and for generating a composite image based on the composite information.

Claims

1. A system, comprising:

a plurality of multiple resolution cameras, wherein each of the plurality of multiple resolution cameras has a single type of image sensor and is configured for;

communicably receiving resolution selection instructions;

detecting a presence of a predefined object of interest within a single frame of a captured image;

generating location information describing a location of the predefined object of interest within the captured image;

generating a first image at a first resolution comprising at least a first portion of the captured image;

generating a second image at a second resolution comprising at least a second portion of the captured image, the second portion including the detected predefined object of interest, wherein the first resolution is lower than the second resolution;

communicably sending composite information, wherein the composite information comprises the first image, the second image and the location information; and

a computing device communicably connected to the plurality of multiple resolution cameras, wherein the computing device is configured for;

receiving a first composite information comprising the composite information from one of the plurality of multiple resolution cameras;

generating user focus information;

generating a display image including a multiple resolution composite image based on the first composite information, wherein each portion of the multiple resolution composite image is generated at the same frame rate; and

generating the resolution selection instructions in response to the user focus information; and

communicably sending the resolution selection instructions to one or more of the plurality of multiple resolution cameras; and

a display device communicably connected to the computing device, the display device being configured to display at least the display image.

2. The system of claim 1, further comprising an eye tracking device communicably connected to the computing device, wherein the eye tracking device communicates eye tracking information to the computing device, the computing device being further configured to generate the user focus information by associating the eye tracking information with a first location on the display device.

3. The system of claim 1, further comprising an eye tracking device communicably connected to the computing device, wherein the eye tracking device communicates eye tracking information to the computing device, the computing device being further configured to generate the user focus information by associating the eye tracking information with a first image on the display device.

4. The system of claim 1, further comprising an eye tracking device communicably connected to the computing device, wherein the eye tracking device communicates eye tracking information to the computing device, the computing device being further configured to generate the user focus information by associating the eye tracking information with a first one of the plurality of multiple resolution cameras.

5. The system of claim 1, wherein the display image further comprises additional low resolution images generated by one or more of the plurality of multiple resolution cameras.

10. The system of claim 1, wherein the display image further comprises additional low resolution images generated by one or more of the plurality of multiple resolution cameras.

6. A method, implemented at least in part by a computing device, the method comprising:

utilizing a plurality of multiple resolution cameras having a single type of image sensor to capture, at an instance in time;

an image with a first one of the plurality of multiple resolution cameras at a first resolution; and

a plurality of other images with the other of the plurality of multiple resolution cameras at a second resolution, wherein the first resolution is different from the second resolution;

generating, at the first one of the plurality of multiple resolution cameras, composite information by;

creating a first image based on the captured image at a resolution that is lower than the first resolution;

detecting a presence of a predefined object of interest within a single frame of the captured image;

creating object of interest information that identifies a location of the predefined object of interest relative to the first image; and

creating a second image at the first resolution, wherein the second image comprises a portion of the captured image that is associated with the predefined object of interest; and

generating, at the computing device, user focus information;

receiving, at the computing device, the composite information sent from the first one of the plurality of multiple resolution cameras; and

generating, at the computing device, resolution selection instructions based on the user focus information;

sending the resolution selection instructions to the plurality of multiple resolution cameras; and

generating, at the computing device, a display image which includes a multiple resolution composite image based on the received composite information, wherein each portion of the composite image is generated at the same frame rate; and

sending the display image to a display device.

7. The method of claim 6, wherein generating the composite image comprises superimposing a portion of the second image on the first image based on the defined location information.

8. The method of claim 6, wherein generating the composite image comprises:

deleting a portion of the first image; and

filling the deleted portion with a portion of the second image.

9. The method of claim 6, further comprising receiving, at the computing device, eye tracking information from an eye tracking device;

wherein generating user focus information further comprises associating the eye tracking information with a location on the display device.